4

We want to implement a workflow that includes having YAML configuration files where we wish to change some of the contents before actually using them, and we are looking for a good command line tool to do this.

This can be as simple as setting a target-specific value (like an URL) in an existing node, but preferably would be able to add or modify whole blocks of the YAML file. In a perfect world, I would like something as powerful as the axis in XSLT.

We will use this with Maven in Docker files, where we prefer Debian-based images, but not a strict requirement.

Kusalananda
  • 320,670
  • 36
  • 633
  • 936
Thorbjørn Ravn Andersen
  • 1,031
  • 11
  • 19

2 Answers2

8

Mike Farah’s yq is my go-to tool for YAML manipulation. For example, using a Kubernetes namespace definition file:

  • extract the namespace name:

    yq read deploy/namespace.yaml metadata.name

  • add a label:

    yq write deploy/namespace.yaml metadata.labels.name demo

    (this will replace an existing node, or add a new one if there isn’t one already)

The -i option can be used to update files in-place.

You won’t find everything that XSLT tools can typically do (or jq for JSON), for example you won’t find tests on sub-expressions so you can’t (AFAIK) write the equivalent of “update this sub-node on nodes which have this other sub-node with this value”.

Stephen Kitt
  • 411,918
  • 54
  • 1,065
  • 1,164
1

Andrey Kislyuk's yq (which is a different yq from the one mentioned by Stephen) is a Python wrapper around the well known JSON processor jq.

As it's a wrapper around jq, it allows you to do any processing on YAML files that jq would normally do on JSON files. It also adds the ability to do in-place editing of files.

The yq utility can also output JSON, TOML, and XML. By installing yq, you get additional command line tools for working with these other types of structured document formats (tomlq and xq for TOML and XML, respectively).

The yq utility is available on OpenBSD, FreeBSD, and NetBSD as the yq package and on macOS as the Homebrew package python-yq. On GNU systems, you may install yq using pip3 (pip3 install yq, or variations thereof). See also Andrey's installation instructions for yq.

Kusalananda
  • 320,670
  • 36
  • 633
  • 936