jq - add objects from file into json array

Question

I want to add an array with elements and value into an existing json file using jq.

I already have a file (input.json) with

{
  "id": 9,
  "version": 0,
  "lastUpdTs": 1532371267968,
  "name": "Training"
}

I want to add this into another groups array into this json (orig.json)

[
  {
    "name": "JAYS",
    "sourceConnection": {
      "name": "ORACLE_connection",
      "connectionType": "JDBC",
      "commProtocol": "JDBC"
    },
    "checked": true,
    "newlyAdded": false,
    "id": null,
    "groups": [],
    "displayName": "SCOTT",
    "defaultLevel": "MANAGED"
  }
]

The end result should look like

[
  {
    "name": "JAYS",
    "sourceConnection": {
      "name": "ORACLE_connection",
      "connectionType": "JDBC",
      "commProtocol": "JDBC"
    },
    "checked": true,
    "newlyAdded": false,
    "id": null,
    "groups": [
      {
        "id": 9,
        "version": 0,
        "lastUpdTs": 1532371267968,
        "name": "Training"
      }
    ],
    "displayName": "SCOTT",
    "defaultLevel": "MANAGED"
  }
]

I know how to add elements into an array, but not sure how to pass in from a file.

jq '.[].groups += [{"INPUT": "HERE"}]' ./orig.json

Inian · Accepted Answer · 2018-08-07T08:02:55.200

jq has a flag for feeding actual JSON contents with its --argjson flag. What you need to do is, store the content of the first JSON file in a variable in jq's context and update it in the second JSON

jq --argjson groupInfo "$(<input.json)" '.[].groups += [$groupInfo]' orig.json

The part "$(<input.json)" is shell re-direction construct to output the contents of the file given and with the argument to --argjson it is stored in the variable groupInfo. Now you add it to the groups array in the actual filter part.

Putting it in another way, the above solution is equivalent of doing this

jq --argjson groupInfo '{"id": 9,"version": 0,"lastUpdTs": 1532371267968,"name": "Training" }' \
   '.[].groups += [$groupInfo]' orig.json

score 14 · Answer 2 · answered Aug 07 '18 at 07:54

This is the exact case that the input function is for:

input and inputs [...] read from the same sources (e.g., stdin, files named on the command-line) as jq itself. These two builtins, and jq’s own reading actions, can be interleaved with each other.

That is, jq reads an object/value in from the file and executes the pipeline on it, and anywhere input appears the next input is read in and is used as the result of the function.

That means you can do:

jq '.[].groups += [input]' orig.json input.json

with exactly the command you've written already, plus input as the value. The input expression will evaluate to the (first) object read from the next file in the argument list, in this case the entire contents of input.json.

If you have multiple items to insert you can use inputs instead with the same meaning. It will apply across a single or multiple files from the command line equally, and [inputs] represents all the file bodies as an array.

It's also possible to interleave things to process multiple orig files, each with one companion file inserted, but separating the outputs would be a hassle.

But this probably doesn't let me incorporate non-json files. With `--arg` I can do: `jq --arg "$(<1.txt)" '{file: $file}'`. — x-yuri, Jul 05 '19 at 09:57
Also, the file might not be on disk. You might have retrieved the file from the internet to a variable (`file=$(curl ...)`). In which case, there's probably no way to use `input` function. `<(echo "$file")` doesn't work. — x-yuri, Jul 05 '19 at 11:02
@MichaelHomer how if we want to do changes in input.json before merging? — ImranRazaKhan, Nov 21 '19 at 15:11

score 3 · Answer 3 · answered Aug 04 '20 at 00:01

3

New versions of jq support slurpfile

This achieves the result you are looking for: jq '.[].groups += $inputs' orig.json --slurpfile inputs input.json

Tested in jq 1.6

answered Aug 04 '20 at 00:01

Efren

178
1
8

score 0 · Answer 4 · answered Jan 28 '21 at 06:42

Use dot as placeholder

echo '{
  "id": 9,
  "version": 0,
  "lastUpdTs": 1532371267968,
  "name": "Training"
}' | jq '[
  {
    "name": "JAYS",
    "sourceConnection": {
      "name": "ORACLE_connection",
      "connectionType": "JDBC",
      "commProtocol": "JDBC"
    },
    "checked": true,
    "newlyAdded": false,
    "id": null,
    "groups": [.],
    "displayName": "SCOTT",
    "defaultLevel": "MANAGED"
  }
]'

Output:

[
  {
    "name": "JAYS",
    "sourceConnection": {
      "name": "ORACLE_connection",
      "connectionType": "JDBC",
      "commProtocol": "JDBC"
    },
    "checked": true,
    "newlyAdded": false,
    "id": null,
    "groups": [
      {
        "id": 9,
        "version": 0,
        "lastUpdTs": 1532371267968,
        "name": "Training"
      }
    ],
    "displayName": "SCOTT",
    "defaultLevel": "MANAGED"
  }
]

jq --version
jq-1.6

jq - add objects from file into json array

4 Answers4

Linked