Bash throwing errors when parsing a JSON field with empty lines to env var: Invalid environment variable format ' '

Question

I'm saving a JSON field, .body, to an ENV variable. The content of this field will always be changing with multiple strings and characters, it's the body message of a Pull Request in GitHub. So markdown will also get passed:

Below would be an accurate example extracted from .body

#Title

## SubTitle

*date*

1. Item 1
2. Item 2
3. Item 3

When I parse the JSON using jq and -r for its raw content:

echo "PR_BODY=$(jq -r '.body' $HOME/pr.json)" >> $GITHUB_ENV

I get errors from GitHub actions:

For reference, my plan is to save .body data into a markdown file like so after managing to save it to an env variable:

      run: |
        ed changelog.md <<'END_ED'
        1i

        ${{ env.PR_BODY }}

        .
        wq
        END_ED
        echo >> changelog.md

How can I go about extracting the data in .body without empty spaces throwing errors? Perhaps an ENV variable is not the way to go about this?

EDIT: Here is an original raw JSON in the .body:

"body": "Manually configuring ports for DEV, CAT/QA environments. Migrating from managed service to self-serve \"GCP resource\". \r\n\r\n- [x] DEV\r\n- [x] CAT/QA\r\n- [ ] PROD\r\n\r\n```\r\n# Allow healthcheck on ports 80, 443\r\nresource \"google_compute_firewall\" \"allow-healthcheck\" {\r\n  name = \"${format(\"%s\",\"${var.gcp_resource_name}-${var.gcp_env}-fw-allow-healthcheck\")}\"\r\n  network = \"${google_compute_network.vpc.name}\"\r\n  allow {\r\n    protocol = \"tcp\"\r\n    ports    = [\"80\",\"443\"]\r\n  }\r\n  source_ranges = [\"2.2.0.0/16\", \"1.1.0.0/22\"]\r\n}\r\n```\r\nConfiguring for bug, link here:\r\n[Link To Terraform provider update](https://www.terraform.io/)"

Please see my new update! Ive added the key and value for `.body` — peso, Jun 16 '22 at 18:29
Reference - [How to parse markdown text from JSON and appending it to a file without ruining the markdown formatting?](https://unix.stackexchange.com/a/705478/100397) — roaima, Jun 16 '22 at 18:31
is `$GITHUB_ENV` a shell script, or something else? `echo "PR_BODY=$(jq -r '.body' $HOME/pr.json)" >> $GITHUB_ENV` will run the command substitution right then and there, so if the output of `jq` has multiple lines, they'll go into multiple lines in the output. That might break the expected syntax at the other end. — ilkkachu, Jun 16 '22 at 18:33
Also I'm not sure why you're saying it's Bash throwing errors? That doesn't look like any error Bash would give, and you're getting it from some Github... thing. — ilkkachu, Jun 16 '22 at 18:37
@ilkkachu, `$GITHUB_ENV` is an environment file path native to Github actions. This is where someone can save env variables they they might use later down the road for another job. The expected syntax at the other is a simple mark down file. I demonstrated above how I would pass it to the markdown file. — peso, Jun 16 '22 at 18:38
@andres, I have no idea what the syntax of "an environment file path native to Github actions" is. Is it interpreted by a shell? Or is it something else, possibly just some simpler key-value format (like `/etc/environment` is on Linuxes)? If you want to shove a multi-line string in there, the first thing to do would be to find out what syntax it supports. — ilkkachu, Jun 16 '22 at 18:43
Your raw JSON is not valid. Run it though, for instance, [this](https://jsonlint.com/) to check. I am not endorsing this particular site, just picked it out of a long list of validators. — doneal24, Jun 16 '22 at 18:59

score 2 · Accepted Answer · answered Jun 16 '22 at 19:01

2

Based on "Workflow commands for GitHub Actions", it looks like the syntax for multiline values in $GITHUB_ENV would be something like:

PR_BODY<<EOF
multiline string
here
...
EOF

With the value coming from the command substitution you had there, you could produce that from the shell with e.g.:

printf "PR_BODY<<EOF\n%s\nEOF\n" "$(jq -r '.body' $HOME/pr.json)"

(You most likely want to use printf instead of echo there, since it makes it easier to insert newlines while not mangling the data from the command substitution. See Why is printf better than echo? for discussion.)

answered Jun 16 '22 at 19:01

ilkkachu

133,243
15
236
397

1

@roaima, well... what exactly reads that file and how was the thing I tried to ask the original poster. They didn't answer and I'm still not sure, but at least the example I linked to there is _not_ shell syntax. (There's no `=` there, and you can't assign directly from a here-doc a variable like that anyway.) – ilkkachu Jun 16 '22 at 22:42
Sorry, you're right it's not shell syntax. Deleting... – roaima Jun 17 '22 at 06:15
Did you mean to add << $GITHUB_ENV at the end of that 'printf' line? – Lee Meador Nov 29 '22 at 00:10
@LeeMeador, first, definitely not `<<`, as that would send input to `printf` from a here-doc, and `printf` won't read it. Output redirection with `>> "$GITHUB_ENV"` would come to question if we want to put the output directly to the file. But I don't think that's the main issue here, and it might make sense to view the output first. If it's only a one-time job, one might then just copy it and paste to the file via an editor or whatever. – ilkkachu Nov 29 '22 at 11:25

Bash throwing errors when parsing a JSON field with empty lines to env var: Invalid environment variable format ' '

1 Answers1