0

I am working on an automated pull request check via GitHub actions and I want to preserve some data from a command's stderr output between jobs.

For this, I need to write the stderr to an artifact file, but before that I need to also remove some control chars from it via sed, otherwise I end up with something like:

\x1b[31mFound 344 errors!\x1b[39;49m

I then want the main command to return the same exit code in order to fail the check and prevent merging of the pull request.

I can probably take care of a subset of what needs doing, but am unable to take care of all 3 together together (sed > file write > stderr). If it makes it easier, I am okay with writing both stdout and stderr to the file as well.

Open to suggestions to do this differently.

AdminBee
  • 21,637
  • 21
  • 47
  • 71
Abhishek Jain
  • 119
  • 4
  • 1
    Are those color/attribute codes? Normally there is an option to disable them in the command that generates them. – Quasímodo Nov 28 '20 at 18:19
  • 1
    Please show us a minimal reproducible example of your script that we can use to test any solutions. – terdon Nov 28 '20 at 18:24
  • @Quasímodo indeed, they are. The command I'm referring to comes from another project so it's out of my control. – Abhishek Jain Nov 28 '20 at 18:45
  • @terdon So far I am able to ensure that both streams get copied to a file as well as be displayed on terminal via `tee`: `somecommand 2&>1 | tee ./artifact.txt`. However, doing so loses the original exit status. – Abhishek Jain Nov 28 '20 at 18:56
  • @terdon pardon me. I did not know about the `pipefail` shell option which essentially solves my problem. The final command being: `set -o pipefail ; somecommand 2&>1 | tee ./artifact.txt` ; set +o pipefail` – Abhishek Jain Nov 28 '20 at 18:58
  • @AbhishekJain I'm sorry but I simply have no idea what you mean since you haven't shown us any code to demonstrate the issue. As I said before, please [edit] your question and add a small example that reproduces the problem, and explain how you would want that example to work. – terdon Nov 28 '20 at 19:05
  • Yes, minimal example would be really useful now. – Eduardo Trápani Nov 28 '20 at 19:22
  • Apologies for not being able to provide a minimal repro-able code, mainly because there is none. I am trying to construct a command from scratch based on my requirements. Fortunately I have been able to (mostly) solve it. Hopefully, the answer I posted will bring more context to you guys :) – Abhishek Jain Nov 28 '20 at 19:40
  • It seems like a XY problem, as your sample output hints that you want to remove the color control characters of a `git` output thgough a `sed`, which indicates that you use a specific configuration for that characters to survive a pipe pass (like color "always" and not just "auto"). – thanasisp Nov 29 '20 at 00:12

2 Answers2

1

I was able to solve it using a combination of shell redirection operators, the tee utility and pipefail shell option:

set -o pipefail; somecommand 2&>1 | tee ./artifact.txt; set +o pipefail

Basically this:

  1. sets the pipeline's return status as the value of the last command to exit with a non-zero status, or zero if all commands exit successfully.
  2. duplicates the stderr from stdout with 2&>1 (or its shorthand |&) which is then passed as stdin to tee with the
  3. invokes tee which copies its stdin to stdout while making a copy in artifact.txt.
  4. restores the original pipe setting.

I had to live with both stdout and stderr being copied to the file because, apparently, Bash has no shorthand syntax that allows piping only stderr to a second command.

Other refs:

Abhishek Jain
  • 119
  • 4
  • The question is about filtering the stderr of a process through another command . All you do here is to write both stdout and stderr into the terminal and the file. Also I don't see the reason you set pipefail here. – thanasisp Nov 29 '20 at 00:13
  • For correctness you should run this in a subshell (surround it in parentheses) rather than executing `set +o pipefail` at the end, since you don't know the original value. – xeruf Nov 25 '21 at 11:18
0

In bash you can run:

$ somecommand 2>&1 | tee ./artifact.txt;echo "${PIPESTATUS[@]}"

And get the exit codes all along the pipe.

Eduardo Trápani
  • 12,032
  • 1
  • 18
  • 35
  • The exit status of this pipeline will be that of the last command. It will lose the exit status of `somecommand`. – Abhishek Jain Nov 28 '20 at 19:10
  • Please do check the last command `echo "${PIPESTATUS[@]}"` show the array of all the exit codes in the pipe. – Eduardo Trápani Nov 28 '20 at 19:20
  • I don't want to show the exit codes, I, _basically_, want the entire thing to return the exit code of `somecommand`. – Abhishek Jain Nov 28 '20 at 19:38
  • Of course can do whatever you want with ${PIPESTATUS[@]} We haven't seen a minimal example of what you wan to do, so I added the code to print them, you can obviously do whatever you want with it, including using the first element of the array to do an 'exit`. You really have to provide code if you expect people to tailor their answers to your needs. – Eduardo Trápani Nov 28 '20 at 20:00
  • Thank you for helping. I am an amateur at shell scripting so apologies for not framing the question properly . – Abhishek Jain Nov 28 '20 at 20:11