0

My code goes something like this:

echo "$result" | xargs -P50 -I@ bash -c '{ printf "@ $(curl --write-out "%{http_code}" -L -s --output /dev/null @)\n"; }'

Where result is containing a list similar to :

https://example.com/somepath
https://example.com/somepath&quot
https://example.com/anotherpath?par1&par4=gg

The expected output [URL STATUS-CODE]

https://example.com/somepath 200
https://example.com/somepath&quot 200
https://example.com/anotherpath?par1&par4=gg 200

The issue is when running this snippet, i get the error bash: quot: No such file or directory which is separated by the andsign & and i cant seem to work around it

I tried wrapping @ in single/double quotes but nothing worked

Raywando
  • 427
  • 1
  • 4
  • 7
  • 1
    Please [edit] your question and show how you set the value of `result`. Maybe it would be easier without putting the multi-line data in a single variable and splitting it using `xargs`. Avoiding subshells and using `printf` with a proper format string and modified quoting may also help. Explain in your question what you want to achieve. Based on guessing from the code and example data you somehow get a list of URLs and want to display the reply code of the web server when you try to access the URL using `curl`. Do you want to further process the reply code later? – Bodo Mar 08 '21 at 12:29
  • Use `printf` correctly. `printf FORMAT [ARGUMENT]` like @Bodo said, then check [this](https://unix.stackexchange.com/questions/156008/is-it-possible-to-use-find-exec-sh-c-safely) for safe usage of `bash -c` (the question is about `find -exec sh -c` , but the very same applies here. Also, you might get rid of the superfluous `{` and `}`. – pLumo Mar 08 '21 at 12:36
  • @Bodo The `result` variable is set in different ways in the code, I showed how the final value would be, and yes you are right, i want to get the status code of each URL and i will need both urls along with the status code for later in the code. I shared a sample of `result` variable and the expected output. – Raywando Mar 08 '21 at 12:38
  • @Raywando Explaining what is already clear from the question does not help much. Please [edit] your question to add clarification or requested information. Don't use comments for this purpose. "The `result` variable is set in different ways" doesn't clarify anything. An example why it may be useful to know how you construct the `result` value: If you generate the URLs in a loop it might be easier to call `curl` in the same loop. When you generate multi-line output where every line is one value, it might be better to feed this into a `while IFS= read -r variable` loop. – Bodo Mar 08 '21 at 12:47

1 Answers1

1

Embedding the replacement in command strings introduces command injection vulnerabilities. Which is what you are seeing in action: instead of being (just) used as a string argument to curl, the content of result is being parsed as code.

To prevent that, you may use (focusing on the main issue only; other improvements are possible):

printf '%s\n' "$result" | xargs -P50 -I@ bash -c 'printf "%s %s\n" "$1" \
  "$(curl --write-out "%{http_code}" -L -s --output /dev/null "$1")"' mybash @
fra-san
  • 9,931
  • 2
  • 21
  • 42
  • 1
    Maybe nitpicking, but as `http_code` will be always a digit, you should use `%d` for it. You could also add `$1` to write-out and completely omit `printf`, but not sure about the safety implications of it .... – pLumo Mar 08 '21 at 12:42
  • Thanks! @fra-san, but whats `mybash` here? – Raywando Mar 08 '21 at 12:43
  • 1
    @Raywando `mybash` is an arbitrary string used as the command name when a shell is invoked with the `-c` option. See the description of `-c` in [the Bash manual](https://www.gnu.org/software/bash/manual/html_node/Invoking-Bash.html#Invoking-Bash). – fra-san Mar 08 '21 at 12:52
  • @fra-san I suggest to add the explanation about `mybash` to the answer. – Bodo Mar 08 '21 at 12:57
  • maybe you could do `--write-out "%{url_effective} %{http_code}\n"` and skip that second printf altogether. (maybe even the bash -c command_string too) –  Mar 08 '21 at 13:01
  • @pLumo Not sure about `--write-out`, I'd leave it to others. Also not sure whether `%d` would be better, given that for the shell it is just text anyway and there seems to be no need for number formatting here (alignment, padding, ...). Please feel free to directly edit my answer if you see room for improvement. – fra-san Mar 08 '21 at 13:07
  • `xargs <<< "$result" -n 1 curl --write-out "%{url_effective} %{http_code}\n" -L -s --output /dev/null` –  Mar 08 '21 at 13:07
  • 1
    That is what I thought too @bac0n, but `url_effective` is maybe not what OP wants, it may differ from the input. – pLumo Mar 08 '21 at 13:15
  • or ... `-I @ curl --write-out "@ %{http_code}\n" ... @` –  Mar 08 '21 at 13:19