3

I want know if I can use variables inside the url used by wget? I'm trying to recreate in BASH: I currently have:

WK_ARTIST="$(echo $ARTIST | sed 's/\ /\_/g' )"
WK_ALBUM="$(echo $ALBUM | sed 's/\ /\_/g' )"
echo "$WK_ARTIST"
echo "$WK_ALBUM"

wget https://en.wikipedia.org/wiki/File:"$WK_ARTIST"_-_"$WK_ALBUM".jpg

I get

Warning: wildcards not supported in HTTP.
--2021-06-10 07:24:20--  https://en.wikipedia.org/wiki/File:Lamb_Of_God_-_%1B[1;36mLamb_Of_God.jpg

Can this be done.

Freddy
  • 25,172
  • 1
  • 21
  • 60
Dark_Stoner
  • 137
  • 1
  • 10
  • 2
    Pleae add `echo "ARTIST=$ARTIST, ALBUM=$ALBUM"` and its result to your question – roaima Jun 10 '21 at 10:21
  • Does this answer your question? [Why does my shell script choke on whitespace or other special characters?](https://unix.stackexchange.com/questions/131766/why-does-my-shell-script-choke-on-whitespace-or-other-special-characters) – roaima Jun 10 '21 at 10:28
  • @roaima quoting will solve it, of course, but whitespace isn't the issue here. It will also fail if the values have no whitespace since the shell is looking for `$ARTIST_` here instead of `$ARTIST`. – terdon Jun 10 '21 at 10:39
  • @terdon oh yes, missed that one – roaima Jun 10 '21 at 11:09
  • @terdon In the current installment of the question, quoting is not an issue (wel, not in the `wget` bit anyway). However, it would be good to know what exactly is in those variables and where that data comes from. It looks as if the `ARTIST` variable may possible contain junk at the start. – Kusalananda Jun 16 '21 at 09:33
  • @Kusalananda yes, see the comments under my answer. – terdon Jun 16 '21 at 09:38
  • By inserting a line `set -x` into your script or running it like `bash -x yourscript` you should be able to debug similar problems. – Bodo Jul 13 '21 at 17:26

2 Answers2

4
WK_ARTIST="$(echo "$ARTIST" | sed 's/ /_/g' )"
WK_ALBUM="$(echo "$ALBUM" | sed 's/ /_/g' )"
echo "$WK_ARTIST"
echo "$WK_ALBUM"

wget "https://en.wikipedia.org/wiki/File:${WK_ARTIST}_-_$WK_ALBUM.jpg"

  1. Shell variables should be quoted whenever you use them (including inside command substitution). The sole exception is when you know with absolute certainty that you actually want the variable's value to be subject to shell's whitespace splitting.

  2. $WK_ARTIST needs to be disambiguated from the _ by surrounding it with curly braces ({}) so that the shell doesn't interpret the variable name as $WK_ARTIST_ (which probably does not exist so evaluates as the empty string).

    This is necessary for WK_ARTIST but not later in the string for WK_ALBUM because _ is a valid character in a shell variable name (just like A-Z, a-z. and 0-9 for characters after the first), while . is not.

    i.e. FOO, FOO_, FOO1, FOO_1, and FOOBAR are all valid and completely distinct variable names.

Or, using bash's built-in search & replace feature instead of echo ... | sed:

WK_ARTIST="${WK_ARTIST// /_}"
WK_ALBUM="${WK_ALBUM// /_}"
echo "$WK_ARTIST"
echo "$WK_ALBUM"

wget "https://en.wikipedia.org/wiki/File:${WK_ARTIST}_-_$WK_ALBUM.jpg"

Or, if you're going to use the URL multiple times (or just to make the script easier to read), it's worth doing something like:

...
URL="https://en.wikipedia.org/wiki/File:${WK_ARTIST}_-_$WK_ALBUM.jpg"
...
wget "$URL"
cas
  • 1
  • 7
  • 119
  • 185
1

You can use variables, yes, but you need to quote them so the shell can know where the variable name ends:

$ ARTIST=foo
$ ALBUM=bar
$ echo $ARTIST_-_$ALBUM.jpg
-_bar.jpg

So, the shell is trying to expand a variable called $ARTIST_, which doesn't exist. But, if you quote:

$ echo "$ARTIST"_-_"$ALBUM".jpg
foo_-_bar.jpg

So, try:

wget https://en.wikipedia.org/wiki/File:"$ARTIST"_-_"$ALBUM".jpg
terdon
  • 234,489
  • 66
  • 447
  • 667
  • Thanks but I've already tried that and still get the same error – Dark_Stoner Jun 10 '21 at 10:49
  • 2
    @Dark_Stoner that seems unlikely, I just tried and could download the file perfectly well with `ARTIST=Lamb_of_God; ALBUM=Lamb_of_God; wget https://en.wikipedia.org/wiki/File:"$ARTIST"_-_"$ALBUM".jpg`. Please edit your question and show us how you define your variables as Roaima asked. – terdon Jun 10 '21 at 10:51
  • I see my error is due to a color code injected by grep! – Dark_Stoner Jun 10 '21 at 11:46
  • My error is that I didn't read the complete error. A color code got introduced into the code because I used grep to filter out the tuxi results. – Dark_Stoner Jun 10 '21 at 11:51
  • Ah yes, now that you show the full error that can be seen. So, just don't use color with `grep`. Either use `/bin/grep` instead of `grep` (many Linux systems have `grep` aliased to `grep --color=always`) or explicitly turn the color off with `grep --color=no`. – terdon Jun 10 '21 at 12:02