Bash string eval unclear

Question

Consider the following in a bash script:

STOP_SEARCH_STR="'""for $TASKNAME""'"
echo $STOP_SEARCH_STR               # prints 'for Our Special Task_4'
echo "$STOP_SEARCH_STR"             # prints 'for Our Special Task_4'
echo "grep -F "$STOP_SEARCH_STR" "$TEMP"/draft" # prints grep -F 'for Our Special Task_4' /tmp/draft
echo "$(grep -F "$STOP_SEARCH_STR" "$TEMP"/draft)"  # prints nothing!
echo $("grep -F "$STOP_SEARCH_STR" "$TEMP"/draft")  # prints nothing!

And the following content in the $TEMP/draft text file:

2019-11-21 08:13:58,825 Task started: 'Our Special Task_4' of type 'Our - Special Task' 
2019-11-21 08:14:10,509 Task ended: 'Success' for Our Special Task_4 -- Elapsed time: 11.0 seconds

If I manually type the command grep -F 'for Our Special Task_4' /tmp/draft, I receive the second line in the draft text file:

2019-11-21 08:14:10,509 Task ended: 'Success' for Our Special Task_4 -- Elapsed time: 11.0 seconds

But the the last 2 commands above (inside a bash script) print nothing!

Any idea why?

@Jesse_b I am pretty sure you are right, but at the moment I have no clue how to fix this. Can you elaborate on your comment? — datsb, Nov 23 '19 at 18:34
@datsb Remove the single-quotes from `STOP_SEARCH_STR`. They are being treated as part of the string to search for. — Gordon Davisson, Nov 23 '19 at 18:36
@GordonDavisson I did that. It makes matters worse: The command fails even when typed manually. It doesn't like the spaces in `for Our Special Task_4`. — datsb, Nov 23 '19 at 18:50
That last command _ought_ to give you an error! There is no command named literally "`grep -F `". — Kusalananda, Nov 23 '19 at 19:20
@Kusalananda You are correct. See my comment to the answer below. — datsb, Nov 23 '19 at 20:20

ilkkachu · Accepted Answer · 2019-11-23T20:41:50.340

3

Quotes aren't special after a variable is expanded.

Assuming TASKNAME contains Our Special Task_4, then STOP_SEARCH_STR is set to 'for Our Special Task_4', including those quotes. That's what your echo shows.

When you run grep -F "$STOP_SEARCH_STR" "$TEMP"/draft, you're giving grep the string 'for Our Special Task_4' to look for. That string doesn't exist in the file, so no lines match, and grep prints nothing.

Remove the quotes, you're not looking for them:

task="Our Special Task_4"
str="for $task"
grep -F "$str" "$TEMP/draft"

Note: you do need the double quotes around the variable expansion "$str" or "$STOP_SEARCH_STR" to prevent word-splitting, see Why does my shell script choke on whitespace or other special characters?

When you manually type the command grep -F 'for Our Special Task_4' /tmp/draft, the quotes are part of the shell syntax and effectively remove the special meaning of the spaces within them. This is different from quotes inside a variable which act like ordinary characters.

As an aside: 1) echo "$(somecmd)" is usually redundant, you could just run somecmd directly. 2) Here: echo $("grep -F "$STOP_SEARCH_STR" "$TEMP"/draft"), because of how the quotes surround the whitespace, you're trying to run a command called grep -F 'for. You should probably get an error for that:

$ echo $("grep -F "$STOP_SEARCH_STR" "$TEMP"/draft")
bash: grep -F 'for: command not found

edited Nov 23 '19 at 20:41

answered Nov 23 '19 at 19:09

ilkkachu

133,243
15
236
397

I was too quick to celebrate: I removed the quotes but now the command effectively becomes `grep -F for Our Special Task_4 /tmp/draft` which finds all lines with the word `for` in them but complains "No such file or directory" for the rest of the words in the search string (Our, Special, Task_4). How do I add quotes around the "string with spaces to be searched for" so that work as if the command is typed manually? – datsb Nov 23 '19 at 20:19
@datsb, what's the exact command you used? – ilkkachu Nov 23 '19 at 20:20
The exact command is just as I originally posted, except that I removed the quotes. It's basically the same as `grep -F "$str" "$TEMP/draft"` that you suggested. I am now totally confused, although you explanation is excellent. Note that the challenge here is the spaces in the search string. – datsb Nov 23 '19 at 20:28
@datsb, yes, I know. As long as you have the double-quotes around the variable expansion, it should work fine. They're the quotes that **are** necessary. (I should have been explicit about that from the start.) – ilkkachu Nov 23 '19 at 20:41
@datsb "Always double quote any expansion" is a simple rule of thumb that you could follow, and which will be correct more often that not. – Kusalananda Nov 23 '19 at 20:45
see also: https://mywiki.wooledge.org/WordSplitting – ilkkachu Nov 23 '19 at 20:45

score 1 · Answer 2 · answered Nov 23 '19 at 23:39

You should change this line:

STOP_SEARCH_STR="'""for $TASKNAME""'"

to this line (yes, no single quotes needed, they are not useful):

STOP_SEARCH_STR="for $TASKNAME"

And use this line in the script (yes, quoted as shown):

grep -F "$STOP_SEARCH_STR" "$TEMP"/draft

If you want/need to add an echo (which is superfluos and may even cause problems) use this line (yes, quote all the expansions (variables and subshell)):

echo "$(grep -F "$STOP_SEARCH_STR" "$TEMP"/draft")"

Thanks. @ilkkachu already helped me resolve my problem. But your answer further clarifies the issue and may help others in the future. — datsb, Nov 24 '19 at 08:17

Bash string eval unclear

2 Answers2