2

Ran into some unexpected but interesting behavior. I was doing a somewhat complex execution when I ran into a situation that from my understanding of how Bash handles characters shouldn't have happened.

Smaller example of issue:

$ arr=( "$(echo '1 && !/==/')" )
-bash: !/=: event not found

What is going on here? To my understanding, the single quotes should force Bash to interpret all the characters literally without doing any type of expansion.

Using Bash 4.1.2.

EDIT: Simplified question for reproduction.

Kusalananda
  • 320,670
  • 36
  • 633
  • 936
AnthonyBB
  • 351
  • 1
  • 2
  • 14
  • 1
    How about the single quote in "don't"? –  Feb 12 '21 at 23:58
  • I can't reproduce the problem (after removing the single-quote in `don't`). Can you post an exact example that shows the problem (along with the exact error message you get)? Also, what version of bash are you using (`echo "$BASH_VERSION"` will tell you)? – Gordon Davisson Feb 13 '21 at 00:43
  • Replace `!/pat/` by `(/pat/==0) ` so that atleast history expansion error goes away. Then see what you get. – guest_7 Feb 13 '21 at 00:45
  • I don't see any input for your awk command... which makes me believe that what you posted is not actually representative of your command. Please post an actual MWE. – steeldriver Feb 13 '21 at 01:06
  • I apologize for the original post not having all the available context, I ran into this issue at work and it bothered me through the weekend and I posted as much as I could remember thinking it was enough. – AnthonyBB Feb 16 '21 at 12:51
  • @AnthonyBB, if at all possible, please try to reduce commands like that to some smaller examples, such that don't depend on your specific case and your specific files. That makes it easier for others to test, when they don't have to worry about getting "file not found" errors or figuring what your script will do to their files... – ilkkachu Feb 16 '21 at 13:39

2 Answers2

3

bash exits with an error about a bad event

You have been bitten by the dreaded history substitution (the ! character inside your double quotes). You can disable it with set +H. That won't happen in a script BTW.

About word splitting: you can use readarray (or its alias mapfile)

readarray -t array < <(df -Ph ...)
xhienne
  • 17,075
  • 2
  • 52
  • 68
  • Why is history substitution even occurring when the ```!``` is within single quotes? ["When the command history expansion facilities are being used (see History Interaction), the history expansion character, usually ‘!’, must be quoted to prevent history expansion."](https://www.gnu.org/software/bash/manual/html_node/Quoting.html) ["Enclosing characters in single quotes (‘'’) preserves the literal value of each character within the quotes."](https://www.gnu.org/software/bash/manual/html_node/Single-Quotes.html) – AnthonyBB Feb 16 '21 at 12:53
  • @AnthonyBB It is not inside single quotes: `arr=( "$(....)" )` – xhienne Feb 16 '21 at 13:00
  • Are the single quotes inside of the command substitution superseded or something? This is the behavior I am trying to understand.```arr=( "$(... '... !/= ...' ...)" )``` – AnthonyBB Feb 16 '21 at 13:04
  • @AnthonyBB The history substitution occurs when your line is read by `readline`, not when it is interpreted by `bash`. From `readline`'s point of view, there is no notion of nesting and all that it sees is that your `!` is not inside single quotes. – xhienne Feb 16 '21 at 13:07
  • Up to the moment you just brought it up, I was under the impression that the readline libraries were for command-line editing. Could you point me in the direction of documentation on how it can interpret before bash? Anything I find is related to using shortcuts basically. – AnthonyBB Feb 16 '21 at 13:40
  • 1
    @AnthonyBB From my perception, [history expansion](https://www.man7.org/linux/man-pages/man3/history.3.html) is command-line editing. I often trigger the history expansion mechanism manually (meta-^) and continue editing my command line. The documentation your are looking for is the "History expansion" section in bash's manual. Here is a quote: "History expansion is performed immediately after a complete line is read, before the shell breaks it into words" but there is more to be read. – xhienne Feb 16 '21 at 13:50
  • Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/119810/discussion-between-anthonybb-and-xhienne). – AnthonyBB Feb 16 '21 at 13:52
1

Smaller example (but probably not smallest), in Bash 4.1:

$ set -H
$ arr=( "$(echo '1 && !/==/' )" )
bash4.1: !/==/': event not found
$ arr=(  $(echo '1 && !/==/' )  )              # no error
$

and in Bash 4.4:

$ set -H
$ arr=( "$(echo '1 && !/==/' )" )              # no error
$ arr=(  $(echo '1 && !/==/' )  )              # no error
$

That's history expansion allright. It happens quite early in the command line processing, and doesn't parse the whole command before that:

History expansion is performed immediately after a complete line is read, before the shell breaks it into words, and is performed on each line individually.

No idea why it works like that, why the double quoting makes a difference, but it's not the only bug with history expansion I remember hearing about.

You can turn history expansion off with set +H, or set +o histexpand, and it's not enabled in scripts by default anyway.

Also note that Bash 4.1 is more than 10 years old now, even Bash 4.4 (the last with a 4.x version number) was released in 2016. Other bugs have been fixed since 4.1, too, as well as useful features added.

ilkkachu
  • 133,243
  • 15
  • 236
  • 397
  • Yeah, I noticed that removing the double quotes removed the issue, but it doesn't stop Bash from doing word splitting, which I wanted to disable for this case. – AnthonyBB Feb 16 '21 at 13:42
  • 1
    @AnthonyBB, I know, it was really just to point out how fiddly it is, when such a minor change changes if it's recognized as history expansion. I think the real solution is to disable history expansion anyway. (Which I'd recommend even without issues like this, except for people who really _like_ it.) Also, consider updating. – ilkkachu Feb 16 '21 at 13:46