sed with here-string fails, but succeeds when echo output piped to sed

Question

After updating sed to 4.4 version, sed doesn't replace spaces with commas in the output of find command given to it as here-string:

sed --version
sed (GNU sed) 4.4

ls -l /tmp/test/
total 0
-rw-r--r-- 1 root root 0 Jan  9 17:25 a
-rw-r--r-- 1 root root 0 Jan  9 17:25 b

# NOT EXPECTED
sed "s: :,:g" <<< $(find /tmp/test/ -type f)
/tmp/test/b
/tmp/test/a

There are no issue in sed 4.2

sed --version
sed (GNU sed) 4.2.2

ls -l /tmp/test/
total 0
-rw-r--r-- 1 root root 0 Jan  9 17:25 a
-rw-r--r-- 1 root root 0 Jan  9 17:25 b

# as expected
sed "s: :,:g" <<< $(find /tmp/test/ -type f)
/tmp/test/a,/tmp/test/b

As a workaround, storing the result in variable and using echo helps:

a=$(find /tmp/test/ -type f)
echo $a | sed "s: :,:g" 
/tmp/test/b,/tmp/test/a

How to achieve the same output in sed 4.4 using here-string ?

Update

version of bash changed as well between the two systems:

bash --version
GNU bash, version 4.4.20

old version

bash --version
GNU bash, version 4.3.48

The behavior is likely due to the version of the (bash?) shell that you are using, rather than the version of sed - see for example [Split string using IFS](https://unix.stackexchange.com/questions/397205/split-string-using-ifs/397208#397208) — steeldriver, Jan 09 '22 at 17:42
This looks more like a change in shell quoting/parsing. Has the version of your shell also changed between the two systems? — roaima, Jan 09 '22 at 17:43
FWIW best practice in this scenario would more usually be `find ... | sed ...` — roaima, Jan 09 '22 at 17:44
@roaima yes, shell version changed as well, i've updated my question — rok, Jan 09 '22 at 17:49
Does this answer your question? [Why is a here-string command substitution considered as a single line?](https://unix.stackexchange.com/questions/164110/why-is-a-here-string-command-substitution-considered-as-a-single-line) — Kusalananda, Jan 09 '22 at 17:51
@they how is this a duplicate? It's a change in behaviour (bug fix) in bash — roaima, Jan 28 '22 at 00:42
@roaima You are correct. If the question had been "why does this happen?", then it would have been a dupe, but now they ask "how can I work around it?", which makes it different. I have retracted my close vote. — Kusalananda, Jan 28 '22 at 06:19

score 3 · Accepted Answer · answered Jan 09 '22 at 17:57

This is a change between bash versions 4.3 and 4.4

Bash no longer splits the expansion of here-strings, as the documentation has always said.

The correct behaviour is your new version as you were relying on a bug with your old code.

This will give you a comma-separated list of files,

find -type f | tr '\n' , | sed 's/,$/\n/'

However, seeing as filenames themselves can contain newlines and commas, it's all too easy to break this kind of fragile code. If you care to share your processing - in a new question - I'm sure someone will recommend a better way of processing the file names reliably and safely.

sed with here-string fails, but succeeds when echo output piped to sed

1 Answers1