2

I created shell script, that logs into FTP server and renames some directories. I put FTP commands in multiline string like so

# !/bin/sh

ftp -inv $HOST << EOF
user $USER $PASSWORD
$COMMANDS
bye
EOF

the $COMMANDS contains commands separated by new line. This approach works, when I construct command like so

NL=$'\n'
COMMANDS="command one$NL"
COMMANDS+="command two$NL"
COMMANDS+="command three"

I firstly tried populate $COMMANDS with function printf, but even after several trials, I was unsuccessful. The expression

COMMANDS="$(printf "%s\n" "command one" "command two" "command three")"

evaluated to string with blank spaces between commands and when used in script as FTP command, it was processed as single line.

I tried replace \n with \r which results in variable $COMMANDS not beeing populated with any string.

Can someone more educated please explain to me, whats the problem with printf function here?

Update

I've manage how to achieve my goal via rsync, so the FTP approach I've abandoned, but I'm still curious why this happens.

Problem is with storing printf in variable. I made test script:

echo "==== printf directly"
printf "%s\n" "foo" "bar" "baz"

echo "==== stored in variable"
VAR_1=$(printf "%s\n" "foo" "bar" "baz")
echo $VAR_1

running ./test.sh outputs

==== printf directly
foo
bar
baz
==== stored in variable
foo bar baz

setting IFS=$'\n' does not help.

Update 2

I can get multiline from variable if I wrap it in double qotes

echo "$VAR_1"

but I'm unable to get the same result if used inside heredoc

echo `cat <<EOF
$VAR_1
"$VAR_1"
EOF`

I get output

foo bar baz "foo bar baz"

note: there is even no newline between the two variable outputs in heredoc, which is also strange to me.

Rudolf Gröhling
  • 121
  • 1
  • 5
  • 1
    I think you have to mess with IFS (set it to \n only, not leave it to be the default \n or tab or space) to get it right –  Feb 21 '15 at 18:19
  • What is your `/bin/sh`? Is it a symlink to `bash`? – cuonglm Feb 21 '15 at 18:30
  • 1
    A word of advice: trying to do this with strings, command line ftp, and bash, is unlikely to work even if you get past this hurdle (been there, done that etc.). You should give serious consideration to using 'expect', or using some other language with an FTP library (e.g. python). – Craig Miskell Feb 21 '15 at 19:41
  • Or use a command line FTP client such as `lftp` – roaima Feb 21 '15 at 20:37
  • for what it's worth, the `printf` thing works fine here. – wurtel Feb 23 '15 at 09:00
  • @cuonglm here is [perfect explanation](http://stackoverflow.com/questions/5725296/difference-between-sh-and-bash) – Rudolf Gröhling Feb 23 '15 at 11:25
  • @Vaclav: I mean what is your `/bin/sh` link to? `/bin/bash` or anything else? – cuonglm Feb 23 '15 at 11:39
  • @cuonglm my scripting provider is `dash` (running debian/wheezy) – Rudolf Gröhling Feb 25 '15 at 10:00
  • @Vaclav: So you need quote your variable `"$COMMANDS"` and later solution won't work. `$'\n'` does not work in `dash`. – cuonglm Feb 25 '15 at 10:09

1 Answers1

2

When you do not use quotes, word splitting will split the string on any whitespace (by default), and echo will join the split parts with spaces. In effect, this will "eat" the newlines.

You already found the improvement

echo "$VAR_1"

Now do the same for the here-document:

echo "`cat <<EOF
$VAR_1
EOF`"

Edit: Not echo was eating the newlines, it was the shell.

ilkkachu
  • 133,243
  • 15
  • 236
  • 397
Walter A
  • 694
  • 4
  • 11
  • 5
    It's not echo doing it, the shell does wordsplitting after parameter expansion for command arguments. – jthill Feb 25 '15 at 21:46
  • Technically `$(cmd)` or the deprecated `\`cmd\` ` form strip *all* trailing newline characters, and when those are unquoted in list context (such as in arguments to a simple command like `echo`), they are subject to split+glob, and the newline character happens to be in the default value of `$IFS`. So upon expansion (not *parsing*), the string will be split into separate arguments to `echo` and `echo` happens to print its arguments space-separated (and followed by *one* newline character). – Stéphane Chazelas Jan 17 '22 at 10:37