Preserve presence of last newline when assigning to variable

Question

In bash, when an output is captured into variable using $(...), an annoying newline is appended. However, I have an output that sometimes ends in a newline, and sometimes not. I want the output to be captured as-is.

In my real problem, the strings may contain several lines, but the last may or may not close with a newline, and this property should be preserved.

In the style of this answer at Stack Exchange, my minimal working example looks like this:

#!/bin/bash

newlinetest() {
    if [ "$1" = 'with' ]; then
        printf '%s\n' 'Text with newline'
    else
        printf '%s' 'Text without newline'
    fi
}

s="$(newlinetest with ; printf '%s' 'x')"
s="${s%?}"
printf '%s%s%s\n' '(' "${s}" ')'
s="$(newlinetest without ; printf '%s' 'x')"
s="${s%?}"
printf '%s%s%s\n' '(' "${s}" ')'

It does what it should do, but IMHO this looks like an ugly hack. Is there any other elegant way to solve this? Maybe something that involves mapfile or read? A solution without external tools would be very welcome.

@Jesse_b This is more or less what he's using, but with `printf` instead of `echo`, which makes it more wordy. I'm unsure what's a "hack" about it though. — Kusalananda, Mar 15 '19 at 14:55
@Kusalananda: Yeah it seems like this is just an obfuscated way to use the solution Gilles suggested and then requesting to deobfuscate it. — jesse_b, Mar 15 '19 at 14:56
@Kusalananda Well, `echo x` appends the letter *x* followed by a newline. The newline gets trimmed by `$()`, and afterwards the *x* is removed. IMHO, the obvious newline makes the variant with `echo` even more obfuscated. — rexkogitans, Mar 15 '19 at 17:44

ilkkachu · Accepted Answer · 2019-03-18T11:59:17.857

2

Actually, command substitution removes trailing newlines, it doesn't append them. But if you use echo to print the variable later, it adds one by default.

Adding a trailing x or such is indeed somewhat hackish, but it's probably the easiest way, and works in all shells. You can make it somewhat shorter by using echo x instead of printf '%s' 'x'.

Since you mention read, you could use something like this (in Bash), too:

IFS= read -rd '' var < <(newlinetest with)

I'll leave it to you to decide if that's more pretty than this:

var=$(newlinetest with; echo x)
var=${var%x}

Though there is a potential performance difference between the two. In the first one, the process substitution creates a pipe, which read reads from one byte at a time to avoid overrunning a delimiter (That's not relevant here, but the shell doesn't know that.) The command substitution will read in larger blocks, and is therefore likely to be faster with a large output.

edited Mar 18 '19 at 11:59

answered Mar 15 '19 at 15:12

ilkkachu

133,243
15
236
397

ah, setting `-d` to empty string -- I missed that possibility. IMHO, it is more intuitive. – rexkogitans Mar 18 '19 at 11:13
@rexkogitans, actually, Bash uses the NUL byte (`\0`) as the delimiter with `-d ""`. That's close to having no delimiter at all, since variables in Bash can't contain NULs anyway. About that process substitution, there's a potential performance issue which I just realized, see edit. – ilkkachu Mar 18 '19 at 12:01
`read` is a performance issue on its own, but in this case I prefer to care for other developers who want to read the script, as I find the hack-ish solution far from obvious. – rexkogitans Mar 18 '19 at 12:59

Preserve presence of last newline when assigning to variable

1 Answers1