Removing all spaces, tabs, newlines, etc from a variable?

Question

This is the error I am getting and it's failing because of a variable whose value is supposed to be 2 (I am getting this using a select * from tabel). I am getting spaces in that variable.

+ 0 != 
         2
./setjobs[19]: 0:  not found.

How do I remove all those spaces or a newline from that variable? Can tr, sed, or anything help?

This what I am doing:

set_jobs_count=$(echo  "set heading off;
      select count(*) from oppar_db
      where ( oppar_db_job_name, oppar_db_job_rec ) in ($var) ;" | \
  sqlplus -s ${OP_ORA_USER}/${OP_ORA_PASS}@$OPERATIONAL_DB_NAME)

This works as suggested:

| sed 's/[[:space:]]//g'

But I still obtain a value like :

  set_jobs_count=
  2

You can cast a string to an int in the select statement. How that is done depends on database, Sybase, Oracle, MySQL, etc. — bsd, Feb 24 '12 at 13:05
using sed it's `| sed 's/[[:space:]]//g'` to collapse whitespace — bsd, Feb 24 '12 at 14:06
thanks works up to some extent but still the values of variable comes like `set_jobs_count= 2` — munish, Feb 24 '12 at 14:37
Depending on the shell, you may be able to do it without any external tools. See http://stackoverflow.com/a/3352015/587717 — Edd Steel, Feb 24 '12 at 16:30
In my case `tr -d '[:blank:]'` does a better job (than `[:space:]`) because it take case of tab characters as well. — Terry Wang, Jun 26 '20 at 05:33

score 72 · Accepted Answer · edited Dec 28 '20 at 09:43

72

The reason sed 's/[[:space:]]//g' leaves a newline in the output is because the data is presented to sed a line at a time. The substitution can therefore not replace newlines in the data (they are simply not part of the data that sed sees).

Instead, you may use tr

tr -d '[:space:]'

which will remove space characters, form feeds, new-lines, carriage returns, horizontal tabs, and vertical tabs.

edited Dec 28 '20 at 09:43

Kusalananda

320,670
36
633
936

answered Feb 24 '12 at 16:23

Kyle Jones

14,845
3
40
51

Note this approach is POSIX compliant, meaning it's very portable across shells https://pubs.opengroup.org/onlinepubs/009696899/utilities/tr.html – KFL Jul 11 '22 at 23:19
Sed absolutely sees new line characters, and *can* replace them. It just isn' – NeoVance Jan 11 '23 at 00:43

score 8 · Answer 2 · edited Dec 28 '20 at 08:50

8

In ksh, bash or zsh:

set_jobs_count=…
set_jobs_count=${set_jobs_count//[[:space:]]/}

In any Bourne-like shell, you can remove leading and trailing whitespace and normalize all intermediate whitespace to a single space like this:

set -f
set -- $set_jobs_count
set_jobs_count="$*"
set +f

set -f turns off globbing; if you know that the data contains none of the characters \[?*, you can omit it.

That assumes $IFS contains its default value of space, tab, newline. Other whitespace characters (CR, VT, FF...) are left alone.

edited Dec 28 '20 at 08:50

Stéphane Chazelas

522,931
91
1,010
1,501

answered Feb 27 '12 at 02:14

Gilles 'SO- stop being evil'

807,993
194
1,674
2,175

@BinaryZebra The globbing happens at `set -- $set_jobs_count`. `set_jobs_count=$*` is equivalent to `set_jobs_count="$@"` since `$*` and `$@` are only equivalent when unquoted and the right-hand side of an assignment is parsed the same way as a double-quoted string. – Gilles 'SO- stop being evil' Jul 26 '15 at 22:55

Gyre · Answer 3 · 2020-12-30T17:09:59.093

As already suggested, use "tr -d '[:space:]'", BUT if that doesn't work, possibly on an older Unix system, one needs to cover all these characters:

\011 HT    '\t' (horizontal tab)
\012 LF    '\n' (new line)
\013 VT    '\v' (vertical tab)
\014 FF    '\f' (form feed)
\015 CR    '\r' (carriage ret)
\040 SPACE ' '  (space)

with this:

tr -d '\011\012\013\014\015\040'

However, perhaps there are other non-visible characters garbling the output, The result from 'select count(*) ...' should be only an integer, so to leave only digits:

tr -dc '0-9'

However, that won't work if there are other characters after the first digits, and then more digits. So, to only return the first digit sequence found:

perl -ane '/\d+/ && print($&) && exit'

## example: will return '11', but raw output looks like '22 33'
(echo -e '\a11\b\b22 33') | perl -ane '/\d+/ && print($&) && exit'

Do you have any idea of when the standard `[:space:]` character class first came into use, given that you refer to unnamed systems that may not have it? I'm assuming it must be after the introduction of Perl, as you suggest using it on these systems. I've found no reference about this and the POSIX standard does not contain historical annotations. — Kusalananda, Dec 28 '20 at 11:28

Removing all spaces, tabs, newlines, etc from a variable?

3 Answers3