3

I'd like to know which control sequences are sent to bash by programs to format their outputs.

For example, if I dump man less I can discern which parts are bold or underlined.

DDEESSCCRRIIPPTTIIOONN
       _L_e_s_s is a program similar to _m_o_r_e (1), but which allows backward  move-
       ment in the file as well as forward movement.

But if I send ls -G's output to a file there's no data regarding its colors format.

I'm using Mac OS X.

1.61803
  • 1,201
  • 2
  • 15
  • 23

6 Answers6

4

Use script /tmp/output to start recording in a new shell, then type your commands and look in the /tmp/output file, e.g. with an editor or cat -vet. Type exit to the shell to exit the recording.

meuh
  • 49,672
  • 2
  • 52
  • 114
  • Great answer! One detail though is, my aliases and functions stopped working after running `script`. My `/etc/profile` sources env vars, aliases and functions from a file in a custom location. – 1.61803 Aug 30 '15 at 18:33
  • By default, `script` uses $SHELL as the command to run. You might have more success starting a login shell, eg `script -c 'bash -l' /tmp/output`. – meuh Aug 30 '15 at 18:40
  • My version of `script` doesn't have `-c` option, but allows the syntax `script [-akq] [-t time] [file [command ...]]` so `script typescript bash -l` works fine. – 1.61803 Aug 30 '15 at 20:20
  • btw, for cat, `-e` is equivalent to `-vE` and `-t` is equivalent to `-vT` - so `-et` is the same as `-vet` or `-vET`. For understanding the coloring, `-v` should suffice, the others only add control characters for Tab and EOL, which can be more annoying than helpful. – xeruf Jun 07 '20 at 17:32
3

Most core utilities behave differently when they output to an interactive terminal. If they detect you are outputting into a file or a pipe, they don't format (reasonably so: we don't want color escape sequences when we are processing file lists with a script). For ls, you can force it:

CLICOLOR_FORCE=1 ls -G

(That's for OSX; on Linux, that would be ls --color=always.)

Other tools have similar switches (grep, for instance).

Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
orion
  • 12,302
  • 2
  • 31
  • 41
  • 1
    I know about the switches. What I want to achieve is display the stream of chars _with_ control sequences sent to the display by a program. – 1.61803 Aug 30 '15 at 14:53
  • 1
    The question was **how** can you get the raw formatting output. – MichalH Aug 30 '15 at 14:53
  • @1.61803 Doesn't `ls --color=always >file` do what you want? – jimmij Aug 30 '15 at 15:02
  • @jimmij, no, in bash 3.2.53(1)-release (x86_64-apple-darwin12) I get only text without any control sequence – 1.61803 Aug 30 '15 at 15:25
  • @1.61803 `bash` action is reduced here to redirect stdout of `ls` to a file with `>` so doesn't matter that much. When you open the output `file` in the text editor like `emacs` don't you see control characters? – jimmij Aug 30 '15 at 15:31
  • @jimmij, opened it in vi, no control characters — does it in your system? – 1.61803 Aug 30 '15 at 15:33
  • With `always` switch, it should output the control characters to a file; tell us how are you viewing the file. It is possible that whatever editor you are using to view the file is ignoring the control characters. If unsure, see a hexdump. Or at least a graphical viewer to remove the possibility that the terminal is parsing the control characters again. It is also possible that your implementation of `ls` does not behave this way - I'm familiar with the gnu version, you are on a pretty different system, there are always some pitfalls. – orion Aug 30 '15 at 15:34
  • 2
    @1.61803 oh, wait, you have apple, so `ls -G` instead of `ls --color=always` is the way to go. And concerning control characters - try `ls -G | cat -vte`. – jimmij Aug 30 '15 at 15:36
  • `ls -G > file` and then open with vi, no dice – 1.61803 Aug 30 '15 at 15:38
  • @1.61803 On OSX, you need to set the environment variable `CLICOLOR=FORCE`. See the man page. – Gilles 'SO- stop being evil' Aug 30 '15 at 23:26
  • @Gilles, checked the man page, doesn't work. Btw, it's not only `ls` I'm interested in. – 1.61803 Aug 31 '15 at 16:26
3

Probably best to run the output through a hex viewer (e.g. od, hexdump, xxd):

% man less | hexdump -C | head -5
00000000  4c 45 53 53 28 31 29 20  20 20 20 20 20 20 20 20  |LESS(1)         |
00000010  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20  |                |
*
00000040  20 20 20 20 20 20 20 4c  45 53 53 28 31 29 0a 0a  |       LESS(1)..|
00000050  0a 0a 4e 08 4e 41 08 41  4d 08 4d 45 08 45 0a 20  |..N.NA.AM.ME.E. |
%

As then you can lookup any non-printable characters in ascii(7).

As for the "doesn't work" nonsense, a quick look through the ls(1) manual results in:

% env TERM=xterm-color CLICOLOR_FORCE=1 ls -G | hexdump -C | head -3
00000000  1b 5b 33 34 6d 41 70 70  6c 69 63 61 74 69 6f 6e  |.[34mApplication|
00000010  73 1b 5b 6d 1b 5b 6d 0a  1b 5b 33 34 6d 44 65 73  |s.[m.[m..[34mDes|
00000020  6b 74 6f 70 1b 5b 6d 1b  5b 6d 0a 1b 5b 33 34 6d  |ktop.[m.[m..[34m|
%
thrig
  • 34,333
  • 3
  • 63
  • 84
3

For ls you can do

export CLICOLOR_FORCE=X
ls -G | cat -vet
fd0
  • 1,430
  • 1
  • 10
  • 14
  • That's a great answer, too! What does the value X of CLICOLOR_FORCE actually do? I searched the `man` pages but only `ls` refers to it, but doesn't tell anything about that or any value. If I set to any other value it has the same effect. – 1.61803 Sep 01 '15 at 15:16
  • @1.61803 or would you prefer golden mean- The X set the value of CLICOLOR_FORCE to a value other than null, as you found out. – fd0 Sep 01 '15 at 16:15
  • phi is always better. I just couldn't find a reference for setting the var to not null. I just made this an aliased function in my profile. Thanks. – 1.61803 Sep 01 '15 at 19:27
0

Maybe a simple:

 $ ls --color=always  |  od -cAn

will be enough.

0

The simple solutions are somewhat hidden among the existing answers and comments, so let me sum it up:

Sending the sequences

Programs that send formatting sequences only send them in interactive terminals by default, to not clutter automation. So unless you use script as outlined in this answer (works always, but can be a little more verbose and complicated to set up), you need to force your program to always output its formatting sequences, regardless of where the output is going.

In the case of ls, use ls -G on Mac and ls --color on Linux. Common utilities, another example is git, usually have such a flag.

If that is not available, you can trick the program into thinking it outputs to a tty.

Viewing the sequences

Pipe the output into cat -v, e.g. ls -G | cat -v

xeruf
  • 499
  • 4
  • 16