Make xxd break lines

Question

For showing the content of a file with both formatted text and binary data inside in another post on SE I couldn't decide on wheather to show the ASCII rendered output or a the xxd output.

As a middle course I thought about a hexdump that does respect newlines.

(Other) example:

00000000: 636f 6d6d 6974 2031 3933 0074 7265 6520  commit 193.tree
00000010: 3331 3439 3530 3736 3263 6561 3232 3364  314950762cea223d
00000020: 3230 6233 3931 6235 3162 3336 3463 6134  20b391b51b364ca4
00000030: 3165 3636 3234 3931 0a61 7574 686f 7220  1e662491.author
00000040: 4578 616d 706c 6520 5573 6572 203c 6578  Example User <ex
00000050: 616d 706c 6575 7365 7240 7374 6163 6b6f  ampleuser@stacko
00000060: 7665 7266 6c6f 772e 636f 6d3e 2031 3439  verflow.com> 149
00000070: 3831 3236 3633 3320 2b30 3230 300a 636f  8126633 +0200.co
00000080: 6d6d 6974 7465 7220 4578 616d 706c 6520  mmitter Example
00000090: 5573 6572 203c 6578 616d 706c 6575 7365  User <exampleuse
000000a0: 7240 7374 6163 6b6f 7665 7266 6c6f 772e  r@stackoverflow.
000000b0: 636f 6d3e 2031 3439 3831 3236 3633 3320  com> 1498126633
000000c0: 2b30 3230 300a 0a69 6e69 740a            +0200..init.

would become

00000000: 636f 6d6d 6974 2031 3933 0074 7265 6520  commit 193.tree
00000010: 3331 3439 3530 3736 3263 6561 3232 3364  314950762cea223d
00000020: 3230 6233 3931 6235 3162 3336 3463 6134  20b391b51b364ca4
00000030: 3165 3636 3234 3931 0a                   1e662491.

00000030:                       61 7574 686f 7220           author
00000040: 4578 616d 706c 6520 5573 6572 203c 6578  Example User <ex
00000050: 616d 706c 6575 7365 7240 7374 6163 6b6f  ampleuser@stacko
00000060: 7665 7266 6c6f 772e 636f 6d3e 2031 3439  verflow.com> 149
00000070: 3831 3236 3633 3320 2b30 3230 300a       8126633 +0200.

00000070:                                    636f                co
00000080: 6d6d 6974 7465 7220 4578 616d 706c 6520  mmitter Example
00000090: 5573 6572 203c 6578 616d 706c 6575 7365  User <exampleuse
000000a0: 7240 7374 6163 6b6f 7665 7266 6c6f 772e  r@stackoverflow.
000000b0: 636f 6d3e 2031 3439 3831 3236 3633 3320  com> 1498126633
000000c0: 2b30 3230 300a                           +0200.

000000c0:                0a                              .

000000c0:                  69 6e69 740a                   init.

or

00000000: 636f 6d6d 6974 2031 3933 0074 7265 6520  commit 193.tree
00000010: 3331 3439 3530 3736 3263 6561 3232 3364  314950762cea223d
00000020: 3230 6233 3931 6235 3162 3336 3463 6134  20b391b51b364ca4
00000030: 3165 3636 3234 3931 0a                   1e662491.
00000030:                       61 7574 686f 7220           author
00000040: 4578 616d 706c 6520 5573 6572 203c 6578  Example User <ex
00000050: 616d 706c 6575 7365 7240 7374 6163 6b6f  ampleuser@stacko
00000060: 7665 7266 6c6f 772e 636f 6d3e 2031 3439  verflow.com> 149
00000070: 3831 3236 3633 3320 2b30 3230 300a       8126633 +0200.
00000070:                                    636f                co
00000080: 6d6d 6974 7465 7220 4578 616d 706c 6520  mmitter Example
00000090: 5573 6572 203c 6578 616d 706c 6575 7365  User <exampleuse
000000a0: 7240 7374 6163 6b6f 7665 7266 6c6f 772e  r@stackoverflow.
000000b0: 636f 6d3e 2031 3439 3831 3236 3633 3320  com> 1498126633
000000c0: 2b30 3230 300a                           +0200.
000000c0:                0a                              .
000000c0:                  69 6e69 740a                   init.

or

00000000: 636f 6d6d 6974 2031 3933 0074 7265 6520  commit 193.tree
00000010: 3331 3439 3530 3736 3263 6561 3232 3364  314950762cea223d
00000020: 3230 6233 3931 6235 3162 3336 3463 6134  20b391b51b364ca4
00000030: 3165 3636 3234 3931 0a                   1e662491.
                                61 7574 686f 7220           author
00000040: 4578 616d 706c 6520 5573 6572 203c 6578  Example User <ex
00000050: 616d 706c 6575 7365 7240 7374 6163 6b6f  ampleuser@stacko
00000060: 7665 7266 6c6f 772e 636f 6d3e 2031 3439  verflow.com> 149
00000070: 3831 3236 3633 3320 2b30 3230 300a       8126633 +0200.
                                             636f                co
00000080: 6d6d 6974 7465 7220 4578 616d 706c 6520  mmitter Example
00000090: 5573 6572 203c 6578 616d 706c 6575 7365  User <exampleuse
000000a0: 7240 7374 6163 6b6f 7665 7266 6c6f 772e  r@stackoverflow.
000000b0: 636f 6d3e 2031 3439 3831 3236 3633 3320  com> 1498126633
000000c0: 2b30 3230 300a                           +0200.
                         0a                              .
                           69 6e69 740a                   init.

or similar

score 2 · Accepted Answer · answered Jun 22 '17 at 23:43

Not exactly what you ask for but very close.

There are null bytes (of value 0x00) in your file, that makes things more complex. I am assuming that those bytes could be changed to DELs (bytes of value 0x7F) if that byte value doesn't exist in your input.

That is done with tr:

$ cat infile | tr '\00' '\177'

After processing the file with xxd that change could be recovered.

Run this script:

#!/bin/bash

while IFS=$'\n' read line; do
    length=$((${#line}+1))
    { 
        dd if=/dev/zero bs=1 count=$((position%16)) 2>/dev/null;
        echo "$line"; 
    } |
         tr '\177' '\0' |
         xxd -o "$((16*(position/16)))" -l "$((length+position%16))"
    echo
    (( position=position+length ))
done < <(cat infile | tr '\0' '\177')

To get:

$ ./so
00000000: 636f 6d6d 6974 2031 3933 0074 7265 6520  commit 193.tree
00000010: 3331 3439 3530 3736 3263 6561 3232 3364  314950762cea223d
00000020: 3230 6233 3931 6235 3162 3336 3463 6134  20b391b51b364ca4
00000030: 3165 3636 3234 3931 0a                   1e662491.

00000030: 0000 0000 0000 0000 0061 7574 686f 7220  .........author
00000040: 4578 616d 706c 6520 5573 6572 203c 6578  Example User <ex
00000050: 616d 706c 6575 7365 7240 7374 6163 6b6f  ampleuser@stacko
00000060: 7665 7266 6c6f 772e 636f 6d3e 2031 3439  verflow.com> 149
00000070: 3831 3236 3633 3320 2b30 3230 300a       8126633 +0200.

00000070: 0000 0000 0000 0000 0000 0000 0000 636f  ..............co
00000080: 6d6d 6974 7465 7220 4578 616d 706c 6520  mmitter Example
00000090: 5573 6572 203c 6578 616d 706c 6575 7365  User <exampleuse
000000a0: 7240 7374 6163 6b6f 7665 7266 6c6f 772e  r@stackoverflow.
000000b0: 636f 6d3e 2031 3439 3831 3236 3633 3320  com> 1498126633
000000c0: 2b30 3230 300a                           +0200.

000000c0: 0000 0000 0000 0a                        .......

000000c0: 0000 0000 0000 0069 6e69 740a            .......init.

Not exactly what you asked for (due to many zeros added) but I believe that that is close enough to get the idea across.

If you want an output without the zeros, but the numbering of bytes position will not be a multiple of 16, please ask for the script.

Wow, I didn't really expect a sane solution/answer to this :D. Kudos! — stuquus, Jun 23 '17 at 06:59

Make xxd break lines

1 Answers1