xxd with 64 bit addresses

Question

I find xxd to be nice and convenient for simple manipulation of binary data in the terminal. However it has a limitation in that all addresses outputted by the program is limited to 32 bits. E.g.

root:/# xxd -s 0x5baae0000 /dev/sda
baae0000: 2d7c 6176 6976 6172 7c61 7469 7a61 720a  -|avivar|atizar.

Notice the 5 at the beginning is gone. This means that using this directly with xxd -r will end up patching a region over 10 billion bytes earlier than what I want. In other words I want to e.g. do this,

root:/# echo HELLO | xxd -o 0x5baae0000 | xxd -r - /dev/sdb

but I end up having to work around this by doing something like this,

root:/# echo HELLO | xxd -o 0x5baae0000 | xxd -r -seek 0x500000000 - /dev/sdb

which adds back the missing 0x5000000 bytes offset into the target file when parsing the patch.

---

In any case I want to know if there is either a unknown/clever way to have xxd use 64 bit addresses or an alternative program that operates like xxd without these limitations?

Answer 1

xxd uses longs for the address but explicitly truncates it to 32bit and there doesn't seem to be any way around that. That's vim issue #3791 (thanks for reporting it), now fixed in patch 8.1.0854.

It doesn't however seem it ever had that limitation on input (with -r), so you can use standard od to print the hex dump:

od -j 0x5baae0000 -Ax -vtx1 -N16 /dev/sda

Which will output something like:

5baae0000 2d 7c 61 76 69 76 61 72 7c 61 74 69 7a 61 72 0a
5baae0010

However note that several od implementations, including GNU od¹ don't seek to reach the offset specified by -j but read and skip all the data which makes it impractical for a large block device like in your case.

Or if you care about the ASCII side, using BSD hexdump's

$ hexdump -ve '"%_ax:" 16/1 " %02x"' -e '"  " 16 "%_p" "\n"' -s 0x5baae0000 -n 16 /dev/sda
5baae0000: 2d 7c 61 76 69 76 61 72 7c 61 74 69 7a 61 72 0a  -|avivar|atizar

Both of which you can feed to xxd -r

You can actually reproduce xxd's output format (without the 32 bit address limitation) with hexdump with:

hexdump -ve '"%08_ax: " 2/1 "%02x"" " 2/1 "%02x"" " 2/1 "%02x"" " 2/1 "%02x"" "
                        2/1 "%02x"" " 2/1 "%02x"" " 2/1 "%02x"" " 2/1 "%02x"
            ' -e '"  " 16/1 "%_p" "\n"'

Neither od nor hexdump support an equivalent of xxd's -o to offset the address, but you could always post-process their output to add the offset to the address fields with for instance:

perl -pe 's/^\w+/sprintf "%08x", 0xabcdef + hex$&/e'

In any case, to write data at specific offset into a file, you don't need xxd, you can use dd:

echo HELLO | dd bs=1 seek="$((0x5baae0000))" of=/dev/sda

Or ksh93 >#((...)) seeking operator:

echo HELLO 1<> /dev/sda >#((0x5baae0000))

or zsh's sysseek builtin:

zmodload zsh/system
{
  sysseek -u 1 $((0x5baae0000)) &&
    echo HELLO
} 1<> /dev/sda

You can also use xxd with a 0 offset and use dd/ksh93/zsh to do the seeking:

echo HELLO | xxd | { sysseek -u 1 $((0x5baae0000)) && xxd -r; } 1<> /dev/sda

or:

echo HELLO | xxd | { dd bs=1 seek="$((0x5baae0000))" count=0 && xxd -r; } 1<> /dev/sda

---

^{¹ Looking at the source, GNU od uses fstat() to get the size of the input (instead of lseek(SEEK_END)) so as not seek past the end of the file, which on systems like Linux where st_size doesn't reflect the size of the block device doesn't work for block devices}

Answer 2

A patch with a fix for the issue has now been officially included as a part of vim as patch 854. So either try to update to a newer version of vim (v8.1.0854+) that includes the patch or compile from source yourself if not yet available.