How do you write specific bytes to a file?

Question

Given a file myfile with the following contents:

$ cat myfile
foos

A hexdump of the file gives us the contents:

$ hexdump myfile
6f66 736f 000a

Currently, I can create the file by specifying the contents in ascii like this:

$ echo foos > myfile

Is it possible to create the file by giving it the exact bytes in hexadecimal rather than ascii?

$ # How can I make this work?
$ echo --fake-hex-option "6f66 736f 000a" > myfile
$ cat myfile
foos

Update: For clarity's sake, I worded the question to ask how to write a small number of bytes to a file. In reality, I need a way to pipe a large amount of hexadecimal numbers directly into a file rather than just 3 bytes:

$ cat hexfile
6f66 736f 6f66 736f ...
$ some_utility hexfile > myfile
$ cat myfile
foosfoosfoosfoos...

I think you might like the -C flag for hexdump. `hexdump -C` displays a byte by byte output instead of 16 bit words obfuscated up by endianess. — jippie, Nov 15 '12 at 07:46

score 7 · Answer 1 · answered Nov 14 '12 at 23:36

7

You can use echo -e:

echo -e "\x66\x6f\x6f"

Do note that hexdump -C is what you want to dump the contents of the file in byte order instead of being interpreted as 4-byte words in network byte order.

answered Nov 14 '12 at 23:36

Dennis Kaarsemaker

8,420
3
29
31

Although this does answer my question, it doesn't help in my situation. I have a very large input of hexadecimal numbers that I would like to pipe into a file. I will update the question with this. – Cory Klein Nov 14 '12 at 23:40

score 6 · Accepted Answer · edited Nov 15 '12 at 09:31

6

This is the hexundump script from my personal collection:

#!/usr/bin/env perl
$^W = 1;
$c = undef;
while (<>) {
    tr/0-9A-Fa-f//cd;
    if (defined $c) { warn "Consuming $c"; $_ = $c . $_; $c = undef; }
    if (length($_) & 1) { s/(.)$//; $c = $1; }
    print pack "H*", $_;
}
if (!eof) { die "$!"; }
if (defined $c) { warn "Odd number of hexadecimal digits"; }

edited Nov 15 '12 at 09:31

Chris Down

122,090
24
265
262

answered Nov 14 '12 at 23:45

Gilles 'SO- stop being evil'

807,993
194
1,674
2,175

jippie · Answer 3 · 2012-11-15T08:00:37.987

3

Simulate a byte train:

echo 41 42 43 44 |

Change spaces into newlines so the while/read can easily parse them on by one

tr ' ' '\n' |

Parse byte by byte

while read hex; do

Convert hex to ascii:

  printf \\x$hex

until end of input

done

If the files to parse are seriously big, you probably don't want to use bash because it is slow. PERL for example would be a better choice.

edited Nov 15 '12 at 08:00

answered Nov 15 '12 at 07:55

jippie

13,756
10
44
64

score 3 · Answer 4 · answered Nov 15 '12 at 08:49

The *some_utility* you're finding is dd (manual). You can copy any valid bytes from any valid position of a file to another file by specify bs(block size), count, and skip options.

Example

Copy the first 1024 bytes of a file to another file.

$ dd if=liveusb-creator-3.11.7-setup.exe of=test.ex_ bs=1 count=1024
1024+0 records in
1024+0 records out
1024 bytes (1.0 kB) copied, 0.03922 s, 26.1 kB/s

Zimba · Answer 5 · 2021-01-29T05:44:57.933

2

To write arbitrary hex data to binary file:

echo -n 666f6f | xxd -r -p - file.bin

For hex (input) data stored in some file to be written to binary file:

xxd -r -p file.hex file.bin

To read binary data:
hd file.bin or xxd file.bin

To read data only (without offsets):

xxd -p file.bin

edited Jan 29 '21 at 05:44

answered Jan 29 '21 at 03:17

Zimba

141
3

mikeserv · Answer 6 · 2015-03-13T02:19:12.713

1

Here's an example of how you can use dc to Print the (UCHAR_MAX+1) value of a byte:

printf %dP 104 101 121 32 116 104 101 114 101 10 |dc

...which prints...

hey there

The default input radix is 10 - decimal - but you can set it with $val i where $val is any number between 2 and 16 (note that if the current input radix is not 10 you'll have to use the current base's value for 10 to get it back - else you can always do Ai).

Here is a more complicated example:

LC_ALL=C man man 2>/dev/null | 
od -v -An -t x1 |
tr -s '[:space:]' P | {
    echo 16i0
    tr '[:lower:]' '[:upper:]'
} | dc | head

...which translates man man's output into hexadecimal and back again in-stream and prints:

MAN(1)              Manual pager utils              MAN(1)



NAME
       man - an interface to the on-line reference manuals

SYNOPSIS
       man [-C file] [-d] [-D] [--warnings[=warnings]] [-R
       encoding] [-L locale] [-m system[,...]]  [-M  path]

Just ensure that all of your alphabetic [:hexdigit:]s are uppercase and sandwich a P between every pair then pipe it at dc.

edited Mar 13 '15 at 02:19

answered Mar 12 '15 at 20:52

mikeserv

57,448
9
113
229

That's a pretty impressive 3-liner. I'd be interested if you cared to elaborate on that regex! – Cory Klein Mar 12 '15 at 21:33
@CoryKlein - the regex isn't really what matters - `sed` just inserts a `P` after every two not-space chars and appends the string `[]p` to every line. It is `dc` that is interesting. First `echo` sets its `i`nput radix to 16 *(so it will interpret the hex char pairs as numbers)* then for every byte it reads in it `P`rints its *(UCHAR_MAX+1)* value to stdout and pops it off the stack. Try `echo 8i141P12PAi97P10P16i61P0AP |dc` and see what you get. – mikeserv Mar 12 '15 at 22:04
With that, I get `a\na\na`, is that supposed to be what I see? – Cory Klein Mar 12 '15 at 23:22
@CoryKlein - yeah. It sets the input radix to octal then prints the ascii char for the octal bytes 141 and 12 - then sets it to `A` *(10 - decimal)* and prints the ascii chars for 97 and 10, then sets it to hex and prints the ascii chars for 61 and 0A. All of those char pairs are `a\n`. – mikeserv Mar 12 '15 at 23:27

How do you write specific bytes to a file?

6 Answers6

Linked