Why does dd dislike taking input from process substitution?

Question

Is there something about process substitution (which I think is implemented with unnamed pipes) that is incompatible with dd?

For example, this fails:

$ dd if=<(unzip -p raspbian.zip) of=/dev/sdb status=progress        
dd: unrecognized operand ‘/dev/fd/4’
Try 'dd --help' for more information.

But this works fine:

$ unzip -p raspbian.zip | dd of=/dev/sdb status=progress       
108458496 bytes (108 MB) copied, 19.446285 s, 5.6 MB/s

Is the use of if somehow implicitly telling dd that it should be able to seek?

@Kusalananda right -- that's why I'm wondering if `if` is implying that the input *should be* seekable. — jhfrontz, Nov 14 '19 at 15:12
Your first example lists an error from `dd`, which doesn't show up under bash. What shell do you use? — user3840170, Nov 14 '19 at 15:16
I can't reproduce this with `cat` replacing `unzip` (I don't have Zip archives) in `bash` nor `zsh` on OpenBSD using either of GNU `dd` or OpenBSD `dd`. Did you originally have a space before the process substitution? That would make GNU `dd` complain as you show. — Kusalananda, Nov 14 '19 at 15:17
Simple test: `dd if=<(echo foo) of=/dev/null` works for me, but `dd if= <(echo foo) of=/dev/null` gives an error `dd: unrecognized operand ‘/dev/fd/63’` similar to that described in the question. I don't know which shell will pick fd 4 for its first process substitution. Try also `echo if=<(:)` to see if your shell is introducing a space where it shouldn't. — Toby Speight, Nov 14 '19 at 15:43
@TobySpeight you got it! I'm using ksh (specifically "Version AJM 93u+ 2012-08-01"). Trying `$ echo if=<(:)` produces `if= /dev/fd/4` (with a rogue leading space). ARGH! — jhfrontz, Nov 14 '19 at 17:29
In any case, you don't want to use `dd` on a pipe or to write to block devices. Just use `unzip -p raspbian.zip > /dev/sdb` or `unzip -p raspbian.zip | pv > /dev/sdb` if you want a progress status. — Stéphane Chazelas, Nov 14 '19 at 17:46
@StéphaneChazelas thanks. I left off some of the extras in my pipeline -- e.g., I use the `bs` parameter of `dd` to (perhaps vainly) optimize the drive I/O. — jhfrontz, Nov 14 '19 at 18:57

score 2 · Accepted Answer · answered Nov 14 '19 at 17:35

This seems to be caused by your shell adding leading whitespace (or otherwise performing argument splitting) when expanding the process substitution. We can demonstrate this:

$ echo if=<(:)
if= /dev/fd/4

Assuming that this behaviour is reproducible on the most recent version you can get, I think there's nothing you can do here other than submit a bug report, including the justifying case of dd to demonstrate that the bug causes real problems.

You might consider switching shells, but I can understand if that's unpalatable to you!

Why does dd dislike taking input from process substitution?

1 Answers1