6

the card reader on my laptop doesn't want to work. can I use dd (or some other tool) to write an image to a networked disk.

I am trying to replace one raspberry pi distro with another. The SD card has 6gb free and uses only 2gb.

from the SD card

$ sudo parted -l
Model: SD SU08G (sd/mmc)
Disk /dev/mmcblk0: 7948MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End     Size    Type      File system  Flags
 1      1049kB  95.4MB  94.4MB  primary   fat16        lba
 2      95.4MB  1878MB  1783MB  extended
 5      96.5MB  1878MB  1782MB  logical   ext4
 3      1878MB  7948MB  6070MB  primary   ext4

And Please, before you criticize why I want to do this, answer the question... then tell me how stupid I am.

Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
kevcoder
  • 445
  • 1
  • 4
  • 12
  • so you want to write directly to the device file on the target system, is that right? – Graeme Mar 28 '14 at 16:31
  • yes. I want to set up a new raspberry pi install (debian wheezy) on an SD card that already has the arch version, thus I can ssh into the exiting system. – kevcoder Mar 28 '14 at 16:37
  • So you want to update the raspberry pi system image while it is running? – Graeme Mar 28 '14 at 16:40
  • yes.. and I know that sound ridiculous, but yes. All of the instructions on using dd comment that you need to be careful with the **of** parameter because you could end up overwriting your own system. If that is possible, then shouldn't I be able to mount a remote file system and dd over it using sshfs? – kevcoder Mar 28 '14 at 16:53
  • 3
    using `dd` to overwrite a mounted filesystem is a recipe for disaster. The easiest way to do what you want to do is if you have a free partition on the raspberry pi which is large enough to hold the image. You can `dd` to that partition and set up to boot from there. If not, what you are asking is still possible, just a *lot* more complex than a simple `dd` over the network. – Graeme Mar 28 '14 at 17:01
  • the card has 6gb unused space and a 2gb used partition. So your saying... from mint laptop, scp the img file to the 6gbpartition on the pi, then log into the pi and run dd if=/dev/6gbpartition/something.img of=/dev/Current2gbPartition? Add that as an answer. – kevcoder Mar 28 '14 at 17:21
  • No need for the `scp`, it will be faster to do the `dd` over the network as per frostschutz's answer. Please post the output of `parted -l` on the raspberry pi and I will answer with the steps you need to take. Also it would be good if you can update the question as per your above comments so that if anyone else is trying to do same thing, they are more likely to find this question. – Graeme Mar 28 '14 at 17:30
  • @Graeme the output of parted is attached – kevcoder Mar 28 '14 at 20:03
  • The problem with overwriting a live system isn't *just* that the live system will be overwritten. The rest of the system may very well choose to write something to the disk that it thinks it's under its control (such as cron activities, any background process reading a file and changing the atime, etc). The data written by dd on the target system may not be all the writes happening to the disk! If everything is under your control, at the very least you should try to remount all partitions as read-only on the target system. – chexum Jul 15 '23 at 17:31

4 Answers4

4

You can pipe through SSH:

dd if=something | ssh host dd of=something

But you should have a better reason for using dd than a simple file copy operation where you are better off with scp, rsync, and the like.

frostschutz
  • 47,228
  • 5
  • 112
  • 159
  • To be honest I believe passing `dd`'s output directly over SSH could save quite a bit of time and temporary space. – RAKK Mar 28 '14 at 17:26
  • `cat` also would work this way. – Keith Mar 28 '14 at 18:00
  • 6
    @RAKK Get rid of `dd` altogether: `something'`. It's [faster](http://unix.stackexchange.com/questions/9432/is-there-a-way-to-determine-the-optimal-value-for-the-bs-parameter-to-dd/9492#9492) and [more reliable](http://unix.stackexchange.com/questions/121865/create-random-data-with-dd-and-get-partial-read-warning-is-the-data-after-the/121888#121888). – Gilles 'SO- stop being evil' Mar 28 '14 at 23:07
  • I got bitten by this answer not being reliable, I believe that (if you want to use `dd`) you need to use `dd if=something iflag=fullblock | ssh host dd of=something oflag=fullblock` for this to work properly. – Compholio Jun 21 '22 at 20:02
1

Assuming you have an SSH server on your Raspberry and the SD card on the Raspberry shows up on /dev/sda, you would do something like this:

dd if=SDcardimage.img | ssh -o MACs=hmac-ripemd160 -l raspberry-pi-user <your Raspberry's IP address> 'dd of=/dev/sda'

I explain:

  1. dd outputs to standard output when no of output file is specified, and reads from standard input when no input file is specified on if.
  2. When you pass directly a command to SSH, your client will forward its standard input over to the server, and then the server passes that data to the standard input of the process specified on the command sent to the server.
  3. Because the SSH client's standard input in this case comes from dd, and the standard input of the remote dd comes from the SSH server, this will result in that the SSH client-server pair will act as an encrypted data transport, bridging the standard output from your local dd to the standard input of your remote dd. (This is in fact how SSH does remote login, with the standard input coming from your keyboard and the remote process receiving standard input being an UNIX shell).
  4. The MACs option I pass to SSH forces the connection to use the RIPEMD160 algorithm as message integrity digest. Doing this protects the transported data so that any transmission error or intentional tampering will be detected. This important here considering you're transmitting a disk image using radio waves. You might want to add o Ciphers=aes256-cbc or o Ciphers=aes256-ctr to force encryption just in case the Raspberry does a weaker encryption by default.
  5. -l indicates the user you're using to log in simply because I tend to use that syntax.
RAKK
  • 1,332
  • 2
  • 17
  • 35
  • 1
    Do not use `dd`. `/dev/sda'`. It's [faster](http://unix.stackexchange.com/questions/9432/is-there-a-way-to-determine-the-optimal-value-for-the-bs-parameter-to-dd/9492#9492) and [more reliable](http://unix.stackexchange.com/questions/121865/create-random-data-with-dd-and-get-partial-read-warning-is-the-data-after-the/121888#121888). – Gilles 'SO- stop being evil' Mar 28 '14 at 23:08
  • I actually have ditched `dd` altogether as of lately. It's the command OP posted though, so I kept it for consistency. – RAKK Mar 28 '14 at 23:19
  • It would be better to explain that in your answer. By the way, the `-o MACs=…` part is unnecessary, as would `-o Ciphers`: SSH protects the integrity and confidentiality of the data as a matter of course, you're merely selecting a non-default algorithms which is very rarely of any use. Don't pick AES-256, by the way: it's sometimes slower than AES-128, and never more secure (with all the computer power in the world today, you can't break an AES-128 key in a million years). – Gilles 'SO- stop being evil' Mar 28 '14 at 23:24
  • Does OpenSSH uses MD5 by default? If this is the case it would be a very good idea to enforce a stronger digest, since MD5 hashes can nowadays be very easily forged. – RAKK Mar 28 '14 at 23:29
  • You're overestimating the brokenness of MD5. There is no publicly known method, given X, to find Y such that MD5(X)=MD5(Y). MD5 is broken in that we can find Z such that MD5(X++Z)=MD5(X++Y). This is bad enough for many applications (e.g. SSL certificates), but does not allow forging a preimage for an MD5 hash. What SSH uses anyway isn't a hash but a MAC. I don't know if it uses HMAC-MD5 by default, but even if it did, HMAC-MD5 is not broken. It's deprecated because people fear that the attacks on MD5 will improve, but at the moment, nobody (publicly) knows a way to break HMAC-MD5. – Gilles 'SO- stop being evil' Mar 28 '14 at 23:42
  • @RAKK I tried this (minus the MAC and Ciphers stuff I'm using cable and not wifi), but I got a message that I ran out of space. I ran ` sudo dd if=*.img | ssh -l kevin ipaddress 'dd of=/dev/mmcblk05' `. maybe the of should have been to /dev/mmcblk0? thanks for the effort and the info. – kevcoder Apr 03 '14 at 03:30
  • @kevcoder Try using `pv | ssh...` instead and see if the entire file transfers. If it doesn't it means you should use a larger card. Or maybe you're flashing the image file to the wrong card indeed. You might need to install `pv`, most mainstream distros have it on their official package repositories. – RAKK Apr 03 '14 at 22:26
  • @kevcoder, you tried to copy the whole disk image into a single partition, no wonder you ran out of space. As commented above, using `dd` to overwrite mounted partitions will **not** work. – Graeme Apr 04 '14 at 13:03
1

Updating the running system so that the new root image will be used on the next reboot is a little more complicated than just doing a straight copy across the network. Assuming that the root image is on /dev/mmcblk0p5 (as indicated by the output of parted -l and the comments above), the OP should should be able to copy the root partition from the image to /dev/mmcblk0p3 without any problems (provided of course that you do not have any data you want to keep on this partition). You can follow the instructions below. However please make sure that you carry out the checks suggested and change the commands as appropriate, blindly copying the commands below is a good way to hose your system.

  1. I will assume that the root partition on the new image is partition 5, however it might not be. You can print the partition table of an image called pi.img with parted pi.image print. If your unsure, please add the output to your question.

  2. Since we are only looking to copy the image for the root partition and not the entire disk image (which will also contain its own partition table and boot partition) you need to separate this from the rest of the image. One way to do this (on a Linux system) is using kpartx:

    sudo kpartx -av pi.img

    If there are no other loop devices in use, this will use loop0, however please check the output and change further commands accordingly.

  3. Make sure the target partition is not mounted on the Raspberry Pi by doing:

    sudo umount /dev/mmcblk0p3

  4. For the next step you need to be able to pipe data through ssh and have root access. There are few ways to do this, but I think the easiest is just to enable the root login. Usually this isn't advised, but since you want to upload a new system image I don't think it will be a problem. On the Raspberry Pi do:

    sudo passwd root

  5. Next copy the image across to the Raspberry Pi (using RAAK's suggestion for data integrity):

    sudo dd if=/dev/mapper/loop0p5 |
  ssh -o MACs=hmac-ripemd160 root@host dd of=/dev/mmcblk0p3

    host can be whatever is in /etc/hostname on the Raspberry Pi (seemingly mypi for the OP) or the IP address of the Raspberry Pi. Also remember to change this accordingly if you are not using loop0.

  6. Next change the root partition for the Raspberry Pi, as far as I can see (I don't have a Raspberry Pi to test on) you have to change /boot/cmdline.txt. You would change the part that says root=/dev/mmcblk0p5 to root=/dev/mmcblk0p3.

  7. You can delete the loop and mapper devices on the system you copied from by doing:

    sudo kpartx -d pi.img

  8. Now reboot the Raspberry Pi and you should be using the new image!

Community
  • 1
Graeme
  • 33,607
  • 8
  • 85
  • 110
-1

What if you boot Pi using a USB ARM image, connect to net and this way the storage will be tamed enough. I am not a pro, just spit balling here. Experts please correct if I'm wrong.

Tis
  • 1
  • 1
    This does not provide an answer to the question. Once you have sufficient [reputation](https://unix.stackexchange.com/help/whats-reputation) you will be able to [comment on any post](https://unix.stackexchange.com/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). - [From Review](/review/late-answers/432634) – Stephen Kitt Dec 28 '22 at 15:03