2

I was editing a script I wrote some time ago to automatically backup the contents of my Raspberry Pi SD card, and in the course of editing, I missed a "/", and so here is the entire shell script:

#!/bin/bash
date
echo BACKING UP RASPBERRY PI
sudo dd if=/dev/mmcblk0p2  of=/media/USBDRIVE/RPi-Backups/RPi-Backup-$(date +"%m_%d_%Y").img bs=10M
echo COMPRESSING RASPBERRY PI BACKUP
sudo gzip -9 -c /media/USBDRIVE/RPi-Backups/RPi-Backup-$(date +"%m_%d_%Y").img > /media/USBDRIVE/RPi-Backups/RPi-Backup-$(date +"%m_%d_%Y").img.gz

and once I did this, I realized that I made a mistake and stopped the script. I checked the directory where my backups are kept, and they were all gone. What was strange, was that there was as much space taken up in the drive as there had been before I ran the script, about half the 500 GB USB drive.

As far as I can tell, the backups are gone, which kind of undermines the whole point of the backups.

Is there a way to recover these files?

If not, is there a way to recover the space without wiping and reformatting the drive?

Update 1/2015: This has since happened to me on a brand new drive, different make and model, in both cases making over 200 GB simply disappear, with the space still somehow in use. To happen twice in a row, with two different drives made by two different manufacturers seems fairly unlikely. Could there be something in my backup script causing these failures? If the files are truly gone, why is the directory still there? Why does the disk and OS still think that the same amount of space is still in use as opposed to just calculating it as if the files were wiped (since they are apparently gone)?

Rui F Ribeiro
  • 55,929
  • 26
  • 146
  • 227
iwantmyphd
  • 71
  • 4
  • 1
    That doesn't make sense to me. Multiple slashes are just ignored by the VFS. I just tried that with `dd`: no problem. And why should a problem with the input path delete your backups? You should run your script through `bash -vx script` and show us the output (edit your question). – Hauke Laging May 20 '14 at 03:40
  • 1
    Can we see the rest of the command? It's the `of` part that can overwrite, not the `if`. In any case, as Hauke pointed out, the `//` is irrelevant, it will be ignored. – terdon May 20 '14 at 03:41
  • @terdon just added the code, I hope this clarifies what I was trying to do, thanks! – iwantmyphd May 20 '14 at 03:55
  • 1
    There's absolutely no way that this script would delete any files in your backup directory. It will create the `.img` and `.img.gz` files, but that's the only modifications it does. Something else deleted them. – Barmar May 20 '14 at 04:41
  • @Barmar I've been running it again, and the dd function worked, but as soon as the gzip line executed, the the .gz file was created, and about 30 sec late, both the .gz and .img files disappeared. I went up one level and back into the directory again... gone. If it isn't this code, I'm not sure what I should be looking for that would cause this. – iwantmyphd May 20 '14 at 04:43
  • 2
    How could this code cause something to happen 30 seconds after the script finishes? Maybe there's a cron job or a program using inotify that's cleaning up the backup directory. – Barmar May 20 '14 at 05:11
  • //dev is exactly the same as /dev. And I don't see any problem in your script, it had to be harmless. – peterh May 20 '14 at 07:02
  • Is it possible that your USB drive was unmounted in the meantime? That would make the files disappear. – terdon May 20 '14 at 11:16
  • @terdon ok, perhaps it's not the script, this just seemed to happen concurrently with running the script. With regard to mounting, this is absolutely possible. I tried unmounting and remounting, but no change. The good news is, everything outside of /media/USBDRIVE/RPi-Backups/ is still there, so it's only the one directory. Not sure what to do next... – iwantmyphd May 20 '14 at 12:07
  • [`//dev` is equivalent to `/dev`](http://unix.stackexchange.com/questions/12283/unix-difference-between-path-starting-with-and) (except in some exotic settings such as Cygwin, but nothing that would affect you). There are some oddities in your script, like calling `sudo` to run `gzip` and not compressing on the fly, but they wouldn't explain the files disappearing. If you yanked the stick out without unmounting at some point, that could explain it. Are you sure there wasn't another editing mistake that you fixed since then? – Gilles 'SO- stop being evil' May 20 '14 at 23:13
  • 1
    @Gilles Absolutely possible that there was something else missed in my script, but I'm not recognizing anything that looks odd (see above). Also, after re-running, and stopping the script, as I mentioned I can't see it in the GUI, and when I try to ls the directory, I get "ls: reading directory /media/USBDRIVE/RPi-Backups: Input/output error". Does the fact that the other directories and everything else on the drive is fine give any more clues? It seems odd to me that the issue is confined to just one directory. And since it seems that //dev means nothing, should I move to a new question? – iwantmyphd May 21 '14 at 03:26
  • What is the filesystem? NTFS? – mikeserv Jan 28 '15 at 06:19
  • Or maybe it is FAT and you're exceeding the 4gb filesize limitation. – mikeserv Jan 28 '15 at 06:54

1 Answers1

1

ls: reading directory /media/USBDRIVE/RPi-Backups: Input/output error

This is your problem. Always read error messages! The files didn't vanish: they didn't make it, and the dd or gzip process would also have exited with an I/O error.

This error can be a bit generic sometimes, but here, you're dealing with a straightforward filesystem-over-hardware situation. An I/O error means that the hardware is defective. There may be additional messages in the system logs. Usually, the problem is with the hard disk. Sometimes the cable goes bad — try another cable (it rarely solves anything, but it's cheap). It can also be a controller problem, or even a driver problem, but these tend to turn up pretty quickly, not the 50th time you use the drive.

Community
  • 1
Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
  • Thanks for identifying this! Yeah, it didn't occur to me at first to check the directory from the console since I was in the GUI while writing and running the script, but right you are. Since it seems to be a hardware issue, would this be something like bad blocks/sectors, since the rest of the files are intact? Also, I've had this external drive for about 9 months, so as you said, it's an issue that tends to happen early. I think however, it's still under warranty, so perhaps I can get a replacement/refund. Is there anything I can run to get more specifics on what went wrong? – iwantmyphd May 21 '14 at 13:17
  • @iwantmyphd Check the kernel logs (`/var/log/kern.log`). Look for messages containing `sdb` or whatever the device letter for the disk is. – Gilles 'SO- stop being evil' May 21 '14 at 20:42
  • Couldn't find a *kern.log* in */var/log* (or elsewhere), and all the other log files I checked (*user.log*, *auth.log*, *daemon.log*) had nothing about it. I did see a */var/log.hdd*, which was an empty file. Where else might it be? The good news is, it's still under warranty from my documentation, I'd just like my backups back! At least everything else is intact (for now)! – iwantmyphd May 21 '14 at 23:58
  • @iwantmyphd What unix variant/distribution is it? There's no standard for log location — `/var/log/kern.log` is common but not universal. Try `grep sdb /var/log/*` (or `grep sda /var/log/*` or whichever it is). – Gilles 'SO- stop being evil' May 22 '14 at 00:05
  • Sorry I forgot to mention that. I'm running Raspbian (based on Debian Wheezy) and a Raspberry PI. – iwantmyphd May 22 '14 at 00:44
  • So if the files are truly gone, why is the directory still there? Why does the disk and OS still think that the same amount of space is still in use as opposed to just calculating it as if the files were wiped (since they are apparently gone)? This has since happened to me on a brand new drive, different make and model, in both cases making over 200 GB simply disappear, with the space still somehow in use. To happen twice in a row, with two different drives made by two different manufacturers seems fairly unlikely. Could there be something in my backup script causing these failures? – iwantmyphd Jan 18 '15 at 07:10