2

All apt-like commands failed to create the lock file, because /var/lib/dpkg/ doesn't exist. Also, /lost+found/ has contents:

pi@pi-top:~ $ sudo ls -al /lost+found/
total 102456
drwx------ 11 root root      16384 Apr  3 16:26 .
drwxr-xr-x 23 root root       4096 May  5 17:00 ..
-rw-------  1 root root  104857600 Apr  3 16:30 #29025
lrwxrwxrwx  1 root root          9 Mar 29 10:05 #87867 -> /run/lock
lrwxrwxrwx  1 root root          4 Mar 29 10:05 #87868 -> /run
drwxr-xr-x  2 root root       4096 May  5 10:35 #89863
drwxr-xr-x 12 root root       4096 Apr  3 16:41 #89864
drwxr-xr-x 44 root root       4096 Apr  3 16:30 #89865
drwxrwsr-x  2 root staff      4096 Mar 12 14:03 #89866
drwxr-xr-x  6 root root       4096 May  5 16:30 #89867
drwxrwsr-x  2 root mail       4096 Mar 29 10:05 #89868
drwxr-xr-x  2 root root       4096 Mar 29 10:05 #89869
drwxr-xr-x  5 root root       4096 Mar 29 10:32 #89870
drwxrwxrwt  3 root root       4096 May  5 16:31 #89871

Lots of /var/lib/ is also missing, though the system isn't showing any other symptoms. Is it possible to recover the system (or at least dpkg)? If so, how?

Rui F Ribeiro
  • 55,929
  • 26
  • 146
  • 227
wizzwizz4
  • 527
  • 5
  • 19
  • @Downvoters Why is this a bad question? I'm generally familiar with Debian (& derivatives) but I am not intimately familiar with `dpkg`'s directory and files. – wizzwizz4 May 06 '18 at 17:49
  • Have you had a look inside the various sub-directories of `lost+found`? – Stephen Kitt May 06 '18 at 18:10
  • 2
    The downvotes may be because you're going at it wrong, which makes your question an [XY problem](https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem). I don't think it's a reason to downvote, but I suspect other people do. You're focusing on one symptom, but you have a deeper problem. A good answer (which I've tried to provide) should try to solve your problem rather than answer the question as asked. If, for example, you had accidentally deleted `/var/lib/dpkg`, it would be a completely different question. – Gilles 'SO- stop being evil' May 06 '18 at 18:23
  • @RuiFRibeiro Duly noted. – wizzwizz4 May 07 '18 at 12:50

1 Answers1

5

I'm sorry, but at this stage, you're probably better off restoring from a clean backup. When fsck places so many directories in /lost+found, that's a sign that there's been a lot of corruption. It's quite possible that there's more corruption, but because it's in the file contents rather than in metadata, fsck has no way to know.

When restoring from a backup, make sure that it's a clean backup. The corruption may have started before it was detected.

The only way to identify what the files in lost+found are is to look at them and figure it out. There is no systematic way. If there was one, fsck would do it.

Looking at the contents you show for /lost+found, it looks like the /var directory got damaged. You can try to repair it by creating /var and moving the appropriate entries in /lost+found to /var.

# Running as root, of course
umask 022
mkdir /var
mv /lost+found/\#87867 /var/lock
mv /lost+found/\#87868 /var/run
mv /lost+found/\#89866 /var/local
mv /lost+found/\#89868 /var/mail
…

I figured out the entries above from the metadata (ownership and symlink targets). You can figure more out by looking at the directory contents. Compare with an existing system installation (preferably the same distribution or at least something close, but the processor architecture doesn't matter).

/var/lib may be #89865 because it tends to have a lot of subdirectories, but that's only a guess. It could be from another part of the system altogether.

Don't focus on recovering /var/lib/dpkg and ignore the rest. The lack of /var/lib/dpkg is just the first symptom you noticed.

On a PC, I'd suggest to do a RAM test (with Memtest86+ which is available as a package on most distributions and is installed by default at least on Ubuntu). On a Raspberry Pi, if your system is on an SD card, I recommend replacing the SD card: SD cards are the least reliable part of the system, and if you keep using it, it'll probably keep corrupting your data.

Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
  • Some of the contents of `/var/` still exist, but fewer than I would expect (I glossed over this because I was panicking) so I think you're right about `/var` being corrupted. I'll replace the SD card (I need a bigger one anyway) as a lot of the problems I've been having could be attributed to "bad SD card corrupting binaries" instead of "shoddy programming". – wizzwizz4 May 06 '18 at 18:40
  • `#29025` is full of `^@` bytes; do you think it could be a false positive? – wizzwizz4 May 06 '18 at 18:41
  • 1
    @wizzwizz4 Given the size, `#29025` may be a swap file, or it may be some garbage that fsck mistakenly identified as a file because there was some corrupted content that looked like a file, but wasn't one. – Gilles 'SO- stop being evil' May 06 '18 at 19:02