1

I applied the following wrong Sed command to my nginx configuration:

sed 's/php7.2-fpm/php7.4-fpm/g' /etc/nginx/sites-enabled/default > /etc/nginx/sites-enabled/default

And the entire file is gone. I have no backups.

Is there a way to restore the original file? Re-configuring everything will take me days.

Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
Madno
  • 153
  • 1
  • 3
  • 11
  • The first thing is to stop writing to this filesystem immediately. But even if you do that, don't get your hopes up. You're unlikely to recover a short text file. You may be able to recover a slightly older version, if you're very lucky. – Gilles 'SO- stop being evil' Jul 23 '20 at 15:17
  • Ah shit, we are screwed. – Madno Jul 23 '20 at 15:25
  • What filesystem is the file on? Is it running on bare metal or in some kind of virtualization or container? – Gilles 'SO- stop being evil' Jul 23 '20 at 15:35
  • Ext4 on DigitalOcean VPS. – Madno Jul 23 '20 at 15:59
  • Why screwed? All people do backup, right? Did you have a backup? Never edit/modify part of system where you need superuser, without having a copy. [and please: document all changes you do, this will help you in future: we do not have so much time to figure out things we did in past, so start the good habit to backup and document things] – Giacomo Catenazzi Jul 24 '20 at 07:49

1 Answers1

3

Exploring the running Nginx process

If Nginx is still running (don't restart it!), you can ask it to dump its configuration. I have no experience with Nginx, so this is just something I read on the web, I don't know how much information you can recover this way or what might go wrong if you try it.

/usr/sbin/nginx -c /some/other/config -T

On older versions, you can try to dump the configuration from the running process the hard way. I don't know how hard this might be or whether there is a hope of reading the configuration file after the server has been running for a while.

You may be able to dump the configuration information in its internal binary representation even if the original text of the configuration file is no longer available. You'd at least have the configuration, but not the comments.

Looking for deleted files

Deleted content remains on the disk until it's overwritten. But finding it can be very difficult. Disk blocks with deleted content immediately become available for reuse, and there's no particular reason why long-deleted blocks would be overwritten before freshly deleted blocks. So don't get your hopes up.

Another reason not to get your hopes up is that you're likely to find many old copies of the file. If the file is larger than a filesystem block (often 4kB but it depends on the filesystem and configuration), it might be difficult or impossible to piece the parts of a single version together. But there is a small chance that you can find something.

Make sure you don't write to the disk anymore. Ideally, you should mount the filesystem read-only. The command mount -o remount,ro / does that, but it only works if there are no files open for writing, so for example it won't work if /var/log is on the same filesystem as /etc. You may want to shut down the logging service to avoid logs overwriting the freshly deleted file. You may have to force the issue. If you decide not to force it, be very careful to avoid writing as much as possible. If you need to install new software, make sure to write to some other filesystem. If you've recovered some things, write them to some other filesystem.

Recovering files is hard. It's an act of desperation. If you really need to do this, I suggest starting with the Arch Wiki guide (the Arch wiki tends to have good information even if you aren't running Arch Linux).

Preventing similar incidents in the future

  1. Make backups.

  2. Keep configuration files under version control. I use etckeeper for /etc on all the Linux machines I maintain. Activate the daily autocommit if you aren't disciplined about committing. In addition to storing copies of old versions, it gives you the opportunity to document why you made a change.

Gilles 'SO- stop being evil'
  • 807,993
  • 194
  • 1,674
  • 2,175
  • Thanks for this. Sadly I had already overwritten the file and restarted nginx by the time I asked, so there's nothing left for me but to do the whole thing from zero. A good lesson for the future, though. – Madno Jul 23 '20 at 16:11