I accidentally deleted the /var/log/mail file. Until that point I was able to monitor it using postfix stuff. Now, it seems that Postfix doesn't send its logs to /var/log/mail, since the file is not getting updated with new log messages.
- 807,993
- 194
- 1,674
- 2,175
- 2,222
- 8
- 25
- 35
5 Answers
When you delete mail.log file, rsyslog (on ubuntu) loose handle to file. To get it working back on Ubuntu, please give:
sudo service rsyslog restart
This will not only create new file but also start to write logs.
- 319
- 2
- 4
-
1Instead of saying what you did, make the answer more general (in an instructive manner) telling us why. – Julie Pelletier Sep 09 '16 at 17:17
Even after creating an empty file
touch /var/log/mail
you have to restart the syslog
service syslog restart
and then it's logging gain :)
- 2,222
- 8
- 25
- 35
This is a bug in syslog, but illustrates a common issue when one deletes a file while it's open by a program. When you do an "rm ", you are removing a directory entry, but you are NOT removing the underlying file. The operating system keeps a count of references to the file, and will not actually delete the underlying file data until the reference count goes to zero. In the case of an average file, the reference count of the unopened file is one (the directory entry). When the file is opened, the count is incremented to two. If a second program opens the same file, the count will be incremented to three. If the directory entry is now deleted, the count is decremented to two -- which means that the file is anomymous (has no name), but will not be deleted until both programs which have it open close -- at which case the OS will delete the underlying disk storage associated with the file.
When you delete /var/log/mail, the system logger still has the file open for writing. If you create a new /var/log/mail, it will point to a file different from the one the system logger is currently writing. The only way to make everything consistent is to restart the system logger. When the original system logger terminates, all files associated with it are closed -- including the anonymous mail log whose directory entry you deleted. When you restart the system logger, it will re-open /var/log/mail when it needs to write a log message, and will keep it open thereafter.
Another way this is often discovered is when a running program fills up all of a disk with file data; the user deletes the very large file, but the disk space is not freed, because the file still exists, and is taking up disk space, but the directory entry has been removed. When the program ends (either because the user killed it or it ended itself), the disk space will be recovered because the reference count on the file will have gone to zero.
What the logger might do to prevent this is to first write the log message, check to see if the log file directory entry exists, and if it doesn't exist, close the original log file, open a new one, and then rewrite the message -- so that the message doesn't get lost. But to do all of that would require much more complexity than the system logger ought to have -- for each message it writes will take quite a bit longer to be written due to the extra directory check -- which will succeed every time the file has NOT been deleted.
To understand all of the above more clearly, the following command is instructive, for it describes the system call that performs the directory entry removal and the reference decrement: "man 3 unlink"
- 61
- 1
- 1
-
Cool. So, how do you view all the messages that were logged to the anonymous file? – Adam Winter Feb 05 '21 at 21:39
That's not the problem on CentOS 7. Someone thought it would be a great idea to have the postfix mail logs to go through the journaler. If you want to see postfix logs:
journalctl -u postfix
(to see whole log)
journalctl -u postfix -f
(to tail the log)
You may also need in the main.cf for postfix
syslog_name = postfix
- 805
- 2
- 10
- 19
- 61
- 1
- 1
-
1I spent three hours trying to repair a system where the logging disappeared. Has I not found this post I would have spent another 3 hours. My journal had 10000s of lines so this command also helped me a lot `journalctl --vacuum-time=1d` – Eugene van der Merwe Jan 22 '19 at 06:45
-
My left pinky to know how to change it back to normal logging. Using journalctl is cumbersome. – Eugene van der Merwe Feb 02 '21 at 08:25
-
@EugenevanderMerwe My searching has not been very successful — what terms should I search on to learn what your left pinky knows? How do I move postfix logs back to [r]syslogd, and out of this journalctl thing? – Ricky Morse Jul 15 '21 at 19:32
-
@RickyMorse unfortunately I couldn't find anything as well. It's a real hard search. Possibly it's really complicated because I imagine using one logging engine versus the next is integral to the functioning of the OS. What would help is if we knew the converse of Journalctl, like is it SystemD logging? – Eugene van der Merwe Jul 16 '21 at 09:05
-
-
This put me on the right track. Comparing the old and new rsyslog.conf files, I found that if one adds the following, rsyslogd will slurp in the Postfix log entries from journal: `module(load="imjournal" StateFile="imjournal.state")` – stevieb Oct 20 '22 at 15:04
fwiw newer versions of postfix log to /var/log/mail.log and i also had to run sudo chmod a+w /var/log/mail* and service postfix restart to get my postfix logs back after deleting it
- 131
- 3