No space left on device

Question

Hello I'm running openVZ on my debian machine with debian containers and now I can't run apache, because of no space left.

I checked df -h for finding which partions is full and here is an output:

Filesystem                                              Size  Used Avail Use% Mounted on
udev                                                     10M     0   10M   0% /dev
tmpfs                                                   4.8G  356K  4.8G   1% /run
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /
tmpfs                                                   5.0M     0  5.0M   0% /run/lock
tmpfs                                                    12G     0   12G   0% /run/shm
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web3/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web4/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web5/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web6/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web8/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web10/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web11/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web12/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web13/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web2/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web14/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web15/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web16/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web17/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web18/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web19/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web21/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web22/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web23/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web25/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web26/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web27/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web28/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web29/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web30/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client1/web31/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client4/web32/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client4/web33/log
/dev/disk/by-uuid/a75ab882-5448-4448-9344-6a93161bc2e5  225G  223G     0 100% /var/www/clients/client4/web34/log
/dev/ploop19310p1                                       4.7G  1.2G  3.3G  27% /var/lib/vz/root/101
devtmpfs                                               1000M     0 1000M   0% /var/lib/vz/root/101/dev
tmpfs                                                  1000M     0 1000M   0% /var/lib/vz/root/101/dev/shm
tmpfs                                                  1000M  109M  892M  11% /var/lib/vz/root/101/run
tmpfs                                                   5.0M     0  5.0M   0% /var/lib/vz/root/101/run/lock
tmpfs                                                  1000M     0 1000M   0% /var/lib/vz/root/101/sys/fs/cgroup
tmpfs                                                  1000M     0 1000M   0% /var/lib/vz/root/101/tmp
none                                                   1000M     0 1000M   0% /var/lib/vz/root/101/run/shm
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102
devtmpfs                                                 16G     0   16G   0% /var/lib/vz/root/102/dev
tmpfs                                                    16G     0   16G   0% /var/lib/vz/root/102/dev/shm
tmpfs                                                    16G   25M   16G   1% /var/lib/vz/root/102/run
tmpfs                                                   5.0M     0  5.0M   0% /var/lib/vz/root/102/run/lock
tmpfs                                                    16G     0   16G   0% /var/lib/vz/root/102/sys/fs/cgroup
tmpfs                                                    16G  868K   16G   1% /var/lib/vz/root/102/tmp
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client4/web37/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web36/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web35/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client4/web34/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client4/web33/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client4/web32/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web31/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web30/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web29/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web28/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web27/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web26/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web25/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web23/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web22/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web21/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web19/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web18/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web17/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web15/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web16/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web14/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web13/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web12/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web11/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web10/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web8/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web6/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web5/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web4/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web3/log
/dev/ploop54675p1                                       221G   94G  118G  45% /var/lib/vz/root/102/var/www/clients/client1/web2/log
none                                                     16G     0   16G   0% /var/lib/vz/root/102/run/shm

I don't understand, why /dev/disk/by-uuid/ disc are full, because there is only one error.log file, which has 0b size...

Can you give me an advice, which disc or partition can be full? Because on virtual machine I have 50% free space...

Are there any deleted but still open logs? `lsof | fgrep deleted` — thrig, Nov 13 '16 at 15:22
It shows me an long output, mainly: lsof: no pwd entry for UID ... cron 235668 root 5u REG 0,46 2220 69676093 (deleted)/var/lib/vz/root/102/tmp/tmpfzr8ypz — Michal, Nov 13 '16 at 17:03

score 1 · Answer 1 · edited Nov 14 '16 at 13:49

1

I am not able to commen coz of my reputation but Michal it seems that you have already figured out the answer through lsof. Cron is holding a deleted file. You can either kill cron with the pid as:

sudo kill -9 235668

else you can just empty the file descriptor as:

> /proc/235668/fd/5 " or "> /var/lib/vz/root/102/tmp/tmpfzr8ypz

edited Nov 14 '16 at 13:49

answered Nov 14 '16 at 13:12

Ashwin

146
2

score 0 · Answer 2 · edited Nov 14 '16 at 09:31

0

Use the below command to find the largest file and determine if it's useful else rm it. You can change the size of the files below in the command. Also why are the outputs are repeating for the filesystems?

find / -xdev -size +100000 -exec ls -l {} \;

edited Nov 14 '16 at 09:31

Archemar

31,183
18
69
104

answered Nov 14 '16 at 08:21

Sargu Xcode

1
1

No space left on device

2 Answers2