1

The setting that I have is a compute with only 2Gb of free memory. When I upload large images (3.5 Gb) through the Horizon (which uses apache2) the image fills the cache and starts to swap.

I would have thought that the kernel would be able to see when the image starts uploading that the image is larger than the ram memory available and would simply copy directly to disk.

I would be grateful for any comment,

Michael

user295500
  • 19
  • 2
  • Perhaps whatever apache is running for you, eg php, is putting the POST data in a tmp file in /tmp or a similar filesystem that is a tmpfs, and so uses memory. See if you can configure the temp files directory elsewhere. – meuh Jun 25 '18 at 18:57
  • Originally that was the case, but I changed that to var/lib... So, it doesn't seem that this is the issue. Or, maybe I miss understand. – user295500 Jun 25 '18 at 19:10
  • HTTP POST is not meant to transfer such large files. Things work as they are supposed to, not as you wish they would. – Rui F Ribeiro Jun 25 '18 at 20:37
  • Ok, i never said that it was using POST this was edited. When i look at the actual command running it is using curl. – user295500 Jun 26 '18 at 04:08

1 Answers1

1

Without some sort of module to do the lifting for it, Apache will try to read the file into memory before doing anything with it with a simple HTTP POST request. Apache is not a tool for transferring large files. Is there a reason you cannot use a purpose-built tool such as scp?

DopeGhoti
  • 73,792
  • 8
  • 97
  • 133
  • The reason that i can't use scp is that i am working with the Horizon in OpenStack and through the web interface i am uploading the image. – user295500 Jun 25 '18 at 18:41