1

So I'm playing with dirty_background_ratio and dirty_ratio and hoping to find the right parameters with your professional help.

For now I'm using:

vm.dirty_background_ratio = 20 vm.dirty_ratio = 60

The main usage is torrenting, that means that the files will be download through torrent client and then seeded. Possibility of many downloads at once, that's why I should use RAM caching, thinking off correct values.

Maybe you could suggest me the right values?

Viktor
  • 217
  • 3
  • 11

1 Answers1

3

Your idea and the values (roughly doubling them) seem OK to, me, but you did not explain what you exactly mean by RAM caching. Here it is more just a buffering because the dirty pages are all heading unmodified to the disk.

If you have lots of IO on the same block device, it will just collide a bit later. The amount of dirty pages it not the only trigger, there is also (mm/page-writeback.c):

/*
 * The longest time for which data is allowed to remain dirty
 */
unsigned int dirty_expire_interval = 30 * 100; /* centiseconds */

Which gives the default 30 s. This could be just enough. But it means that dirty pages older than that will not be held back (time dimension of buffering/caching).

And iff you have concurrent IO, then these global settings will also affect that.

The best explanation for dirty_ratio and dirty_background_ratio is in the same file:

/* The following parameters are exported via /proc/sys/vm */

/*
 * Start background writeback (via writeback threads) at this percentage
 */
int dirty_background_ratio = 10;
...
/*
 * The generator of dirty data starts writeback at this percentage
 */
int vm_dirty_ratio = 20;

Shows that it is the same thing from different sides (dirty now, clean up later).

Here are some commands with which to analyze dirty pages:

]# cp MAINTAINERS MAINTAINERS-2

]# grep dirty /proc/vmstat 
nr_dirty 135
nr_dirty_threshold 311361
nr_dirty_background_threshold 155490

The threshold values are calculated from the ratio-values (given as percent or bytes). I have 8GB = 2M Pages, so this is the 10% and 20%, respectively.

With the page-types tool you can identify these dirty pages more precisely. This reads /proc/kpageflags, and takes 200ms or so. 

]# ./tools/vm/page-types  -b dirty -b lru -b ~active,~reclaim,~mmap |cut -c-80
             flags      page-count       MB  symbolic-flags                     long-symbolic-flags
0x0000000000000030               1        0  ____Dl__________________________________
0x0000000000000038             130        0  ___UDl__________________________________
0x0000000000044038               1        0  ___UDl________b___u_____________________
0x000000000000403c              23        0  __RUDl________b_________________________
             total             155        0

No matter if I just sit and wait (for 30 secs), or manually sync, the dirty pages are soon gone. 

]# sync
]# grep dirty /proc/vmstat 
nr_dirty 0
...

And the whole 130 pages of "UDl" are gone, ie. ones that are "uptodate, dirty, on LRU list". 

]# ./tools/vm/page-types  -b dirty -b lru -b ~active,~reclaim,~mmap |cut -c-80
             flags      page-count       MB  symbolic-flags                     long-symbolic-flags
0x0000000000044038               1        0  ___UDl________b___u_____________________
0x000000000000403c              23        0  __RUDl________b_________________________
             total              24        0

These 130 + 1 pages difference on two lines are exactly the file's size:

]# ls --block-size=4096 -s MAINTAINERS 
131 MAINTAINERS

These are my professional playing-around tips.

  • Thank you! Will read through. I mean that I do buffering, to gain better performance of the HDD for the read/write simultaneous operations - seeding. As ppl can download many files at once as seed, so i need to adjust the values for that so it would be cached(buffered) to help the HDD to gain best peformance. – Viktor Apr 13 '20 at 14:11