0

The old cfq-iosched.txt mentions "asynchronous" or "async" requests in several places.

CFQ maintains the per process queue for the processes which request I/O operation(synchronous requests). In case of asynchronous requests, all the requests from all the processes are batched together according to their process's I/O priority.

How exactly should we understand the distinction in this doc, from a userspace point of view? I am unclear because there are several distinctions that are named sync/async:

  1. Normal read()/write() calls, v.s. Linux AIO (io_submit()/io_getevents).
  2. The O_SYNC flag, which can be set when you open() a file.

(Note, as I understand it, the quote above about IO priority does not apply to simple buffered writes. IO priority has no effect on such requests.)

sourcejedi
  • 48,311
  • 17
  • 143
  • 296

1 Answers1

1

The form of requests processed by the Linux IO schedulers, have various "hints" set on them. One of them is the "sync" hint.

All IO is handled async in Linux. This is fine for background writes, but for reads or writes that someone waits for completion on, we want to notify the block layer and IO scheduler so that they know about it. That allows them to make better scheduling decisions. So when the below references 'sync' and 'async', it is referencing this priority hint.

-- include/linux/fs.h (2016-11-01)

Read requests are always treated as synchronous.

block: don't use REQ_SYNC in the READ_SYNC definition (2016-11-01)

Reads are synchronous per definition, don't add another flag for it.

Both O_DIRECT and O_SYNC writes are treated as synchronous.

#define READ_SYNC       0
#define WRITE_SYNC      (REQ_SYNC | REQ_NOIDLE)
#define WRITE_ODIRECT   REQ_SYNC

IO requests initiated by fsync() use the same hint as O_SYNC writes. Although if the IO request has already been initiated and fsync() must wait for the existing request, I see no mechanism to adjust the request.

Re: trying to understand READ_META, READ_SYNC, WRITE_SYNC & co (2010)

But that leaves the question why disabling the idling logical for data integrity ->writepage is fine? This gets called from ->fsync or O_SYNC writes and will have the same impact as O_DIRECT writes.

We have never enabled idling for those. O_SYNC should get a nice boost too, it just needs to be benchmarked and tested and then there would be no reason not to add it.

We've only started using any kind of sync tag last year in ->writepage in commit a64c8610bd3b "block_write_full_page: Use synchronous writes for WBC_SYNC_ALL writebacks"

"disabling idling" is a different hint flag, which is a bit harder to explain. But the commit linked in the above quote confirms the use of the WRITE_SYNC hint for fsync().

The code quoted above has since been moved or replaced, but the use of the "sync" hint should still be the same in 2018 as it was in 2016.

So I don't believe it really matters whether the requests are made using Linux AIO or not.

I notice AIO is only supported for O_DIRECT, therefore all AIO should be "synchronous" from the IO scheduler's point of view :-). At least as of v4.20. The proposed new AIO interface supports buffered (non-O_DIRECT) IO. That will go through the same code above, so then the hint will still be set according to whether O_SYNC is used.

sync_file_range() is designed to allow triggering async writeback. The current implementation (Linux v4.20) avoids setting REQ_SYNC, by using the writeback mode WB_SYNC_NONE. This is true even when your program waits for the writeback by including the flag SYNC_FILE_RANGE_WAIT_AFTER. However, kernels between v2.6.29 and v4.4 incorrectly used WB_SYNC_ALL and hence REQ_SYNC for all writeback triggered by sync_file_range().

sourcejedi
  • 48,311
  • 17
  • 143
  • 296