2

I'm trying to run a virtual machine off of a writable 9p root filesystem. When the default caching mode is used, apt and some other tools fail; likely due to limitations with 9pfs itself. The error message produced by apt is:

Apt: Unable to determine file size for fd ... : No such file or directory

A quick search online suggested that I should set caching to loose when mounting the filesystem. Mounting the filesystem with cache=loose on my system did indeed allow me to use apt normally.

However, now I am worried about filesystem consistency. Kernel documentation states that this cache=loose is only meant for read-only systems and sadly doesn't elaborate further.

So I present my question as follows: If I only access the filesystem through a single mount point where caching is set to loose, and I do not access the filesystem or its underlying directory otherwise, would I face any issues with consistency/coherency?

Or stated in another way: Does loose caching effect the consistency of the filesystem when viewed exclusively from a single mountpoint?

I'm using the following mount options: rw,trans=virtio,version=9p2000.L,cache=loose in my kernel command line and fstab.

Tenders McChiken
  • 908
  • 1
  • 9
  • 24

1 Answers1

1

There is another 9p.txt document at https://landley.net/kdocs/Documentation/filesystems/9p.txt that has a detailed listing specifically about some cache modes and other items.

It seems that if you access a single system then the cache should not hinder your activity. It is just that 9p will not be pushing any of the modification from one client to any other mounting the same share. They will only see the new updates on their next request.

This other document is mostly a replacement for the one you have linked except the opening two paragraphs. Both documents probably should be read to understand what you want to do in order to setup your mount points correctly. Below is a reposting of just the cache mode section for quick glance reference.

CACHE MODES
===========

By default, 9p operates uncached, letting the server handle concurrency.
On a modern LAN this as fast or faster than talking to local disk,
and scales surprisingly well (about as well as web servers). Back before
cache support was even implemented, v9fs was tested with ~2000 simultaneous
clients running against the same server, and performed acceptably. (Run
the server as root so setrlimit could remove the per-process filehandle
restriction, give server had plenty of RAM and plug it into a gigabit
switch.)

The "-o cache=loose" mode enables Linux's local VFS cache but makes no attempt
to handle multi-user concurrency beyond not panicing the kernel: updates are
written back when the client's VFS gets around to flushing them, last writer
wins. File locking and fsync/fdatasync are available if an application wants
to handle its own concurrency. Loose cacheing works well for read-only
mounts (allowing scalable fanout in clusters with intermediate servers
re-exporting read-only v9fs mounts to more clients), or mounts with
nonconcurrent users (including only one client mounting a directory,
or user home directories under a common directory).

; multiple users of the
same mount are fine, the potential conflcit is that if multiple systems mount
the same directory and modify the same files under it, the cache won't be
notified of updates on the server. The client pulls data from the server,
the server cannot asynchronously push unrequested updates to the client).

The "-o cache=fscache" mode uses Linux's fscache subsystem to provide
persistent local cacheing (which doesn't help concurrency at all). See
Documentation/filesystems/cacheing/fscache.txt for details.

This code makes no attempt to handle the full range of cacheing corner cases
other protocols wrestle with; v9fs just doesn't go there. The old saying is
"reliability, performance, concurrency: pick two" for a reason. Uncached mode
provides reliability and concurrency, cached mode provides performance and
one other (your choice which).

Even with cacheing, multiple users of the same mount on a client are fine,
the potential conflicit is that if multiple client systems the same directory
from a server and modify the same files under it, the client's cache won't be
notified of updates from other clients before naturally expiring. The client
pulls data from the server, the server cannot asynchronously push unrequested
updates to the client. In 9p the server only responds to client requests, it
never initiates transactions.
Yemi Bedu
  • 11
  • 2
  • Thank you. The document you linked to was *very* helpful. Do you know whether this document is reflective of the 9p implementation in the current mainline kernel, and why it wasn't merged upstream? – Tenders McChiken Sep 29 '20 at 05:19