Is there disk-level caching in Solaris 11?

Question

I have an iSCSI disk at /dev/rdsk/c5t6d0

I have made a partition (slice with UEFI label) at: /dev/rdsk/c5t6d0s0

Now I write some data to the slice:

echo "xyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxyxy" >/text
dd if=/text of=/dev/dsk/c5t6d0s0

If I dump the disk contents I see the UEFI label and also my data at offset 0x5000:

dd if=/dev/rdsk/c5t6d0 bs=512 | xxd
0000000: 0000 0000 0000 0000 0000 0000 0000 0000  ................
0000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
0000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
0000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
0000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
0000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
0000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
...
0000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
0000190: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00001a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00001b0: 0000 0000 0000 0000 0eb5 7760 0000 00ff  ..........w`....
00001c0: ffff eeff ffff 0100 0000 ff3f 0300 0000  ...........?....
00001d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00001e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00001f0: 0000 0000 0000 0000 0000 0000 0000 55aa  ..............U.
0000200: 4546 4920 5041 5254 0000 0100 5c00 0000  EFI PART....\...
0000210: 7187 d278 0000 0000 0100 0000 0000 0000  q..x............
0000220: ff3f 0300 0000 0000 2200 0000 0000 0000  .?......".......
0000230: de3f 0300 0000 0000 fd59 3d67 51a0 4d13  .?.......Y=gQ.M.
0000240: abdd a5e8 8251 c7ac 0200 0000 0000 0000  .....Q..........
0000250: 8000 0000 8000 0000 7e8c 8e06 0000 0000  ........~.......
0000260: 0000 0000 0000 0000 0000 0000 0000 0000  ................
0000270: 0000 0000 0000 0000 0000 0000 0000 0000  ................
0000280: 0000 0000 0000 0000 0000 0000 0000 0000  ................
...
0005000: 7879 7879 7879 7879 7879 7879 7879 7879  xyxyxyxyxyxyxyxy
0005010: 7879 7879 7879 7879 7879 7879 7879 7879  xyxyxyxyxyxyxyxy
0005020: 7879 7879 7879 7879 7879 7879 7879 7879  xyxyxyxyxyxyxyxy
0005030: 7879 7879 7879 7879 7879 7879 0ab6 db6d  xyxyxyxyxyxy...m
0005040: 6db6 db6d 6db6 db6d 6db6 db6d 6db6 db6d  m..mm..mm..mm..m
0005050: 6db6 db6d 6db6 db6d 6db6 db6d 6db6 db6d  m..mm..mm..mm..m
0005060: 6db6 db6d 6db6 db6d 6db6 db6d 6db6 db6d  m..mm..mm..mm..m
0005070: 6db6 db6d 6db6 db6d 6db6 db6d 6db6 db6d  m..mm..mm..mm..m
0005080: 6db6 db6d 6db6 db6d 6db6 db6d 6db6 db6d  m..mm..mm..mm..m
0005090: 6db6 db6d 6db6 db6d 6db6 db6d 6db6 db6d  m..mm..mm..mm..m

But If I do the same dump at storage side (iSCSI target), I do not see my data written! I shutdown the SPARC/solaris machine and voila! the data is now there.

How can this be? Is Solaris caching my data in memory? How can I disable this?

Note1: I tried to use sync but it did not help and anyway it is related to files in a file-system, which I do not have in my scenario.

Note2: My machine is a T5220 UltraSPARC T2 Server with Solaris 11.3

Note3: 'storage' is an Ubuntu server with iSCSI-target on local disk backing-store.

Answer 1

You are writing to the block device (dsk) which is buffered. Should you want to bypass the buffer, you might directly write to the raw device (rdsk, see What are character special and block special files in a unix system? ) by running:

dd if=/text of=/dev/rdsk/c5t6d0s0

Your /text file would probably need to have a size exactly multiple of a block size for dd to succeed though.

Alternatively, you can still use the character device but tell dd to sync its output:

dd if=/text of=/dev/dsk/c5t6d0s0 conv=sync

or, if you want fixed width output records:

dd if=/text of=/dev/rdsk/c5t6d0s0 cbs=512 conv=sync,block