2

I've been playing around with ZFS and using sparse files as virtual devices to learn how to use the zfs tools.

It seems that there isn't much of a hit in performance by creating a raid pool out of sparse files which are on top of JBOD filesystems that are located on different devices.

So I'm wondering if this is a good way to have a flexible system going forward seen as it's not possible to shrink zfs filesystems.

So for example I have four disks. Each one has a single device filesystem on it. I create 4 sparse files, one on each filesystem therefore disk. The load is spread across the devices and they are also providing redundancy.

Until the raid array fills, I still have all the rest of the disk space for some things which may not need raid such as temporary bulky files.

I expect that there would be a major performance penalty when both types of filesystem would be busy at the same time.

Are there some more pros and cons of such an approach?

barrymac
  • 1,115
  • 1
  • 12
  • 18

2 Answers2

1

One issue is indeed you have two superimposed file system layers with redundant caches.

Another one is, unless the data is you write is never updated or deleted, the copy on write nature of ZFS will gradually use all the file back-end area faster than you expect. This will reduce their sparseness defeating your thin provisioning goal.

jlliagre
  • 60,319
  • 10
  • 115
  • 157
  • hadn't thought of the caching issue, and also very good point on the CAW growing the sparse files more than exptected! – barrymac Aug 20 '13 at 11:53
1

I would expect performance issues from the start. Mainly because you will be writing file to ZFS filesystem that is on top of another filesystem.

For every write you will be doing at least the double of the necessary I/O. Without counting with the temporary bulky files.

In any case, if you have your data segmented you can have different ZFS datasets with different block sizes. In additions to compression and deduplication, although this will consume a bit more of CPU. But it would be better than having FS on top of another FS.

BitsOfNix
  • 5,057
  • 2
  • 25
  • 34
  • I'm not sure what you mean by the block size of the dataset, or how that effects things. I guess you get to choose diffrent compression and dedup choices but I didn't think using datasets would allow you to mix redundancy levels on the same underlying devices – barrymac Aug 20 '13 at 11:54
  • the redundancy is on pool level. The datasets are more for performance I would say. For example for normal root fs we leave the default, but if we are putting datafiles for a oracle db the filesystem would have a block size of 4Mb if I'm not mistaken. – BitsOfNix Aug 20 '13 at 14:37
  • 1
    The disk I/Os won't be doubled by this technique. Only the lower layer (presumably ext4 or similar) is accessing the disk. The upper layer (zfs/zpool) is delegating the read/writes to the lower file system. – jlliagre Aug 20 '13 at 20:09
  • I would expect the main performance issue would be seek times due to fragmentation of the file based virtual devices. – barrymac Aug 21 '13 at 15:45
  • @jlliagre you are right, my mistake. But from what I understand ZFS will "order" a write or a read than after that needs to be "translated" for ext4 to get the correct positions. maybe I'll try to setup a similar env to check it out. – BitsOfNix Aug 22 '13 at 09:00
  • @AlexandreAlves There will be indeed an extra layer of indirection and "double buffering" but that shouldn't affect the performance as much as would actual double disk I/Os. Doubling the cache requirements will affect them much more. – jlliagre Aug 22 '13 at 09:40