1

I have an s3 share that's mounted via [s3fs-fuse][1] and would like to run some speed tests to compare throughput on DreamHost's DreamObjects vs Amazon S3. Everything is mounted and working just fine (s3fs testbucket ~/mnt/test -o passwd_file=/path/to/passwd-s3fs -o url=http://objects-us-west-1.dream.io), but traditional tests like dd and hdparm just don't work out.

Any recommendations on running a successful speed test on a block storage device in such a scenario?

Andrew Gaul
  • 127
  • 4
ylluminate
  • 591
  • 7
  • 16
  • What will you be using it for? What metrics are you interested in other than price per gb (both storage and xfer?) – ivanivan Apr 11 '17 at 03:21
  • Primarily I'm attempting to decide whether or not DreamObjects is sufficiently fast from a Linode server at a particular location for utilization as a block storage device for the moment and / or other purposes. I'm comparing a few locations in addition to this and so I need ascertain firstly the throughput and secondly the latency. – ylluminate Apr 11 '17 at 04:59
  • Neither S3 nor DreamObjects is block storage. They are *object* stores, so of course dd and hdparm are completely irrelevant. Additionally, s3fs emulates a filesystem, not a block storage device. – Michael - sqlbot Apr 11 '17 at 09:54
  • The point is to measure it as though it is block storage. Sure, it's going to have the same performance profile whatsoever, but they should be similarly testable. – ylluminate Apr 11 '17 at 14:12

1 Answers1

2

You cannot generally use object store to emulate a block device. There's s3backer which can work but there are caveats.

Using dd with s3fs should work though. I've wrapped a set of benchmarks comparing s3fs/goofys/riofs as a docker container: https://github.com/kahing/goofys/#runbenchmark . Should be simple to modify it to run on a different object store.

khc
  • 201
  • 2
  • 3