1

I'm running MongoDB on my personal computer. I noticed that performance is much slower when the data are on my 2-recent-spinning-hard-drive-software-RAID1 than when they are on an older spinning hard drive without RAID.

Old drive, no RAID

Single operations:

> var d = new Date(); db.test.createIndex({ "test": 1 } ); print(new Date - d + 'ms');
251ms
> var d = new Date(); db.test.createIndex({ "test": "2dsphere" } ); print(new Date - d + 'ms');
83ms
> var d = new Date(); db.dropDatabase(); print(new Date - d + 'ms');
71ms

Whole test suite: 250s

 Recent drives, RAID1 

Single operations:

> var d = new Date(); db.test.createIndex({ "test": 1 } ); print(new Date - d + 'ms');
1220ms
> var d = new Date(); db.test.createIndex({ "test": "2dsphere" } ); print(new Date - d + 'ms');
597ms
> var d = new Date(); db.dropDatabase(); print(new Date - d + 'ms');
671ms
> var d = new Date(); db.dropDatabase(); print(new Date - d + 'ms');
1ms

Whole test suite: 700s

 Configuration files

In case it would be useful (I doubt it):

/etc/fstab

UUID=d719f337-d835-4688-baf2-3e29f147ff15 /               ext4    errors=remount-ro 0       1
# /home was on /dev/md0p3 during installation
UUID=def01643-c71e-47df-9dc8-67096243aee6 /home           ext4    defaults        0       2
# swap was on /dev/md0p1 during installation
UUID=d43319a8-92fb-437d-b576-ef964276cde none            swap    sw              0       0
/dev/sr0        /media/cdrom0   udf,iso9660 user,noauto     0       0

 UUID="dd8b1f05-c65b-42e1-a45e-0ef421faf1df" /mnt/bak ext4 defaults,errors=remount-ro 0 1

/etc/mdadm/mdadm.conf

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR root

# definitions of existing MD arrays
ARRAY /dev/md/0  metadata=1.2 UUID=3a0f91ae:51c48198:3d1e26ed:118a1938 name=bouzin:0

# This configuration was auto-generated on Sun, 24 Jan 2016 18:00:55 +0100 by mkconf

Question

From what I've read, write access on the RAID1 should be roughly equal to the write access on a single equivalent drive.

  • Could this 5400/7200 factor explain the order of magnitude of difference in the tests above?

  • Could it be better without the RAID?

  • Any interesting test/benchmark I could run? For now, I only have Mongo shell tests, but they seem to point to the RAID, or the drives, rather than to Mongo itself. Is there some application-agnostic test I could run to identify anything?

  • Could anything be wrong or suboptimal with the RAID configuration?

EDIT:

Important is, that I mixed 7200 rpm and 5400 rpm drives.

Jérôme
  • 1,898
  • 3
  • 26
  • 47
  • I can't confirm it without full past and current mount info, but based On what I see here I would surmise that previously you had swap on a separate drive while currently your swap partition is sharing a physical drive with your raid array resulting in a measurable reduction in performance. This conjecture is based only on the comment in your fstab that reads `# swap was on /dev/md0p1 during installation` – Elder Geek Jan 30 '17 at 22:38
  • Clearly since a 5400 RPM drive spins at 75% of the speed of a 7200 RPM drive that will have an impact. By itself though, it doesn't explain the magnitude of the difference you are seeing. There may be other variables such as interface speed, seek speed and the aforementioned swap location. – Elder Geek Jan 30 '17 at 22:45
  • There is no before/after. Both the RAID and the 3rd drive are mounted on the machine. Indeed, from what I remember, I assembled the two drives in a RAID then created a logical volume in there, in which I put three partitions: /, /home and swap. I added the third drive afterwards and it has only one partition, no swap. (I'm no RAID/LVM expert, more info here: http://unix.stackexchange.com/questions/256878/recover-lvm-on-raid1-no-volume-groups-found-partition-table-erased) – Jérôme Jan 30 '17 at 22:46
  • So it could be partly the RPM difference, partly the swap issue? I don't get the swap issue, though. What if I don't use swap at that moment? Is there still an impact? – Jérôme Jan 30 '17 at 22:48
  • It's definitely partly the RPM difference. Clearly you have swap defined in fstab `UUID=d43319a8-92fb-437d-b576-ef964276cde none swap sw` how are you determining that the system isn't using swap for any purpose during your entire test suite run? Are your results the average of several runs to minimize the very real chance of erroneous results? – Elder Geek Jan 30 '17 at 22:53
  • Those results are one-shot, not averages, although I ran the commands "a few times" and got similar results. And I doubt I was using swap because I generally don't (enough RAM on the machine) but I should definitely double-check this as I generally don't run Mongo either. The issue with the swap is now clear to me. Thank you. Still interested in reference tests, if such a thing exists, rather than Mongo commands. – Jérôme Jan 30 '17 at 23:00
  • 1
    Please, check drives health by command `smartctl` and see the parameter `current pending sectors`. – Khirgiy Mikhail Jan 31 '17 at 05:36

2 Answers2

0

RAID1 will be slower than a single drive even the drive specs are equal.

The reason for this is while RAID1 improves reliability by performing every write to both drives, this same action reduces performance

RAID0 splits writes between 2 drives which improves performance by sharing the load but reduces reliability for the same reason.

RAID5 is a happy medium that results in better performance than a single drive as well as increased reliability as the failure of a drive won't make the array inoperable (although it will slow down substantially under these conditions).

The above is by no means an exhaustive list of RAID levels and their pros and cons nor a recommendation of any specific RAID level. For hardware recommendations https://hardwarerecs.stackexchange.com/ is a good choice.

Regardless of your method of benchmarking, to obtain accurate benchmarks you should run several tests and average the results and do so in single user mode when the system is not running other tasks as anything else will skew your results, likely resulting in higher iowait times than anticipated.

Another simple form of benchmarking would be to run dd with a sample file of a specific size. say you had (or created) a sourcefile of random data of X GB in size. you could then run time dd if=sourcefile of=target file

by using dd's bs= parameter you could run the test with different block sizes (see man dd) which might be useful for tuning depending on your needs/environment.

It should probably go without saying, but the quality of the drives you use for a RAID array is an important factor. Seagate's Barracuda Pro and WD's Gold datacenter drives are specified to less than 1 Unrecoverable Error in 10^15 bits read (about 125TB).

However, many other large capacity drives aren't at the higher spec. If you use a low-spec drive in a RAID, there's a good chance the rebuild will fail.

Elder Geek
  • 807
  • 1
  • 8
  • 20
  • 2
    Please do yourself a favor, and don't recommend RAID5. By current standards, RAID6 is acceptable. – Vlastimil Burián Jan 31 '17 at 02:41
  • @Vlastimil You make a valid point regarding RAID6 (sadly it requires a minimum of a 100% increase in the number of drives required over the RAID1 that the OP is using where RAID5 only requires a minimum of 50% increase in the number of drives.) Perhaps I was incorrect in my assumption that someone using RAID1 might have budgetary considerations. Furthermore, RAID 6 rebuilds work the array a lot harder during the rebuild; the whole point of RAID is to maintain availability in the presence of a drive failure. The extra work during the rebuild reduces availability. I see both sides. – Elder Geek Jan 31 '17 at 03:51
  • 1
    I read [write performance is equal to worst disk](https://en.wikipedia.org/wiki/Standard_RAID_levels#Performance_2) but that probably only applies to hardware RAID, not software. – Jérôme Jan 31 '17 at 08:28
0

RAID1 will be, at maximum, as fast as the slowest drive in the array.

Even if you had 3 drives in RAID1, where two of them are enterprise SSDs and one is consumer HDD, you will get the speed of that HDD.

For those, who never used or saw RAID1 on 3 or more drives, here is a wiki excerpt (link):

RAID 1 consists of an exact copy (or mirror) of a set of data on two or more disks

Vlastimil Burián
  • 27,586
  • 56
  • 179
  • 309
  • RAID1 on 3 drives? I've never seen a [mirror](https://en.wikipedia.org/wiki/Standard_RAID_levels#RAID_1) use 3 drives. Are you perhaps referring to [RAID1E?](https://en.wikipedia.org/wiki/Non-standard_RAID_levels#RAID_1E) – Elder Geek Jan 31 '17 at 03:56
  • 2
    @ElderGeek No, I am not. You can have mirrored data across as many drives, as you wish. – Vlastimil Burián Jan 31 '17 at 04:08
  • "RAID1 will be, at maximum, as fast as the slowest drive in the array." Yes I know. Both drives in the array are identical. WD Red 1 To NAS 5400 RPM. Can't remember why I picked 5400 RPM. Energy consumption, perhaps. Besides, picking identical disks for a RAID makes it likely to have both disks break at the same time. Using a RAID was probably useless or at least overkill on this home desktop anyway. From further experience, a daily rsync totally suits my needs. – Jérôme Jan 31 '17 at 08:26
  • @Jérôme I can assure you it is not overkill, I use RAID6 for home purposes. It is a matter how valuable the data are to you. If there are just a bunch of movies, you can download them anytime, right, but having e.g. 10TB of movies would be actually quite exhausting to replace. – Vlastimil Burián Jan 31 '17 at 08:53
  • Interesting. I'm curious why you would mirror data over 3 drives in a RAID 1 configuration rather than utilize the same drives in a RAID 5 array resulting in double the storage and less stress on the drives. – Elder Geek Jan 31 '17 at 18:55
  • 1
    @ElderGeek The math says that RAID5 is risky; rebuilding a degraded RAID5 is almost certain to kill one of the other drives in the array, destroying the entire array. It's not much of a step above RAID0. 3 drives in a RAID1 in a home setting is probably overkill (2 drives with one hot spare is more reasonable as the spare isn't getting worn at the same rate as the other two). – cdhowie Sep 06 '19 at 01:07
  • @cdhowie I'm not sure what math you base this opinion on. If you have to rebuild a RAID5 array you likely already have a failing drive. Speaking of Math, RAID1 over 3 drives has to be one of the least cost effective ways to achieve redundancy. Cheers! – Elder Geek Sep 06 '19 at 13:28
  • @ElderGeek Yes, if one drive in a RAID5 fails then you need to read the entire surface of _every other disk in the array_ in order to rebuild the array, and this is very likely to kill at least one other disk in the array, thereby killing the whole array. See [this question](https://superuser.com/q/1334674/55081) which is one of many discussions of the math behind why RAID5 is risky, particularly with consumer-grade disks. The more disks in a RAID5, the more likely it will fail during rebuild. – cdhowie Sep 06 '19 at 19:41
  • @ElderGeek RAID1 with three drives is usually wasteful, yes -- though two active and one hot spare is generally a good idea. For larger arrays, RAID10 with a hot spare is usually advised over RAID5/6. – cdhowie Sep 06 '19 at 19:42
  • @cdhowie if you are using hot spares, then the RAID will start rebuilding immediately. I believe this is the assumption you are making regarding risk. However RAID5 can tolerate losing 1 drive every bit as well as RAID1 and still maintain access to all the data (albeit slower than before) and provides far more storage space and better performance while operating as intended. Having all your eggs in one basket is certainly never a good idea and a solid backup strategy is always preferable to the absence of one regardless of your RAID level. – Elder Geek Sep 06 '19 at 23:08
  • @ElderGeek No, what I'm saying is that using RAID5 is not much different from RAID0 in terms of likelihood of repairing the array. With RAID0 it's 0% since there is no redundancy; with RAID5 it's only ~40% chance of recovering on commodity disks. If a disk fails and you replace it (or use a hot spare) then __every sector on every remaining disk__ needs to be read to rebuild the failed disk, and there is a very high probability (~60%) that this will cause one of the remaining disks to fail. – cdhowie Sep 07 '19 at 00:57
  • @cdhowie Perhaps we had different math teachers, but if you are equating 0% with 40%, I'll gladly loan you money @0% interest, (no make that 40%). Using consumer quality disks for a mission critical RAID is not an assumption I would make nor a path I would take. Cheers, my friend! – Elder Geek Sep 07 '19 at 16:11
  • @ElderGeek Of course, 40% would be the better option but it's still "basically zero" if uptime is the goal (it's slightly worse than coin-flip odds). For a RAID5 rebuild, as a rule the chance of a URE depends on the total capacity of the array; for RAID10 it depends on the capacity of the degraded mirror pair which is likely going to be significantly lower, thus increasing the chances of a successful rebuild. – cdhowie Sep 07 '19 at 19:07