11.17.06
RAID 5 vs RAID 50 (informal comparison/benchmark)
OK, so you just bought that wonderful 4U server, with dual-core Xeon/Core 2 Duo, redundant gigabit ethernet connections, redundant power supplies, KVM over IP / IPMI management card, and most importantly, this wonderful fully hardware-based RAID controller (Semi-Soft RAID has been a disaster for me, by the way).
Now, the question is : Should you use RAID 5, RAID 50 or something else ? Choosing between simple RAID solutions like RAID 0 vs RAID 1 is pretty easy, and one can easily conclude that RAID 0 is for performance (especially writes), whereas RAID 1 is for security. (even though it helps reads). Please note that RAID 1 is very expensive in disk space…
Now, what about the “standard” RAID 3, 4, 5, 5E, 6, 6E levels, or the “nested” ones such as RAID 0+1, 10, 30, 100, 50, 60 ?
In fact, the right choice depends on your specific needs (which is the lovely sentence everyone will use to avoid giving his opinion).
Now, in practice, my personal opinion on that :
- RAID 3 and RAID 4 are only theoretical RAID levels that nobody actually uses.
- RAID 5E, RAID 6 and RAID 6E are probably very good RAID levels, but the RAID card I use doesn’t support it, so I suspect these levels to be only available in higher-end models. If you are lucky enough to try one of those, I would be happy to hear about the performance.
- Most of the other nested RAID levels either are not supported by hardware cards, or are very expensive to implement. So, if you can spend money for them, their evaluation might be good.
After these considerations, and considering that you want a certain balance between security, performance and price, you might want to compare RAID 5 and RAID 50, which are both good compromises between these concerns.
To put it simply let’s consider you have 8 disks. RAID 5 stripes the data over 7 disks, and uses the 8th one for parity checking, so that it can rebuild the data if one disk fails (this is a little simplistic, since no disk is dedicated to partity and the parity blocks are rotated over the disks, but you get the idea).
RAID 5+0 creates 2 RAID arrays with half of the disks, and creates a RAID 0 array (stripping) on top of that. As a result, it is more costly, since 2 disks are used for parity checking instead of one, but is more robust to failures (2 failures are allowed).
It is often said that RAID 5+0 is more efficient for writes… So I wanted to compare how better it is, if ever it is, on a 3ware 9550SX 8 port SATA II RAID controller with write cache enabled, and 8 SATA 300 GB Maxtor DiamondMax 10
So, the result is a set of informal benchmarks, that are probably not related to the way I am going to use the disk anyways, but that has the merit of drawing quick conclusions :
First of all, using RAID 5
sudo dd if=/dev/sda of=/dev/null bs=1M count=1024
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 32.7535 seconds, 328 MB/s
dd if=/dev/zero of=out bs=1M count=10240
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 88.8292 seconds, 121 MB/s
dd if=./out of=/dev/null
20971520+0 records in
20971520+0 records out
10737418240 bytes (11 GB) copied, 45.3929 seconds, 237 MB/s
dd if=/dev/zero of=ok bs=1M count=1
500 MB / sec
dd if=ok of=/dev/null
630 MB / s
dd if=/dev/zero of=ok bs=1 count=1
40/43 kB / sec (Yes, KB)dd if=ok of=/dev/null
0+1 records in
0+1 records out
1 byte (1 B) copied, 1.5e-05 seconds, 66.7 kB/sdd if=./out of=/dev/null
20971520+0 records in
20971520+0 records out
10737418240 bytes (11 GB) copied, 49.5834 seconds, 217 MB/sdd if=/dev/zero of=ok bs=1k count=1
44 MB / secdd if=ok of=/dev/null
2+0 records in
2+0 records out
1024 bytes (1.0 kB) copied, 1.7e-05 seconds, 60.2 MB/srm -f ok ; dd if=/dev/zero of=ok bs=1k count=4
4+0 records in
4+0 records out
4096 bytes (4.1 kB) copied, 3.4e-05 seconds, 120 MB/s
120/ 130 MB /srm -f ok ; dd if=/dev/zero of=ok bs=1k count=16
16+0 records in
16+0 records out
16384 bytes (16 kB) copied, 8.4e-05 seconds, 195 MB/stime ( for (( i=0; i < 100000 ; i++ )) ; do dd if=/dev/zero of=$i.out bs=1k count=1 2> /dev/null; done )
real 10m0.205s
user 1m45.403s
sys 8m2.134stime ( for (( i=0; i < 100000 ; i++ )) ; do dd if=/dev/zero of=$i.out bs=1 count=1 2> /dev/null; done )
real 3m19.841s
user 1m43.242s
sys 1m34.582stime ( for (( i=0; i < 100000 ; i++ )) ; do dd if=/dev/zero of=$i.out bs=1 count=1 2> /dev/null; done )
real 10m52.149s
user 1m46.703s
sys 9m4.218stime ( for (( i=0; i < 1000 ; i++ )) ; do dd if=/dev/zero of=$i.out bs=1
count=1 2> /dev/null; done )real 0m20.931s
user 0m1.068s
sys 0m3.260s
And now, parts of the results on RAID 50 :
dd if=/dev/sda of=/dev/null bs=1M count=10240
Password:
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 40.1916 seconds, 267 MB/sdd if=/dev/zero of=out bs=1M count=10240
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 117.762 seconds, 91.2 MB/sdd if=/dev/zero of=ok bs=1M count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.085963 seconds, 12.2 MB/sdd if=ok of=/dev/null
2048+0 records in
2048+0 records out
1048576 bytes (1.0 MB) copied, 0.001741 seconds, 602 MB/sdd if=/dev/zero of=ok bs=1 count=1
1+0 records in
1+0 records out
1 byte (1 B) copied, 3.5e-05 seconds, 28.6 kB/s
dd if=ok of=/dev/null
0+1 records in
0+1 records out
1 byte (1 B) copied, 1.7e-05 seconds, 58.8 kB/sdd if=/dev/zero of=ok bs=1k count=1
1+0 records in
1+0 records out
1024 bytes (1.0 kB) copied, 3.4e-05 seconds, 30.1 MB/sdd if=ok of=/dev/null
2+0 records in
2+0 records out
1024 bytes (1.0 kB) copied, 1.9e-05 seconds, 53.9 MB/stime ( for (( i=0; i < 1000 ; i++ )) ; do dd if=/dev/zero of=$i.out bs=1M count=1 2> /dev/null; done )
real 0m15.759s
user 0m1.068s
sys 0m3.180stime ( for (( i=0; i < 100000 ; i++ )) ; do dd if=/dev/zero of=$i.out bs=1k count=1 2> /dev/null; done )
real 9m46.112s
user 1m44.779s
sys 7m59.798s
So, my conclusion (which I repeat, is not rocket science) is that RAID 50 isn’t that much better than RAID 5, and is sometimes actually worse, though more expensive. As a result, my personal choice has been RAID 5.