Sat, 18 Dec 2010
HW RAID versus SW RAID
As some of you know, I am not a huge fan of hardware RAID. In my opinion, software RAID can be faster in most cases. This is mainly because the operating system these days has several orders of magnitude larger buffer cache. This means more space for sorting, reordering, and prioritizing requests, fine tailored to the individual disks. Moreover, filesystems like XFS can be teached about the RAID structure of the underlying block device, and can optimize requests based on this knowledge.
The advantages of hardware approach are elsewhere: HW RAID box is usually well tied to the hardware, so for example the disk slot numbers actually match the numbers reported by the storage controller software, it can have battery-backed cache, etc. On the other hand, SW RAID is better tied to the operating system, which can see the SMART health data of the disks, uses standard means of reporting drive failures, etc. HW RAID controllers differ on a vendor-by-vendor basis in reporting, configuring, etc.
Yesterday I have been able to verify the above claims on a real iron: I have a box with LSI Fusion MPT SAS controller, and several 2TB WDC RE-4 black drives. So I have configured a HW RAID-10 volume using 10 disks, and then a SW RAID-10 volume using 10 disks. The initial measurement (after the RAID resync has finished) are here:
time mkfs.ext4 /dev/md0 # SW RAID real 8m4.783s user 0m9.255s sys 2m30.107s time mkfs.ext4 -F /dev/sdb # HW RAID real 22m13.503s user 0m9.763s sys 2m51.371s time sh -c 'dd if=/dev/zero bs=1M count=10000 of=/hwraid/bigfile; sync' real 1m22.967s user 0m0.005s sys 0m11.898s time sh -c 'dd if=/dev/zero bs=1M count=10000 of=/swraid/bigfile; sync' real 0m36.771s user 0m0.008s sys 0m11.224s
I plan to do more tests (with XFS and BTRFS) later.