Here is some file IO performance numbers from DELL MD1120 SAS storage array. Last year I did the same test with HP P800 storage array and numbers were impressive. But when it comes to this high end storage array, few surprises. Before getting into actual details; lets see the test stats and configuration details.
System Configuration:
- DELL R710 with CentOS 5.4
- NOOP IO Scheduler
- MD1120 with 22 15K SAS disks
- 20 disk RAID-10 (hardware)
- 2 hot spares
- Disk Cache disabled
- PERC 6/E RAID controller with BBU
- Connected to DELL MD1120 using SAS
- Write Back
- Read Cache Disabled
Test Configuration:
- Sysbench fileio test with variable modes and threads
- 64 files with 50G total size
- All tests ran in un-buffered mode (O_DIRECT) as most of the workload is InnoDB based.
Test Results:
Number of Threads vs Number of Requests/Sec. Every mode ran with 5 iterations and average is taken.
Random IO:
Sequential IO:
Analysis:
- Overall the numbers are not bad when it comes to writes, but few surprises when it comes to reads. When compared with HP’s P800 storage array, the numbers still dropped by 20%.
- Radon IO:
- Random write requests ranges from 3200-5000 per sec; due to write back mode (512M cache)
- Writes are linearly scaling well with the threads, good sign that controller is able to manage the cache efficiently
- Random reads and writes (rndrw) is also scaling linearly with the threads load, means the IO distribution and cache burst to satisfy reads seems be efficient as it needs to flush the data from controller cache to disk before the read can be satisfied due to O_DIRECT mode.
- Sequential IO:
- Writes seems to be scaling well even in sequential mode without much overhead
- When it comes to reads, big surprise is drop from 5626 requests/sec to 615 from one thread to two threads. Which is really odd. Worst case it should be ~2000-3000 requests/sec; not sure where the overhead is. I can’t believe it could be thread scheduling as there is only 2 threads.
- During 100% IO, on and off I noticed IO serialization with higher queue waits, which indicates that there is some degree of serialization overhead in OS; but not able to track which layer is triggering this. Tried with cfq/deadline, still the same.
- Next attempt will be replacing 3Gb/s SAS to fiber channel HBA or 6Gb/s SAS (PERC H800) to see how it performs along with combination of HW and SW raid instead of only depending on controller.
PlanetMySQL Voting: Vote UP / Vote DOWN