Improving SCSI performance on ESXi Whitebox

OK, the upgrade to SCSI went well, I converted from a SATA to SCSI system.

Added the following components:

• IBM ServeRAID 6M, PCI-X card
• 1 36GB drive to use as boot, and ISO storage
  
• 5 74GB drives in RAID5 to be used as main datastore (272 GB)
• 1 500GB SATA drive from old ESXi Server. Source Datastore to move data onto RAID5

Now everything was going well, but was noticing that the speed performance was slow. Now I expected it to be slower with writes on RAID5, but I was expecting the reads to scream.

That was not what was happening. So I started using HD_Speed and noticed some things.

• Read speed on SATA drives 115
• Read Speed on SCSI RAID5 4.5

That was a big difference. So I did some digging and did the following changes to make some improvements

• Moved the ServerRAID 6M to PCI-X slot 1 because it was 133 Mhz, Removed the other SCSI card because if there was anything in Slot 2, it downgrade to 100 Mhz and the card was only rated at 133.
• Made sure that the Cache was on
• Upped the queue limit from 64 to 96 into the BIOS

After I did that the Read speed moved up to 45. Not bad. One other thing that I noticed is that when you move to larger downloads like 4MB it would move up to the 100 to 200 range.

Also I could benchmark 2 servers simultaneously from the SCSI RAID5 and it did not impact each other.