This was one of the best sessions at VMworld 2010. Two realy good presenters and an interesting topic. Storage I/O Control is a new feature of ESX4.1 available only in the Enterprise Plus Edition of vSphere. What this feature does is control disk access to datastores. SIOC is limited to datastores accessible using iSCSI and FC, so blockbased storage. NFS being filebased storage is not yet supported. However, in the VMware labs SIOC control for NFS is already running so expect NFS support to be added soon.
Why is Storage I/O control such an important feature? Because 90% of all performance issues in which VMware is involved is related to storage! Storage can be seen as the bottle neck of virtualization.
Performance problems related to storage can be seen as noisy neighbors. A single virtual machine can dominate the usage of available storage resource in such way other virtual machines have insufficient i/o access. Those VM’s can be running business critical applications while the VM dominating the usage of storage i/o has an un-important role. You like to control this noisy neighbor so he or she becomes much more quiet!
Storage I/O control is difficult to develop. It took VMware 3.5 years to develop this feature. And it is very simple to enable. SIOC is disabled by default. It can be enabled by selecting a datastore and enable SIOC. The recommended setting is to enable it.
As soon as SIOC is enabled, access to storage is equally divived over the virtual machines sharing the same datastore which is SIOC enabled. So a VM dominating I/O will get less I/O than before SIOC was enabled. To control priorization you can use shares or set the number of IOPS a virtual machine can maximum get.
SIOC starts becoming active (kicks in) when the latency is 30ms or more. VMware found out that 30ms is on average the limit for acceptable performance. Latency more than 30ms results in bad performance. It does not make sense to activate SIOC earlier. It is like a VIP lane on a highway. When there is no congestion, so traffic runs smoothly, the usage of a VIP lane or carpool lane does not have any effect. The same counts for SIOC.
Using shares the access to storage can be priortized. Access is not a static thing. If a virtual machine which a high number of shares, so with a relative high priority is actualy not consuming I/O, other VM’s having a lower share, will get those I/O’s. So SIOC is dynamically assigning I/O based on actual consumption. To control this the host device queue depth is adjusted.
The presenters show different slides showing the effect of not using SIOC and using SIOC. Even when SIOC is enabled, access of I/O for vm’s having a high number of shares will drop. The lose of I/O access can be controlled by adjusting the latency value. By lowering the latency the moment SIOC kicks in can be controlled.
A lot more information on SIOC can be found on the excellent Yellow-Bricks.com
Storage I/O Control best practices http://www.yellow-bricks.com/2010/10/19/storage-io-control-best-practices/
Storage I/O Fairness http://www.yellow-bricks.com/2010/09/29/storage-io-fairness/
SIOC, tying up some loose end http://www.yellow-bricks.com/2010/10/08/sioc-tying-up-some-loose-ends/
Info on vkernel.com
What to Expect When You Enable Storage I/O Controls in ESX 4.1 http://blog.vkernel.com/2010/09/what-to-expect-when-you-enable-storage.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+vkernel/FnyZ+(VKernel+Virtualization+Blog)