Storage issues on Windows Server 2012 Hyper-V. Microsoft struggling to fix!
March 12, 2013 1 Comment
Storage is a crucial component of any virtualization platform. Issues on storage will have an effect of performance and availability of virtual machines.
In Windows Server 2008 Cluster Shared Volumes (CSV) had a couple of issues. I remember redirected io issues during backup operations, the way iSCSI is used to handle reservations of volumes (see this issue) and third party drivers (McAfee Antivirus, see this issue) could harm performance and/or reliability.
It is always a good idea to fully test your backup solution before implementing Windows Server 2012 Hyper-V in your production environment.
Microsoft improved quite a bit on CSV storage in Windows Server 2012 Hyper-V. Also integration between storage arrays and Hyper-V was improved. ODX or Offloaded Data Transfers improve copy performance by offloading the data copy to the storage array instead of doing the work by the Hyper-V host.
ODX enables a large improvement in duration on actions like a storage migration or regular copy of a file.
However still some issues are being noticed.
One of the issues is related to ODX. As ODX is a new interfacing technology it needs to be implemented in the storage array firmware. Not all storage vendors support ODX at the moment. However ODX is default enabled in Windows Server 2012. This might cause issues. Read Aidan Finn blogpost titled Enabled ODX In Hosts With Non-Supporting SAN Can Cause CSV Stability Issues on this.
Veeam Backup & Replication does not like ODX either. It will disable ODX as can be read in this thread. VeeamFCT is a service installed on each Hyper-V host which VMs are protected by Veeam Backup & Replication. It handles Changed Block Tracking (CBT) by installing a driver (VeeamFCT.sys) so only changed blocks of data in the virtual machine disk file are processed during backup. Veeam is working on a fix as can be read in this thread.
Last but not least there has been a major issue on backup of CSV. This could cause VMs to go in pause of CSV volumes to go offline. Microsoft needed several patches to get this fixed and it still seems to be not fixed for all users. I blogged about this here.
An overview of fixes:
KB2799728 did not fix the backup issues as it had a memory leak .
workaround is to de-install the KB2799728 and enable per host and lun serialization in dpm ( http://technet.microsoft.com/en-us/library/hh757922.aspx )
Then Microsoft released KB 2813630 ( available here.) which is supposed to solve both backup issue and memory leak. However users in this Microsoft forum are complaining that this fix of fix does not fix the issues either. They are awaiting v3 of KB2813630.
update June 13
Around the 15th of May Microsoft released a new, important bugfix which solved many issues. Read my post for more information.