When you use a single hypervisor with local storage, you have the ability to manage the prioritization for CPU and RAM usage (which are part of your hypervisor hardware), but also the prioritization for I/O at the local storage level of your server.
However, in a VMware vSphere infrastructure with multiple VMware ESXi hosts, you will be using central storage (shared storage) and therefore you need the VMware vSphere SIOC (Storage I/O Control) feature to be able to manage I/O prioritization at the level of storage that is shared by your hosts.
Thus, the prioritization of inputs/outputs will no longer be carried out by host, but by datastore (regardless of the host where each virtual machine is located).

1. What is the VMware vSphere SIOC (Storage I/O Control) feature used for?
2. How VMware vSphere SIOC (Storage I/O Control) works?
3. Prerequisites for using VMware vSphere SIOC (Storage I/O Control)
4. Enable SIOC (Storage I/O Control) and manage virtual machine prioritization
5. Demonstration of SIOC (Storage I/O Control)

1. What is the VMware vSphere SIOC (Storage I/O Control) feature used for?

The main benefit of the VMware vSphere SIOC (Storage I/O Control) feature is to allow you to prioritize storage I/O for critical virtual machines.
So, in the event of a high I/O load on your shared storage, standard virtual machines will not be able to slow down your critical virtual machines.
This allows you to best consolidate your servers without risking slowing down your critical services.

In the diagram below (from the VMware site), you can see that the Online Store and the Exchange server (which is I/O intensive) can be given priority thanks to SIOC (Storage I/O). Control). (See diagram on the right).
While without SIOC (see diagram on the left), other virtual machines which are not critical may risk using bandwidth at the input/output level at the risk of slowing down your critical applications (in this case, the machines virtual: Online Store and Microsoft Exchange).

Thanks to the diagram below, you can see that:

• without SIOC: users of your critical applications (in this case: online store and Microsoft Exchange) will be unhappy, because these virtual machines will not have the ability to use shared storage as they should.
  Indeed, in the example below, the "data mining" virtual machine has too high a number of inputs/outputs at the storage level, it therefore saturates the storage bandwidth.
• with SIOC: given that you will have prioritized your virtual machines hosting your critical applications, they will be able to use a high number of inputs/outputs if necessary and the other virtual machines will share the rest.

Note that the size of the arrows in the diagram below indicates the amount of I/O used at the storage level by these virtual machines.

2. How VMware vSphere SIOC (Storage I/O Control) works?

VMware vSphere SIOC (Storage I/O Control) dynamically allocates I/O capacity for shared storage usage by multiple VMware ESXi hosts in a VMware vSphere cluster.
These I/O capacities are based on the shares (relative priority) and limits (in I/O per second) defined for storage in the configuration of your virtual machines.

Just like NIOC (which is used for network-level I/O), SIOC (which targets storage) is only active when contention occurs.
To do this, SIOC (Storage I/O Control) monitors the latency for access to your datastores.
When this latency exceeds the threshold defined in percentage or in milliseconds (which by default is 30 ms), SIOC starts so that your critical applications can continue to function correctly by limiting the number of inputs/outputs (I/O) of your non-critical virtual machines.
Note that the latency threshold can be different depending on the cases and this is why the I/O injector (which is part of SIOC) is able to determine this value automatically on certain performance data (including the throughput of crete).

Thanks to SIOC, you no longer need to dedicate specific storage for a single critical application.

Source : vSphere Storage I/O Control for Shared Storage: VMware | United States.

3. Prerequisites for using VMware vSphere SIOC (Storage I/O Control)

To be able to use this VMware vSphere SIOC (Storage I/O Control) feature, you must meet a few prerequisites:

• the desired datastores can only be managed by a single VMware vCenter Server.
• SIOC is supported on datastores connected via iSCSI, NFS (NAS), as well as Fiber Channel (FC).
  However, physical disk mapping (RDM) is not supported.
• the desired datastores cannot contain an extension. That is, each datastore must have been created on a single physical disk. This cannot be composed of several physical disks.
• if you are using storage arrays with automated storage leveling features, check its compatibility with SIOC on the "VMware Compatibility Guide - Storage/SAN Search" page.
• this feature requires the "Enterprise Plus" edition of VMware vSphere.

Source : Storage I/O Control Requirements - VMware Docs.

4. Enable SIOC (Storage I/O Control) and manage virtual machine prioritization

To enable SIOC (Storage I/O Control) on VMware vSphere, select the datastore on which you want to enable this feature.
In our case, we use an iSCSI datastore (which is supported by SIOC, as explained previously).

Click: Actions -> Configure Storage I/O Control.

In the "Configure Storage I/O Control" window that appears, you will see that this SIOC feature is disabled by default.

To enable Storage I/O Control (SIOC), select the "Enable Storage I/O Control and statistics collection" option and set the desired congestion threshold (latency only) in percentage or milliseconds (ms).
By default, the value used will be 90% (of peak flow) or 30 ms.
Note that under vSphere 5 you could only set the value in milliseconds.

The "Include I/O statistics for SDRS" box allows you to include I/O statistics due to SDRS (which allows you to intelligently migrate virtual machines from one VMware ESXi host to another).

Once these options are configured, click OK.

Important : for SIOC to work, it must first analyze the I/O usage on this storage for a certain time and perform certain calculations to then be able to best adapt the I/O usage of your virtual machines on this storage.
It is therefore difficult to test this in the case of a lab.

Once SIOC is activated on a datastore, you will see an "I/O" icon appear.

Note: in recent tasks, you will see the task "Configure Storage I/O Control on the datastore" appear, the target of which corresponds to the datastore concerned.

To manage the prioritization of inputs/outputs of your virtual machines using this datastore, edit the settings of the desired virtual machines.

As explained at the beginning of this tutorial, you will need to use the "Shares" and "Limit - IOPS" options for the virtual hard disks of your virtual machine.

For the shares, you will have the choice between "Low", "Normal" (default), "High" and "Custom".
This allows you to set a relative priority between your virtual machines.
Critical virtual machines that need to be able to use a high number of inputs/outputs if necessary should have a "High" share and the others a "Low" or "Normal" share.

You can also limit the number of default I/O that a virtual hard disk can issue to a datastore by selecting "IOPS Limit: Custom".

Next, specify the number of inputs/outputs (I/Os) that this virtual hard disk will be able to use per second.
Note that by clicking on the arrow on the right, the minimum value and the maximum value will appear.

5. Demonstration of SIOC (Storage I/O Control)

Since SIOC requires analysis and calculation time before it can operate, we suggest you watch the official SIOC demonstration published by VMware on Youtube: Storage I/O Control product demo.