• Persistent memory (PMem) on a virtual ESXi
• VMware Host Client web client presentation

Thanks to the DirectPath I/O (better known under the name of "PCI passthrough") supported by the VMware ESXi hypervisor, you have the possibility of passing different PCI components, compatible graphics cards (with a specific additional configuration), USB controllers, ... to a virtual machine.
This allows you to use some components of the physical computer / server from a virtual machine.

1. What is DirectPath I/O
2. Advantage of DirectPath I/O
3. Disadvantages of DirectPath I/O
4. Prerequisites
5. Enable the passage of a PCI device on your computer / server on VMware ESXi 6.7
6. Configure the virtual machine to pass it a PCI device
7. PCI device visible in virtual machine

1. What is DirectPath I/O

DirectPath I/O allows a virtual machine to use a compatible physical device from the host.

Using DirectPath I/O, you can also divide a physical device into multiple virtual cards.
This is particularly the case for network adapters supporting this DirectPath I/O technology and for NVIDIA GRID graphics cards which allow the allocation of virtual graphics cores (vGPU) rather than passing the entire graphics card to a virtual machine.

Finally, you can also pass physical USB dongles which are sometimes used as a license for some professional software.
In this case, it will suffice to pass the appropriate USB controller to the virtual machine if passing the USB dongle as a USB device doesn't work in your case.

2. Advantage of DirectPath I/O

Using DirectPath I/O, you can improve the network or graphics performance (depending on the type of DirectPath I/O device used) of one or more virtual machines.
Indeed, the virtual machine will be able to directly use the physical device concerned without going through the VMware ESXi virtualization layer.

3. Disadvantages of DirectPath I/O

Warning : although DirectPath I/O can be interesting in some cases, using DirectPath I/O on a virtual machine will also prevent you from using many features of VMware vSphere :

• you can't suspend (pause) a virtual machine or restore (play) it.
• unable to hot add or remove virtual devices.
• no fault tolerance as Fault Tolerance (FT) will no longer be available.
• unable to create snapshots to quickly save virtual machine state
• you can't migrate this virtual machine via vMotion since it's attached to a physical device on the VMware ESXi host where it currently resides.
• no high availability (HA) possible

In other words, only use DirectPath I/O if you really need it. Because it can quickly complicate the creation of backups or fault tolerance.

Sources :

• Enable DirectPath I/O for NVIDIA GRID on a Host - VMware Docs
• DirectPath I/O - VMware Docs
• vSphere VMDirectPath I/O and Dynamic DirectPath I/O: Requirements for Platforms and Devices (2142307)

4. Prerequisites

Using DirectPath I/O requires :

• the use of hardware compatible with VMware ESXi.
• using devices compatible with this DirectPath I/O technology.
• the support of Intel VT-d or IOMMU at AMD by the processor of your server.
• enabling this support (Intel VT-D / AMD IOMMU) in your server's motherboard BIOS.

5. Enable the passage of a PCI device on your computer / server on VMware ESXi 6.7

To activate the passage of a PCI device of your computer / server on VMware ESXi 6.7, connect to the web interface of this one and go to : Host -> Manage.

Then, go to the "Hardware" tab and find the desired component.
In our case, we searched for the keyword "USB" to find the "ASMedia Technology Inc. ASM2142 USB 3.1 Host Controller" corresponding to the USB 3.1 port on the top of our "Asus WS X299 SAGE/10G" motherboard.
For the moment, VMware ESXi tells us that the "passthrough" is disabled. Which means that for the moment, VMware ESXi sees this device and that if we plug our USB key into a USB port of this controller, it will be available from the VMware ESXi hypervisor.

To hide this USB controller from our VMware ESXi 6.7 hypervisor and therefore make it usable by any virtual machine, select this PCI device and click on "Toggle passthrough".

Notes : PCI devices shown in gray can't be passed to a virtual machine because this is not compatible in your case. As indicated by VMware ESXi in the "passthrough" column.

After passthrough has been enabled or disabled for a device, you will need to reboot the host (the VMware ESXi hypervisor) for the change to take effect.
As stated in the warning that appears :

Plain Text

Successfully toggled passthrough for device ASMedia Technology Inc. ASM2142 USB3.1 Host Controller. Changes made will not take effect until the host is restarted.

To do this, click on the "Reboot host" button.

Ignore the fact that the VMware hypervisor is not in maintenance mode if you are sure that no virtual machines are running and none of them can be started by someone else remotely, by clicking on : Restart.

If you go back to "Host -> Manage -> Hardware", you will see that the passthrough is now enabled for the desired PCI device.

6. Configure the virtual machine to pass it a PCI device

After passthrough is enabled on the desired PCI device, edit the desired virtual machine.

And click : Add other device -> PCI device.

A new line "New PCI device" appears with the list of PCI devices available on the host (the VMware ESXi hypervisor).
In our case, there is only our controller "ASM2142 USB 3.1 Host Controller - 0000:03:00:0".

Important : as soon as you pass a PCI device from the host to a virtual machine, you are required to reserve all the RAM allocated to it in order to start the virtual machine.
To do this, in the "Memory" section of your virtual machine's virtual hardware, check the "Reserve all guest memory (All locked)" box.

In other words, when starting the virtual machine, the amount of RAM indicated here will be entirely reserved for this virtual machine when it has started.

The "Virtual machine [VM name] was successfully reconfigured" message appears.
Now, you can start your virtual machine by clicking on "Power on".

If you have not reserved the RAM (random access memory) allocated to your virtual machine in its virtual hardware, your virtual machine will not be able to start and this error will be displayed :

Plain Text

Failed to power on virtual machine [VM name]. Invalid memory setting: memory reservation (sched.mem.min) should be equal to memsize(4096). Click here for more details.

7. PCI device visible in virtual machine

Once the virtual machine is started, you will be able to see that the passed PCI device is visible to the guest operating system.
Note that this may require manual installation of the associated driver in the operating system if the latter is not natively supported by this operating system.
In our case, Windows 10 automatically installed the driver for it, so manual installation of an additional driver is not necessary.

For this tutorial, we had plugged an USB key into the USB 3.1 port of this controller and as expected, it therefore appears in the virtual machine.
But, the advantage of DirectPath I/O (or "PCI passthrough") in this case is that we can unplug the USB key and plug it in again later without having to reconfigure the virtual machine, since we are passing the USB controller directly and not the USB key itself.

If you have passed any other type of PCI device, you can see it in the device manager.
To do this, in Windows file explorer, right click "Manage" on "This PC" (or "Computer" depending on your version of Windows).

Click on "Device Manager" and you will find your PCI device on the right side.
In our case, in the "Universal Serial Bus controllers" section, we can find our "ASMedia USB 3.1 eXtensible Host Controller - 1.10 (Microsoft)".