PCI passthrough with Citrix XenServer 6.5

4.2. Pass the USB 2 controller of the server to the Debian virtual machine

To "pass" a PCI device of the server to a virtual machine, you must know its ID.

To begin, in the XenServer console, type the command :

Bash

lspci

This command will display a list of PCI devices including the USB2 controller of our server :

Plain Text

02:03.0 USB Controller: VMware USB2 EHCI Controller

"02: 03.0" is the PCI device ID on physical server side.

Then, to pass a PCI device to a virtual machine, you must first hide it of the physical server.
To do this, edit the "/boot/extlinux.conf" file to add the "pciback.hide(02:03.0)" option.

Bash

vi /boot/extlinux.conf

Note : to hide many PCI device, use this syntax : pciback.hide=(02:02.0)(02:03.0)

Then, type these commands :

Bash

extlinux -i /boot
reboot

Once XenServer has rebooted, reconnect XenCenter to XenServer and go to properties of the Debian virtual machine.
In the "Boot Options", add the "iommu=soft" option.

For boot options, there are two options available.
As indicated in the Xen wiki :

• the "swiotlb=force" option is required only for para-virtualized machines with older kernels (linux-2.6.18-xen, RHEL5, SLES10/11 et Debian etch/lenny).
• for recent kernels (which is our case with the 3.4.9 kernel), you must only use the "iommu=soft" option.

Still in the XenServer console, type the "xe vm-list" command to list virtual machines available on your server.

Bash

xe vm-list

To pass 1 PCI device to a virtual machine

Use the "xe vm-param-set other-config:pci=0/0000:[PCI ID] uuid=[VM UUID]" command to pass the desired PCI device to the desired virtual machine.

Bash

xe vm-param-set other-config:pci=0/0000:02:03.0 uuid=1f08f35f-12ba-321a-49f4-3ac187776f38

Infos :

• the value "0/0000:" is followed by the device ID to "pass" to the virtual machine. Here the PCI device : 02:03.0.
• the UUID is the UUID of the virtual machine to which you want to pass this PCI device.

To pass multiple PCI devices to a virtual machine

Use the "xe vm-param-set other-config:pci=0/0000:[a PCI ID],0/0000:[another PCI ID] uuid=[VM UUID]" command.

Bash

xe vm-param-set other-config:pci=0/0000:02:02.0,0/0000:02:03.0 uuid=1f08f35f-12ba-321a-49f4-3ac187776f38

Where "02:02.0" is the audio controller and "02:03.0" is the USB2 controller of the server.

To remove PCI devices passed to a virtual machine

Use the "xe vm-param-remove uuid=[VM UUID] param-name=other-config param-key=pci" command.

Bash

xe vm-param-remove uuid=2b1bd48c-fb42-93bb-f91b-f7cbeaf3e0ac param-name=other-config param-key=pci

Start the Debian virtual machine and make sure the "Device Offline" message doesn't appear at boot.
As you can see, Debian shows us this :

Plain Text

... pcifront pci-0: Could not claim resources 0000:00:00.0/0! Device offline. Try using e820_host=1 in the guest config.

And warnings about a 32bit resource.

La solution indiquée dans cette erreur n'a pas fonctionné dans notre cas, et après avoir chercher une solution sur Internet, nous avons trouvé ce signalement de bug :

The solution indicated in this error didn't work in our case, and after having search a solution on Internet, we found this bug report : Bogue 1673 - 3GB memory restriction for pv domU with PCI

As noted in this report, this problem occurs when the guest OS (Debian in our case) has more than 3 GB of RAM. This is indeed the case.
So, we shutdown the Debian virtual machine and have set 2 GB (2048 MB) of RAM for this virtual machine.

Then, start the virtual machine, and the error has disappeared.

We log in as root on the Debian virtual machine and and we list PCI devices with the "lspci" command.

Bash

root@debian:~# lspci
00:00.0 USB Controller: VMware USB2 EHCI Controller

As you can see, the PCI device has a different ID in the virtual machine (00:00.0) and in the physical server (02:03.0).

Then, we list USB devices with the "lsusb" command.

Bash

root@debian:~# lsusb
Bus 001 Device 002: ID 0930:6545 Toshiba Corp. Kingston DataTraveler 102 Flash Drive / HEMA Flash Drive 2 GB / PNY Attache 4 GB Stick
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

So, Debian recognizes our USB key physically connected to the XenServer server.

To use the USB key, simply mount the USB key using the "mount" command.
But for that, you must know the path to the partition of the USB key.

To know it, type the "fdisk -l" command.
This command shows us :

• the hard disk of the virtual machine. Including its size : 53.7 GB.
• and the 16GB USB key

As you can see on this picture, the partition of our USB key is "/dev/sda1" and the file system is FAT32 (compatible with Windows and Linux).

We mount the USB key with these linux commands :

Bash

mkdir /mnt/my-usb-key
mount /dev/sda1 /mnt/my-usb-key

We go to the directory "/mnt/my-usb-key" and list its files.
And as you can see on this image, the "tutorial-iw.txt" file present on our USB key is displayed.
Try to open this file with vi.

Its content is displayed.
Our virtual machine has direct access to the USB key plugged into the XenServer server through the "PCI passthrough" technology.