Enable Jumbo frame support on VMware ESXi 6.7 - VMware - Tutorials

On VMware ESXi 6.7, you have the option of using Jumbo Frames to improve network performance when large amounts of data need to be transferred over the network.
This is particularly the case when migrating virtual machines via vSphere vMotion and what is recommended by VMware in this same case.

The goal is to be able to send much larger network packets (up to 9000 bytes) over the network instead of the usual small packets (up to 1500 bytes).
However, it's important that jumbo frame support is enabled end-to-end for the entire path that packets will take.
This means that you will have to enable support for Jumbo Frames on the source virtual machine, the destination virtual machine, on the virtual switches and on the physical switches (switches and routers) that would be between these virtual machines.
If you want to use jumbo Frames for hypervisor traffic (in the case of vMotion, for example), remember to also enable support for jumbo Frames on the relevant VMkernel interfaces.

Sources :

1. Default MTU

In this tutorial, we used 2 hypervisors on VMware ESXi 6.7.0 u3 with a Windows 10 v2004 x64 virtual machine on each of them.
As you can see, the virtual network adapter used by default with this guest operating system is : Intel(R) 82574L Gigbit Network Connection.
If you go to the properties of this virtual network adapter, you will see that jumbo frame support is disabled by default.

Although the driver for this network adapter allows support for Jumbo Frames, VMware indicates in its official documentation that you must use a VMXNET 2 (Enhanced) or VMXNET 3 virtual network adapter.

If you view the information of the virtual switch (vSwitch0) where your virtual machines are connected by default through the "VM Network" port group, you can see that the MTU is 1500.
Which means the size of network packets can't exceed 1500 bytes at this time. This means that Jumbo Frames will not be able to pass as is. In this case, the switch will fragment the packet sent into small packets of 1500 bytes maximum.

Note that for traffic between VMware ESXi hypervisors, an MTU is also applied.

Indeed, if you try to modify the settings of a VMkernel interface (in this case : vmk0), you will see that an MTU of 1500 is defined by default.

2. Configure virtual hardware for your virtual machines

To use Jumbo Frames with a virtual machine, you must edit the configuration of the virtual machine to use a "VMXNET 2 (Enhanced)" or "VMXNET 3" virtual network adapter.

Note : the use of a "VMXNET 3" network card on Windows requires VMware Tools to be installed.
If you are creating a new virtual machine, leave the network adapter chosen by default and only choose the "VMXNET 3" network card once the guest operating system and VMware Tools are installed.

Sources :

In the "Network Adapter 1" (by default) section, change the adapter type and choose "VMXNET 3" (or "VMXNET 2 (Enhanced)" if version 3 is not available in your case).

Note that you can only change the network adapter type when the virtual machine is powered off. Changing the type of network adapter keeps the MAC address that was associated with this virtual machine.
Which wouldn't be the case if you removed the old virtual network adapter to add a new one in VMXNET 3.

The "Virtual machine [VM name] was successfully reconfigured" message appears.

Do the same on the destination VM.

3. Configure the virtual network of your VMware ESXi hypervisor

As you will have understood, in this tutorial, we will send Jumbo Frames between our virtual machines on Windows 10 v2004 x64.

As explained earlier, to use Jumbo Frames, you must enable end-to-end Jumbo Frame support.
By default, your virtual machines are connected to the "VM Network" port group created on the "vSwitch0" virtual switch.

If you view the information of this virtual switch "vSwitch0", you will see that an MTU of 1500 bytes is set on it by default.
You will also see that your virtual machines are connected to the "VM Network" port group present on this virtual switch.

If you try to edit this "VM Network" port group, you will see that the "MTU" setting doesn't appear.
The port group therefore doesn't need to be changed.

On the other hand, if you try to edit the settings of your virtual switch "vSwitch0", you will see that the MTU is 1500 bytes by default.
In order for Jumbo Frames to pass through this "vSwitch0" virtual switch properly, you need to set the MTU to 9000.
Then, click Save.

Your "vSwitch0" virtual switch now has an MTU of 9000 bytes.

As a reminder, if you want to use Jumbo Frames for network traffic between VMware ESXi hypervisors (in the case of vMotion, for example), you will also have to change the MTU in the settings of the VMkernel interface concerned.
To do this, go to "Networking -> VMkernel NICs", then click on the name of the VMkernel interface you want to edit.

Then, click on : Edit settings.

By default, the MTU of VMkernel interfaces is also 1500 bytes.
If you want it to support Jumbo Frames, set its MTU to 9000, then click Save.

Note that changing the MTU on VMkernel interfaces is optional.
Indeed, you can set an MTU of 9000 on the vSwitch (to support Jumbo Frames between VMs) and leave the VMkernel MTU of 1500 on the VMkernel interfaces (if you don't need Jumbo Frames support for network traffic between VMware ESXi hypervisors) without any problems.
However, the reverse is not possible. If your VMkernel interface must support Jumbo frames, it's necessary that the MTU be set to 9000 on this VMkernel interface, as well as on the vSwitch concerned.

Source : Enabling Jumbo Frames on virtual switches (1038827) - VMware KB.

The "Configuration for VMkernel NIC vmkX was saved " message appears.