As explained in previous tutorials, VMware vSphere vMotion allows you to migrate virtual machines between 2 VMware ESXi hosts. Whether these are in the same data center or not.
But, thanks to VMware vSphere vMotion, you will also be able to migrate virtual machines from one VMware vCenter Server to another. With the VMware vSphere Enterprise Plus license.

1. Network configuration
2. Prerequisites for migration between VMware vCenter Servers (VCSA)
3. Network requirements
   1. Ports used
   2. Use the same IP addressing (L2/L3 bridges)
4. Enable vMotion on your VMware ESXi hosts
5. Test network latency
6. Migrate a virtual machine between 2 VMware vCenter Servers
7. Reconfigure the virtual machine network (if the IP addressing is different)

1. Network configuration

The network configuration and VMware vSphere infrastructure used in this case are the same as those created in our tutorial: VMware vSphere 6.7 - Enhanced Linked Mode (ELM).

In other words, a company located in 2 geographic locations:

• Brussels (in Belgium) with a VMware vCenter Server "brux-vcsa" and 2 VMware ESXi hosts: brux-esxi1 and brux-esxi2
• Paris (in France) with a VMware vCenter Server "paris-vcsa" and 2 VMware ESXi hosts: paris-esxi1 and paris-esxi2

Note that the 2 VMware vCenter Server servers mentioned above are part of the same vSphere SSO domain and that these servers are therefore linked (ELM mode).

For more information on creating a multi-site Active Directory infrastructure on which this tutorial is also based, refer to our tutorial : WS 2016 - AD DS - Configure a multi-site Active Directory infrastructure.

2. Prerequisites for migration between VMware vCenter Servers (VCSA)

The prerequisites are:

• vCenter Server and source and destination ESXi host must be version 6.0 or later.
• migration from one vCenter Server to another and long-distance vMotion migration require an Enterprise Plus license.
• network latency for long distance migrations should be 150 milliseconds maximum.
• the clock on your vCenter Servers must be synchronized (for example, from the time server (NTP) present in any Active Directory infrastructure) for authentication to work correctly.
• if you use the web client, the 2 vCenter Servers must be in Enhanced Linked Mode (ELM) and must therefore be in the same SSO domain.
  This will allow the source vCenter Server to authenticate with the destination vCenter Server.
• if you use the VMware vSphere APIs/SDKs, your 2 vCenter Servers can be in different vSphere SSO (vSphere Single Sign-On) domains, but this will require the use of additional settings.

Optional: if you want to be able to migrate only the computing resource without migrating the storage of a virtual machine, the virtual machine must be on shared storage accessible by your 2 vCenter Servers.

Warning : a virtual machine connected to a distributed switch cannot be migrated to a standard switch.
On the other hand, migration is possible if the type of destination source switch is identical.
Migration is also possible if the source switch type is Standard and the destination switch type is a distributed switch (but not vice versa).

Sources :

• Cross vCenter Migration and Clone requirements in VMware vSphere 6.x and later (2106952)
• vSphere vMotion Networking Requirements - VMware Docs

Important : for other prerequisites specific to vMotion, refer to our tutorial: Migrate virtual machines (VMs) via vMotion on VMware vSphere 6.7.

3. Network requirements

3.1. Ports used

For the network prerequisites, it is necessary that::

• ports 8000 and 902 are open for vMotion and NFC between ESXi hosts.
• port 443 is open between vCenter Servers.
• port 443 is open between the vCenter Server and the ESXi host. Which is also necessary to be able to add the ESXi host in vCenter Server.

Source: VMware KB 2106952 cited previously.

3.2. Use the same IP addressing (L2/L3 bridges)

To ensure that the migrated virtual machines continue to be able to access the network and the Internet (if applicable) without any problems, it is best to have the same IP addressing on the source network and the destination network.
Otherwise, once the virtual machines are migrated, they are likely to no longer have access to the network since their network configuration will no longer correspond to the IP address used on the destination network.

To solve this problem, you can use L2 (Layer 2) or L3 (Layer 3) bridges.
These L2 or L3 bridges will allow you to use the same IP addressing on the network of the source site (Site A below) and that of the destination (Site B).

Source : L2 Bridges - VMware Docs.

As "Sreekanth Setty" indicates in the comments in the article "VMware pushes the envelope with vSphere 6.0 vMotion":

• before vSphere 6, the network dedicated to vMotion requires an L2 (Layer 2) bridge
• from vSphere 6, the network dedicated to vMotion requires an L3 bridge (Layer 3)

Note that Sreekanth Setty is a member of the Performance Engineering team at VMware.

The use of L2 and L3 bridges is possible via VMware NSX (NSX-T feature): Network Transport - VMware Docs.
VMware NSX being a virtualization solution allowing you to define a network in software to extend it across data centers, the Cloud, etc.
With the NSX Gateway, you will be able to bridge Layer 2 VLANs from the physical network to an NSX network.
Source: What is VMware NSX? | Network Security Virtualization Platform.

But, also at Cisco : Layer 3 vs Layer 2 Switching.

4. Enable vMotion on your VMware ESXi hosts

As explained at the beginning of this tutorial, in our case, the 2 VMware vCenter Servers are located in different countries.
Although these are able to communicate with each other via VPN gateways.
To summarize, thanks to the VPN gateways implemented on each physical site, we are able to access any computer or server of the remote site (for example: Paris) from any computer or server of the source site (for example: example: Brussels).

For vMotion to work between your VMware vCenter Servers, it is recommended that they be linked together via ELM (Enhanced Linked Mode) as explained previously.
When you want to use vMotion, it is recommended to use a dedicated network for this feature. As we explained in our "Migrate virtual machines (VMs) via vMotion on VMware vSphere 6.7" tutorial.
However, in our test environment we only have one connection to the Internet. So the traffic would go through that one connection to the Internet anyway.

For this tutorial, we will therefore activate the "vMotion" service on the VMkernel "vmk0" adapter created by default during the installation of VMware ESXi.
To do this, in the VMware vSphere Client, select the source VMware ESXi host and go to: Configure -> Networking -> VMkernel adapters.
Next, select the VMkernel adapter "vmk0" that appears and click Edit.

Note: as you can see, for the moment, the only enabled service is: Management.

In the "vmk0 - Edit Settings" window that appears, additionally check the "vMotion" box and click OK.

As expected, the services enabled on this VMkernel adapter are now: vMotion and Management.

Do the same on the remote VMware ESXi host.

The vMotion service is also enabled on the remote VMware ESXi host's VMkernel adapter.

5. Test network latency

As explained previously in the prerequisites, the maximum latency for long distance migrations must be a maximum of 150 milliseconds.
To know the latency between your 2 physical sites in your case, activate the SSH protocol on the source VMware ESXi host and connect to it with an SSH client (for example: putty).
In our case, we will connect to our server "brux-esxi1".

To test network connectivity between this source and destination VMware ESXi host, use the "vmkping" command specifying:

• -I vmk0: indicates that this command must use the VMkernel interface "vmk0".
  In other words, the one where you activated the vMotion service.
• [domain name or IP address]: indicates the domain name or IP address used to contact the remote server from the source physical site

Important : by default, this command uses the default TCP/IP stack of your source VMware ESXi host.
If you have configured your VMkernel interface to use the "vMotion" TCP/IP stack, you will need to add the "-S vmotion" parameter.

Bash

vmkping -I vmk0 paris-esxi1.informatiweb.lan

Plain Text

PING paris-esxi1.informatiweb.lan (10.0.2.13): 56 data bytes
64 bytes from 10.0.2.13: icmp_seq=0 ttl=62 time=0.740 ms
64 bytes from 10.0.2.13: icmp_seq=1 ttl=62 time=0.722 ms
64 bytes from 10.0.2.13: icmp_seq=2 ttl=62 time=0.828 ms

--- paris-esxi1.informatiweb.lan ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.722/0.763/0.828 ms

Warning : in our case, this is a completely virtual infrastructure with VPN gateways and virtual switches to simulate 2 physical sites that would be connected to each other via this type of VPN device.
However, in reality, everything runs on a single powerful computer. Hence the latency which is extremely low compared to what it would give in reality.