Live migration refers to the process of moving a running

virtual machine In computing, a virtual machine (VM) is the virtualization/ emulation of a computer system. Virtual machines are based on computer architectures and provide functionality of a physical computer. Their implementations may involve specialized h ...

(VM) or application between different physical machines without disconnecting the client or application. Memory, storage, and network connectivity of the virtual machine are transferred from the original guest machine to the destination. The time between stopping the VM or application on the source and resuming it on destination is called 'downtime'. When the downtime of a VM during live migration is small enough that it is not noticeable by the end user, it is called a 'seamless' live migration.

Live migration of Virtual Machines

Two techniques for moving the virtual machine's memory state from the source to the destination are pre-copy memory migration and post-copy memory migration.

Pre-copy memory migration

Pre-copy phase

In pre-copy phase, the

Hypervisor A hypervisor (also known as a virtual machine monitor, VMM, or virtualizer) is a type of computer software, firmware or hardware that creates and runs virtual machines. A computer on which a hypervisor runs one or more virtual machines is called ...

copies all the memory pages from source to destination while the VM is still running on the source. If some memory pages change (become 'dirty') during the pre-copy phase, they will be copied again and again over several 'pre-copy rounds'. Usually the pre-copy phase ends when the number of dirtied pages remaining becomes small enough to yield a short stop-and-copy phase. However, if a VM keeps dirtying memory faster than can be re-copied to the destination, then pre-copy phase will end after a set time limit or maximum number of pre-copy rounds to begin the next stop-and-copy phase.

Stop-and-copy phase

After the pre-copy phase, the VM will be paused on the source host, the remaining dirty pages will be copied to the destination, and the VM will be resumed at the destination. The downtime due to this phase can range from a few milliseconds to seconds depending on the number of dirty pages transferred during downtime. VMs that dirty a lot of memory during the pre-copy phase tend to have a larger downtime.

Post-copy memory migration

Post-copy VM migration is initiated by suspending the VM at the source. With the VM suspended, a minimal subset of the execution state of the VM (CPU state, registers and, optionally, non-pageable memory) is transferred to the target. The VM is then resumed at the target. Concurrently, the source actively pushes the remaining memory pages of the VM to the target - an activity known as pre-paging. At the target, if the VM tries to access a page that has not yet been transferred, it generates a page-fault. These faults, known as network faults, are trapped at the target and redirected to the source, which responds with the faulted page. Too many network faults can degrade performance of applications running inside the VM. Hence pre-paging can dynamically adapt the page transmission order to network faults by actively pushing pages in the vicinity of the last fault. An ideal pre-paging scheme would mask large majority of network faults, although its performance depends upon the

memory access pattern In computing, a memory access pattern or IO access pattern is the pattern with which a system or program reads and writes memory on secondary storage. These patterns differ in the level of locality of reference and drastically affect cache performa ...

of the VM's workload. Post-copy sends each page exactly once over the network whereas pre-copy can transfer the same page multiple times if the page is dirtied repeatedly at the source during migration. On the other hand, pre-copy retains an up-to-date state of the VM at the source during migration, whereas during post-copy, the VM's state is split across the source and the destination. If the destination fails during live migration, pre-copy can recover the VM, whereas post-copy cannot.

VM managers with live migration support

Virtuozzo Virtuozzo is a software company that develops virtualization and cloud management software for cloud computing providers, managed services providers and internet hosting service providers. The company’s software enables service providers to offe ...

* Xen since version 2.0, 2004 for PV guests; since version 3.1, May 18, 2007 for HVM guests *

OpenVZ OpenVZ (Open Virtuozzo) is an operating-system-level virtualization technology for Linux. It allows a physical server to run multiple isolated operating system instances, called containers, virtual private servers (VPSs), or virtual environmen ...

* Parallels Cloud Server *

Workload Partitions AIX Workload partitions (WPARs) are a software implementation of operating system-level virtualization technology introduced in the IBM's AIX 6.1 operating system that provides application environment isolation and resource control. WPARs are s ...

Integrity Virtual Machines Integrity Virtual Machines is software from Hewlett-Packard that allows multiple virtual machines to run concurrently on any Itanium server running HP-UX, notably the HPE Integrity Servers line. It is part of HP's Virtual Server Environment suit ...

* KVM since February 2007, with different techniques. *

Oracle VM Server for x86 An oracle is a person or agency considered to provide wise and insightful counsel or prophetic predictions, most notably including precognition of the future, inspired by deities. As such, it is a form of divination. Description The word ''o ...

Oracle VM Server for SPARC An oracle is a person or agency considered to provide wise and insightful counsel or prophetic predictions, most notably including precognition of the future, inspired by deities. As such, it is a form of divination. Description The word ''o ...

* OVirt * Red Hat Virtualisation * POWER Hypervisor (PHYP) * VMware ESXi * IBM

Live Partition Mobility Live Partition Mobility is a chargeable Live migration feature of IBM POWER6, POWER7, POWER8 and POWER9 servers, available since 2007, that allows a running LPAR to be relocated from one system to another. In concept, it is similar to VMware VMot ...

, since 2007 * Hyper-V Server 2008 R2 *

VirtualBox Oracle VM VirtualBox (formerly Sun VirtualBox, Sun xVM VirtualBox and Innotek VirtualBox) is a type-2 hypervisor for x86 virtualization developed by Oracle Corporation. VirtualBox was originally created by Innotek GmbH, which was acquired by ...

Proxmox Virtual Environment Proxmox Virtual Environment (Proxmox VE or PVE) is an open-source software server for virtualization management. It is a hosted hypervisor that can run operating systems including Linux and Windows on x64 hardware. It is a Debian-based Lin ...

Cloud Platforms with live migration support

Jelastic Jelastic is a cloud platform software vendor that provides multi-cloud Platform as a Service-based on container technology for hosting service providers, ISVs, telecommunication companies, enterprises and developers. The platform is available as ...

Google Cloud Platform Google Cloud Platform (GCP), offered by Google, is a suite of cloud computing services that runs on the same infrastructure that Google uses internally for its end-user products, such as Google Search, Gmail, Google Drive, and YouTube. Alongside ...

CloudEndure CloudEndure is a cloud computing company that develops business continuity software for disaster recovery, continuous backup, and live migration. CloudEndure is headquartered in the United States with R&D in Israel. History CloudEndure was fou ...

Systems providing software live migration

OpenSSI OpenSSI is an open-source single-system image clustering system. It allows a collection of computers to be treated as one large system, allowing applications running on any one machine access to the resources of all the machines in the cluster. ...

MOSIX MOSIX is a proprietary distributed operating system. Although early versions were based on older UNIX systems, since 1999 it focuses on Linux clusters and grids. In a MOSIX cluster/grid there is no need to modify or to link applications with an ...

Single-system image In distributed computing, a single system image (SSI) cluster is a cluster of machines that appears to be one single system. The concept is often considered synonymous with that of a distributed operating system, but a single image may be presented ...

References

{{Reflist

External links

HOWTO Article about Xen migrationOpenVZ checkpointing and live migrationLive migration in KVMVMware VMotionMicrosoft: Step by Step Guide for live migrationMicrosoft Whitepaper: Live Migration Overview & Architecture
Virtualization