VM lifecycle

From Libvirt Wiki
Revision as of 10:41, 29 September 2011 by Akorn (talk | contribs) (Transient guest domains vs Persistent guest domains: ++grammar ++clarity)
Jump to: navigation, search

Virtual Machine Lifecycle

This page describes a basics of virtual machine lifecycle. It's aim is to have fundamental information about creating, running, stopping, migrating and deleting virtual machine in one page.


Since libvirt uses a slightly different terminology, it's important to know what is meant under new terms. Please refer to this page if you are not familiar with it.


Guest domains are described using XML configuration

In libvirt XML format is used to store configuration of everything, be it domain, network, storage, etc. It allows users to edit configuration in their favourite text editor. Devices in domain are represented by XML elements and device configuration is done by assigning attributes and/or child elements. For instance:

<domain type='qemu'>
      <disk type='file' device='disk'> ... </disk>
      <disk type='file' device='cdrom'> ... </disk>
      <input type='mouse' bus='ps2'/>

Libvirt then uses XPath technology to select nodes from XML document.

Transient guest domains vs Persistent guest domains

Libvirt distinguishes two different types of domains: transient and persistent. Transient domains are better suitable for single use, since they exist only until domain shutdown or host server restart. Persistent domains however last indefinitely.

Once a domain is created (no matter what type) its state can be saved into a file and then restored many times as long as the file exists. Thus even a transient domain can be restored over and over again.

Transient domains are created a bit differently than persistent ones. Persistent domains need to be defined first in order to be started up. Transient domains are created and started at once. This also means that different commands are used.

Please keep in mind that although transient domains are created and destroyed on-the-fly, all of their components (storage, networks, devices, ...) need to exist before.

This topic is strongly related to the next one.

States that a guest domain can be in

Domain can be in several states:

  1. Undefined - This is a baseline state. Libvirt does not know anything about domains in this state, because they haven't been defined or created in any way.
  2. Defined - The domain has been defined, but it's not running. That's why this state can be also called stopped. Only persistent domains can be in this state.
  3. Running - The domain has been created and started either as transient or persistent one. All domains in this state are executed on hypervisor.
  4. Paused - The domain execution on hypervisor has been suspended. It's state has been temporarily stored uppon resume. The domain does not have any knowledge wheter it was paused or not. If you are familiar with processes in operating systems, this is the same.
  5. Saved - Similar to previous state, but this time the domain state is stored to persistent storage. Again, the domain in this state can be restored and it does not notice anything.

The next picture shows how domain flows from one state to another:

Vm lifecycle graph.png

The states are described above, the edges represent commands used to change state of the domain.

From the picture one can see, it is possible with one command (shutdown) to either get to Defined or Undefined state. Transient domain would go to Undefined state and persistent one to Defined.


Snapshot is a view of a virtual machine's operating system and all its applications at a given point in time. Sometimes it is good to have restore points to which can one return. And this is the aim of snapshots. For example, user install and set his or hers domain and create a snapshot. From this time on, he or she can do anything with the domain, like chainging domain OS configuration, install or update new applications, etc. But anytime there is a possibility to unroll all changes made and return to previous state of domain.

At this point it is worth mentioning snapshots are static. So when you revert to previous state and make some changes (e.g. modify some files), they get lost unless you create a new snapshot.


A running virtual machine can be migrated to any host within its designated host. This is especially useful if the load on a particular host is too great, and is essential before bringing a server down for maintenance. Migration of virtual machines does not cause any service interruption.

Types of migration:

  • "standard" - Domain is suspended as long as all its resources are being transferred to destination host. Once done, it is again resumed on destination. The time spent in suspended state is directly proportional to domain memory size.
  • peer-to-peer - this type is used whenever source and destination hosts can communicate directly.
  • tunnelled - in this case a tunnel is created between source and destination hosts, e.g. ssh tunnel and all network communication is send through it.
  • live vs non-live - When migrating in live mode, domain is not paused and all services on are still running. Opposite is non-live migration, when a domain is being paused for the time necessary to transfer it's state through network. However, live migration is sensitive for heavy domain load. When live migrating a domain, it's allocated memory is send to destination host and being watched for changes at the same time. After this all changed memory areas are moved to destination host, and again since domain is still active, any other memory areas changed meanwhile must be recognized and tranferred. At the end we have the same memory images and domain can be transfered.
  • direct - libvirt just initiates the migration, but then whole process is under hypervisors control, since they talk to each other, e.g. Xen on source communicate directly to Xen on destination without any libvirt intervention.

Requirements for migration:

  • Shared storage accessible under same paths and locations,e.g. iSCSI, NFS, etc.
  • Exact same version of hypervisors on both hosts
  • Same network configuration
  • Same CPUs, or better said CPUs have to be from the same vendor and CPU flags on the destination host must be superset of CPU flags on source.

A checklist for successful migration is here.

Removing guest domains

When removing a guest domain one can also want to remove and wipe the storage it was using so there are no sensitive data left behind. This have to be done explicitly since in domain removing process libvirt does not release any resources domain was using, although they might be used only by this one domain.

Wiping the storage is critical, since simple file removal does not erase data from the hard drive, it only marks file as deleted and it's blocks as free space. The only option is to rewrite the file content with random data.


Creating a domain

In order to run a domain it is first needed to have one. So in the beginig is creation of a domain. This can be done several ways. The folowing page take you through the process in Virtual Machine Manager GUIT. The second way is using virt-install command line tool.

# virt-install \
             --connect qemu:///system \
             --virt-type kvm \
             --name MyNewVM \
             --ram 512 \
             --disk path=/var/lib/libvirt/images/MyNewVM.img,size=8 \
             --vnc \
             --cdrom /var/lib/libvirt/images/Fedora-14-x86_64-Live-KDE.iso \
             --network network=default,mac=52:54:00:9c:94:3b \
             --os-variant fedora14

This command creates new domain called 'MyNewVM', with 512 MB RAM, 8 GB disk using KVM. Please read manual page for any further information.

The last way is to create XML definition of domain and volume(s) and run virsh with appropiate commands: vol-create and define.

Volumes are joined in pool. By default, there exists one pool called "default". This is a directory-type pool, which means all volumes are stored as files in one directory. But please read this page if you are not completely aware of libvirt storage management. You may also find more suitable storage solution there.

Example of volume XML definition (new_volume.xml):

 <capacity unit="G">10</capacity>

This defines a new volume with capacity of 10 GB. To create volume in "default" pool:

# virsh vol-create default new_volume.xml

Example of domain XML definition (MyNewVM.xml):

<domain type='kvm'>
    <type arch='x86_64'>hvm</type>
    <boot dev='hd'/>
  <clock offset="utc"/>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/MyNewVM.img'/>
      <target dev='vda' bus='virtio'/>
    <disk type='block' device='cdrom'>
      <target dev='hdc' bus='ide'/>
    <interface type='network'>
      <source network='default'/>
      <mac address='52:54:00:9c:94:3b'/>
      <model type='virtio'/>
    <input type='tablet' bus='usb'/>
    <graphics type='vnc' port='-1'/>
    <console type='pty'/>
    <sound model='ac97'/>
      <model type='cirrus'/>

To define a new presistent domain:

# virsh define MyNewVM.xml

Domain XML format has many optional elements which you may find useful. Therefore read this page where is complete domain XML format reference including examples and most common scenarios.

Editing a domain

Any domain can be edited in user's favourite editor. What is needed is to set $VISUAL or $EDITOR environment variable and run:

# virsh edit <domain>

If any of these variables is not set, vi editor is used by default. After closing the editor libvirt will automaticly check for changes and apply them. However, it is also possible to edit domain in Virtual Machine Manager.

Starting a domain

Once a domain is created, one is able to run it. This is possible through Virtual Machine Manager or by running virsh start <domain> command. For example:

# virsh start MyNewVM

This command however performs either so called clean boot up or restores domain from previously saved state. See managedsave virsh command. It is important to notice, a domain can't be started if any of it's components is not up, e.g. network.

As mentioned above, transient domain can be run without previous definition:

# virsh create /path/to/MyNewVM.xml

Stopping or rebooting a domain

To stop running domain just run:

# virsh shutdown <domain>

To reboot a persistent domain:

# virsh reboot <domain>

Rebooting a transient domain is not possible, since right after shutdown are transient domains also undefined.

Not elegant shutdown also known as hard-stop:

# virsh destroy <domain>

This is equivalent to pluging the power-cable off.

Pausing a guest domain

Domain can be paused in virsh:

# virsh suspend <domain>

or in Virtual Machine Manager by clicking Pause button from main toolbar. When a guest is in a suspended state, it consumes system RAM but not processor resources. Disk and network I/O does not occur while the guest is suspended. This operation is immediate

Unpausing a guest domain

Any paused or suspended domain can be resumed by:

# virsh resume <domain>

or by unclicking appropiate Pause button in Virtual Manager.

Taking a Snapshot of a guest domain

Creating a snapshot is done by executing:

# virsh snapshot-create <domain>

Listing the snapshots of a guest domain

All snapshosts of a guest domain can be viewed in virsh:

# virsh snapshot-list <domain>

For instance, output might look like this:

 Name                 Creation Time             State
 1295973577           2011-01-25 17:39:37 +0100 running
 1295978837           2011-01-25 19:07:17 +0100 running

We can see one snapshot created at 17:39:17 local time, with name 1295973577 which is corresponding unix time in fact. The other was created at 19:07:17 with name 1295978837.

Restoring a guest domain from a snapshot

To restore a guest domain from a previous snapshot you can use:

# virsh snapshot-restore <domain> <snapshotname>

This restores a specified domain to a state represented by snapshotname. Please note, that any changes made will be destroyed!

Removing a snapshot from a guest domain

Any snaphsot of a given domain can be removed via:

# virsh snapshot-delete <domain> <snapshotname>


Libvirt provides migration support. It means you can migrate one domain from one host to another over network. Migration can operate in two main modes:

  • Plain migration: The source host VM opens a direct unencryptyed TCP connection to the destination host for sending the migration data. Unless a port is manually specified, libvirt will choose a migration port in the range 49152-49215, which will need to be open in the firewall on the remote host.
  • Tunneled migration: The source host libvirtd opens a direct connection to the destination host libvirtd for sending migration data. This allows the option of encrypting the data stream. This mode doesn't require any extra firewall configuration, but is only supported with qemu 0.12.0 or later, and libvirt 0.7.2.

For successful migration there are couple of things needed to be done. For instance, storage settings have to match. All volumes that migrated domain use have to be stored under the same paths. For full check list follow this page. We also recomend you to use secure migration.

Once pre-migration checks are done, you can migrate machine using virsh:

# virsh migrate <domain> <remote host URI> --migrateuri tcp://<remote host>:<port>

Deletion of a domain

One may delete an inactive domain in virsh:

# virsh undefine <domain>

As usual, there is also possibility to delete it in Virtual Machine Manager, covered in this page .

Wiping the storage used by a guest domain

A volume used by domain can contain confidential data, hence it is necessary to wipe it before removal. Libvirt offers a helping hand for such cases:

# virsh vol-wipe <volume>

which truncates and extends the volume to its original size. This in fact fills file with zeroes. This ensures that data previously stored on volume is not accessible to reads anymore. After this, you can remove volume :

# virsh vol-delete <volume>


These pages may also provide useful further information: