- 1 Libvirt FAQ
- 1.1 General
- 1.1.1 What is libvirt?
- 1.1.2 I heard someone say they 'use libvirt'. What do they mean?
- 1.1.3 How do I know if I am using libvirt?
- 1.1.4 What is some of the major functionality provided by libvirt?
- 1.1.5 What hypervisors does libvirt support?
- 1.1.6 How can I check my libvirt version?
- 1.1.7 What are the libvirt mailing lists/IRC?
- 1.1.8 Where should I report libvirt bugs?
- 1.1.9 Is libvirt the 'lowest common denominator' of hypervisor features?
- 1.2 Common VM Configuration
- 1.2.1 What is the 'virsh edit' command and how do I use it?
- 1.2.2 Where are VM config files stored? How do I edit a VM's XML config?
- 1.2.3 If I change the XML of running machine, do the changes take immediate effect?
- 1.2.4 How do I shutdown my VMs?
- 1.2.5 Will restarting the libvirt daemon stop my virtual machines?
- 1.3 Common Errors
- 1.3.1 My VM doesn't show up with 'virsh list'
- 1.3.2 Error: Failed to add tap interface 'vnet%d' to bridge 'virbr0' No such file or directory
- 1.3.3 Error: unable to connect to '/var/run/libvirt/libvirt-sock'
- 1.3.4 Why doesn't 'shutdown' doesn't seem to work?
- 1.3.5 Error: domain did not show up
- 1.3.6 My VM fails to start. What should I do?
- 1.4 QEMU/KVM
- 1.5 Networking
- 1.6 Developing with libvirt
- 1.1 General
What is libvirt?
Libvirt is collection of software that provides a convenient way to manage virtual machines and other virtualization functionality, such as storage and network interface management. These software pieces include an API library, a daemon (libvirtd), and a command line utility (virsh).
An primary goal of libvirt is to provide a single way to manage multiple different virtualization providers/hypervisors. For example, the command 'virsh list --all' can be used to list the existing virtual machines for any supported hypervisor (KVM, Xen, VMWare ESX, etc.) No need to learn the hypervisor specific tools!
I heard someone say they 'use libvirt'. What do they mean?
When people say they 'use libvirt', this usually means that they manage virtual machines using tools such as 'virsh', 'virt-manager', or 'virt-install', which are all built around libvirt functionality. They likely DO NOT directly use tools like 'xm' for Xen, or the qemu/qemu-kvm binary.
How do I know if I am using libvirt?
You are using libvirt if you manage virtual machines using virsh, virt-manager, or virt-install (pretty much any virtualization tool that starts with virt-*).
If you are using hypervisor specific tools like 'xm', 'qemu-kvm', etc. directly, you probably are not using libvirt.
If you have virtual machines on your existing machine and you are using libvirt, 'virsh list --all' (usually run as root) should show something.
What is some of the major functionality provided by libvirt?
Some of the major libvirt features are:
- VM management: Various domain lifecycle operations such as start, stop, pause, save, restore, and migrate. Hotplug operations for many device types including disk and network interfaces, memory, and cpus.
- Remote machine support: All libvirt functionality is accessible on any machine running the libvirt daemon, including remote machines. A variety of network transports are supported for connecting remotely, with the simplest being SSH, which requires no extra explicit configuration. If example.com is running libvirtd and SSH access is allowed, the following command will provide access to all virsh commands on the remote host for qemu/kvm:
virsh --connect qemu+ssh://firstname.lastname@example.org/system
For more info, see: http://libvirt.org/remote.html
- Storage management: Any host running the libvirt daemon can be used to manage various types of storage: create file images of various formats (qcow2, vmdk, raw, ...), mount NFS shares, enumerate existing LVM volume groups, create new LVM volume groups and logical volumes, partition raw disk devices, mount iSCSI shares, and much more. Since libvirt works remotely as well, all these options are available for remote hosts as well.
For more info, see: http://libvirt.org/storage.html
- Network interface management: Any host running the libvirt daemon can be used to manage physical and logical network interfaces. Enumerate existing interfaces, as well as configure (and create) interfaces, bridges, vlans, and bond devices. This is with the help of netcf.
For more info, see: https://fedorahosted.org/netcf/
- Virtual NAT and Route based networking: Any host running the libvirt daemon can manage and create virtual networks. Libvirt virtual networks use firewall rules to act as a router, providing VMs transparent access to the host machines network.
For more info, see: http://libvirt.org/archnetwork.html
What hypervisors does libvirt support?
A complete list can be found here: http://libvirt.org/drivers.html
How can I check my libvirt version?
For the installed virsh version:
For the libvirt daemon version:
For the library and default hypervisor version:
What are the libvirt mailing lists/IRC?
Where should I report libvirt bugs?
For bug reporting, see: http://libvirt.org/bugs.html
Is libvirt the 'lowest common denominator' of hypervisor features?
Short answer: ABSOLUTELY NOT.
Libvirt's goal is to expose all useful hypervisor features. Period.
The only caveat is that this feature needs to be exposed in a general way that is compatible with the libvirt architecture. Even if only a single supported hypervisor implements feature FOO, the API and XML changes need to be made sufficiently general in case any other hypervisor eventually supports FOO.
Common VM Configuration
What is the 'virsh edit' command and how do I use it?
'virsh edit' is the recommended way to make changes to an existing VM configuration. The command looks like:
virsh edit $your-vm-name
This command will open a text editor containing the existing VM XML: any changes that are made and saved will be checked for errors when the editor exits. If no errors are found, the changes are made permanent.
The text editor used is whatever is specified by the EDITOR environment variable. By default, this is usually 'vi'. You can override this with:
EDITOR=$your-favorite-editor virsh edit $your-vm-name'.
For example, on a gnome system 'EDITOR=gedit virsh edit myvm' will edit myvm's XML in a graphical text editor.
Where are VM config files stored? How do I edit a VM's XML config?
Let me start by saying you SHOULD NOT CARE WHERE THE XML IS STORED (most likely). If you want to edit a VMs XML, use 'virsh edit $vmname'. If you want to backup the XML, use 'virsh dumpxml $vmname'. Editing the XML directly on disk where libvirt stores it can cause SERIOUS COMPLICATIONS.
Where VM confuration is stored depends on the hypervisor. For example, Xen and VMWare store their own configs, and libvirt just translates this to XML when 'virsh dumpxml' is called.. For qemu and lxc, libvirt stores the XML on disk and in memory.
If I change the XML of running machine, do the changes take immediate effect?
NO. Redefining the XML of a running machine will not change anything, the changes will take effect after the next VM start up. Libvirt has a set of commands for making live changes to running guests, which have varying support depending on the hypervisor, ex virsh attach-*. virsh detach-*, virsh setmem, virsh setvcpus
How do I shutdown my VMs?
There are two shutdown operations via virsh:
- virsh shutdown $vm-name : Request a soft shutdown, akin to pressing the power button on a physical machine.
- virsh destroy $vm-name : Hard poweroff a physical machine. Akin to ripping the power cord from a running machine.
The other option it to shut down your VM normally from inside the guest operating system, like you would for a physical machine.
Note: At least for the qemu/kvm guests, shutdown requires ACPI enabled in the guest. See the QEMU/KVM shutdown FAQ entry.
Will restarting the libvirt daemon stop my virtual machines?
NO, as of version 0.6.0 (Jan 2009). Versions older then this will kill VMs if the libvirtd daemon is stopped, so beware.
My VM doesn't show up with 'virsh list'
First, make sure you are passing the --all flag:
virsh list --all
'virsh list' only shows running VMs: --all is required to see every VM.
If your VM is still not listed, determine which URI virsh is defaulting to with:
If the default URI is not as expected, you can manually specify a URI with:
virsh --connect URI
If you are using QEMU/KVM and you created your VM with virt-manager, the URI you probably want is qemu:///system. If that doesn't work, read the FAQ entry for qemu:///system VS. qemu:///session
Error: Failed to add tap interface 'vnet%d' to bridge 'virbr0' No such file or directory
This is commonly caused by 2 things
- The 'tun' module is not loaded. Try running:
- The virtual network used by your VM is not running. This is usually the 'default' network, which can be started with
virsh net-start default
Error: unable to connect to '/var/run/libvirt/libvirt-sock'
This is usually caused by one of the following:
- libvirtd is not running. On fedora systems, this can be fixed with:
service libvirtd restart
- You manually installed from source, and something is screwy. virsh may be looking for the socket in '/usr/local/var', but your installed libvirtd isn't creating it correctly.
It's recommended that you configure a manual libvirt install with --prefix=/usr to correct these issues.
Why doesn't 'shutdown' doesn't seem to work?
If you are using Xen HVM or QEMU/KVM, ACPI must be enabled in the guest for a graceful shutdown to work. To check if ACPI is enabled, run:
virsh dumpxml $your-vm-name | grep acpi
If nothing is printed, ACPI is not enabled for your machine. Use 'virsh edit' to add the following XML under <domain>:
HOWEVER, if your VM is running Windows, this won't be enough. If windows does not detect ACPI at install time, it disables the necessary support. The recommended way to remedy this seems to be a 'repair install' using windows install media. More info can be found here:
Error: domain did not show up
For a while, this was a kind of catch all error for qemu/kvm guest start up failures. See 'My VM fails to start'
My VM fails to start. What should I do?
If your VM is failing to start, and libvirt isn't returning a helpful error message, you can find more info in the log files. For qemu/kvm, this is /var/log/libvirt/qemu/$your-vm-name.log. This will show generated qemu command line, and any error output qemu throws. Certain versions of libvirt weren't good at returning this info to the user, so there may be a fixable error here, like a missing storage file.
If you can't determine anything to fix, please report a bug (see the bug reporting FAQ).
Can I connect to the QEMU monitor with libvirt?
No. Libvirt deliberately does not enable user access to the QEMU monitor. Interacting with the monitor behind libvirt's back can cause reported virtual machine state to be out of sync, which will likely end with errors.
The only way to interact with the monitor is through libvirt APIs (see the following question for a complete list of support)
What monitor commands does libvirt support? What QEMU/KVM command line flags does libvirt support?
For a complete list, please see: http://wiki.libvirt.org/page/QEMUSwitchToLibvirt
What is the difference between qemu:///system and qemu:///session? Which one should I use?
All 'system' URIs (be it qemu, lxc, uml, ...) connect to the libvirtd daemon running as root which is launched at system startup. Virtual machines created and run using 'system' are usually launched as root, unless configured otherwise (for example in /etc/libvirt/qemu.conf).
All 'session' URIs launch a libvirtd instance as your local user, and all VMs are run with local user permissions.
You will definitely want to use qemu:///system if your VMs are acting as servers. VM autostart on host boot only works for 'system', and the root libvirtd instance has necessary permissions to use proper networkings via bridges or virtual networks. qemu:///system is generally what tools like virt-manager default to.
qemu:///session has a serious drawback: since the libvirtd instance does not have sufficient privileges, the only out of the box network option is qemu's usermode networking, which has nonobvious limitations, so its usage is discouraged. More info on qemu networking options: http://people.gnome.org/~markmc/qemu-networking.html
The benefit of qemu:///session is that permission issues vanish: disk images can easily be stored in $HOME, serial PTYs are owned by the user, etc.
Why is libvirt doing with iptables?
By default, libvirt provides a virtual network named 'default' which acts as a NAT router for virtual machines, routing traffic to the network connected to your host machine. This functionality uses iptables.
How can I make libvirt stop using iptables?
WARNING: Any VMs already configured to use these virtual networks will need to be edited: simply removing the <interface> devices via 'virsh edit' should be sufficient. Not doing this step will cause starting to fail, and even after the editing step the VMs will not have network access.
You can remove all libvirt virtual networks on the machine:
- Use 'virsh net-list --all' to see a list of all virtual networks
- Use 'virsh net-destroy $net-name' to shutdown each running network
- Use 'virsh net-undefine $net-name' to permanently remove the network
Why doesn't libvirt just auto configure a regular network bridge?
While this would be nice, it is difficult/impossible to do in a safe way that won't hit a lot of trouble with non trivial networking configurations. A static bridge is also not compatible with a laptop mode of networking, switching between wireless and wired. Static bridges also do not play well with NetworkManager as of this writing (Feb 2010).
You can find more info about the motivation virtual networks here: http://www.gnome.org/~markmc/virtual-networking.html
How do I manually configure a network bridge?
How do I get my VM to use an existing network bridge?
Developing with libvirt
Is libvirt thread safe?
Yes, libvirt is thread safe as of version 0.6.0. This means that multiple threads can act on a single virConnect instance without issue.
Previous libvirt versions required opening a separate connection for each thread: this method has several major drawbacks and is not recommended.