Networking
This page provides an introduction to the common networking configurations used by libvirt based applications. This information applies to all hypervisors, whether Xen, KVM or another. For additional information consult the libvirt network architecture docs.
The two common setups are "virtual network" or "shared physical device". The former is identical across all distributions and available out-of-the-box. The latter needs distribution specific manual configuration.
Contents
NAT forwarding (aka "virtual networks")
Host configuration
Every standard libvirt installation provides NAT based connectivity to virtual machines out of the box. This is the so called 'default virtual network'. You can verify that it is available with
# virsh net-list --all Name State Autostart ----------------------------------------- default active yes
If it is missing, then the example XML config can be reloaded & activated
# virsh net-define /usr/share/libvirt/networks/default.xml Network default defined from /usr/share/libvirt/networks/default.xml # virsh net-autostart default Network default marked as autostarted # virsh net-start default Network default started
When the libvirt default network is running, you will see an isolated bridge device. This device explicitly does *NOT* have any physical interfaces added, since it uses NAT + forwarding to connect to outside world. Do not add interfaces
# brctl show bridge name bridge id STP enabled interfaces virbr0 8000.000000000000 yes
Libvirt will add iptables rules to allow traffic to/from guests attached to the virbr0 device in the INPUT, FORWARD, OUTPUT and POSTROUTING chains. It will also attempt to enable ip_forward. Some other applications may disable it, so the best option is to add the following to /etc/sysctl.conf
net.ipv4.ip_forward = 1
If you are already running dnsmasq on your machine, please see libvirtd and dnsmasq.
Guest configuration
Once the host configuration is complete, a guest can be connected to the virtual network based on its name. eg to connect a guest to the 'default' virtual network, the following XML would be used in the guest:
<interface type='network'>
<source network='default'/>
<mac address='00:16:3e:1a:b3:4a'/>
</interface>
NB, the mac address is optional and will be automatically generated if omitted.
Host configuration
The NAT based connectivity is useful for quick & easy deployments, or on machines with dynamic/sporadic networking connectivity. More advanced users will want to use full bridging, where the guest is connected directly to the LAN. The instructions for setting this up vary by distribution, and even by release.
Fedora/RHEL Bridging
This outlines how to setup briding using standard network initscripts
Disabling Xen's network scripts
If using Xen it is recommended to disable its network munging by editing /etc/xen/xend-config.sxp and changing the line
(network-script network-bridge)
To be
(network-script /bin/true)
Disabling NetworkManager
As of the time of writing (Fedora 12), NetworkManager still does not support bridging, so it is necessary to use "classic" network initscripts for the bridge, and to explicitly mark them as independent from NetworkManager (the "NM_CONTROLLED=no" lines in the scripts below).
If desired, you can also completely disable the NetworkManager:
# chkconfig NetworkManager off # chkconfig network on # service NetworkManager stop # service network start
Creating network initscripts
In the /etc/sysconfig/network-scripts directory it is neccessary to create 2 config files. The first (ifcfg-eth0) defines your physical network interface, and says that it will be part of a bridge:
# cat > ifcfg-eth0 <<EOF DEVICE=eth0 HWADDR=00:16:76:D6:C9:45 ONBOOT=yes BRIDGE=br0 NM_CONTROLLED=no EOF
Obviously change the HWADDR to match your actual NIC's address. You may also wish to configure the device's MTU here using e.g. MTU=9000.
The second config file (ifcfg-br0) defines the bridge device:
# cat > ifcfg-br0 <<EOF DEVICE=br0 TYPE=Bridge BOOTPROTO=dhcp ONBOOT=yes DELAY=0 NM_CONTROLLED=no EOF
WARNING: The line TYPE=Bridge is case-sensitive - it must have uppercase 'B' and lower case 'ridge'
After changing this restart networking (or simply reboot)
# service network restart
The final step is to disable netfilter on the bridge:
# cat >> /etc/sysctl.conf <<EOF net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-arptables = 0 EOF # sysctl -p /etc/sysctl.conf
It is recommended to do this for performance and security reasons. See Fedora bug #512206. Alternatively you can configure iptables to allow all traffic to be forwarded across the bridge:
# echo "-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT" > /etc/sysconfig/iptables-forward-bridged # lokkit --custom-rules=ipv4:filter:/etc/sysconfig/iptables-forward-bridged # service libvirtd reload
You should now have a "shared physical device", to which guests can be attached and have full LAN access
# brctl show bridge name bridge id STP enabled interfaces virbr0 8000.000000000000 yes br0 8000.000e0cb30550 yes eth0
Note how this bridge is completely independant of the virbr0. Do *NOT* attempt to attach a physical device to 'virbr0' - this is only for NAT connectivity
Debian/Ubuntu Bridging
This outlines how to setup bridging using standard network interface config files
Disabling NetworkManager
Stop network manager
sudo /etc/dbus-1/event.d/26NetworkManagerDispatcher stop sudo /etc/dbus-1/event.d/25NetworkManager stop
Create two files with only the word 'exit' in them. These files are:
/etc/default/NetworkManager /etc/default/NetworkManagerDispatcher
from https://help.ubuntu.com/community/NetworkManager#Disabling%20NetworkManager
Altering the interface config
First take down the interface you wish to bridge
ifdown eth0
Edit /etc/network/interfaces and find the config for the physical interface, which looks something like
allow-hotplug eth0
iface eth0 inet static
address 192.168.2.4
netmask 255.255.255.0
network 192.168.2.0
broadcast 192.168.2.255
gateway 192.168.2.2
Remove the 'allow-hotplug eth0' line, replacing it with 'auto br0', and change the next line with iface name to 'br0', so it now starts with
auto br0 iface br0 inet static
And then define the interface as being a bridge and specify its ports
bridge_ports eth0
bridge_stp on
bridge_maxwait 0
bridge_fd 0
The complete config should now look like
auto br0
iface br0 inet static
address 192.168.2.4
netmask 255.255.255.0
network 192.168.2.0
broadcast 192.168.2.255
gateway 192.168.2.2
bridge_ports eth0
bridge_stp on
bridge_maxwait 0
The interface can now be started with
ifup br0
Finally add the '/etc/sysctl.conf' settings
net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-arptables = 0
And then load the settings with
sysctl -p /etc/sysctl.conf
You should now have a "shared physical device", to which guests can be attached and have full LAN access
# brctl show bridge name bridge id STP enabled interfaces virbr0 8000.000000000000 yes br0 8000.000e0cb30550 yes eth0
Note how this bridge is completely independant of the virbr0. Do *NOT* attempt to attach a physical device to 'virbr0' - this is only for NAT connectivity
Guest configuration
In order to let your virtual machines use this bridge, their configuration file should include the interface definition as described in Bridge to LAN. In essence you are specifying the bridge name to connect to. Assuming a shared physical device where the bridge is called "br0", the following guest XML would be used:
<interface type='bridge'>
<source bridge='br0'/>
<mac address='00:16:3e:1a:b3:4a'/>
</interface>
NB, the mac address is optional and will be automatically generated if omitted.
The Guest XML file is located in /etc/libvirt/qemu
Other networking docs/links
- David Lutterkort's guide. NB the naming of devices 'peth0' (physical) and 'eth0' (bridge) does not work in Fedora 9 anymore. Following the 'eth0' (physical) and 'br0' (bridge) naming shown above instead
- Anthony Liguori's guide . Shows tips for 'shared physical devices' on Debian
- manual KVM networking - for people not using libvirt to launch guests
- Hard drive data recovery - for recovery of data.
- Ubuntu libvirt guide with a section on network bridge setup
