Google Summer of Code 2016/Asynchronous lifecycle events for storage objects

From Libvirt Wiki
Revision as of 18:04, 22 August 2016 by Gulic (talk | contribs)
(diff) ←Older revision | view current revision (diff) | Newer revision→ (diff)
Jump to: navigation, search

by Jovanka Gulicoska

During my internship I was working on implementing Asynchronous lifecycle events for storage objects. Lifecycle events allow applications to get notifications about the state of the object, as well as for object creation, deletion etc, without having to pull libvirt at regular intervals asking for the state of the VM. This implementation also includes adding support to virt-manager for storage events. Also, part of the project is extending event-test.py from libvirt-python to watch for storage events and print messages about them, as well as add asynchronous event support for nodedev objects. Extending the python API for the implementation in virt-manager and libvirt-python. After the implementation of the storage API, project includes implementation of asynchronous event support for nodedev objects, followed by extending the python API, implementation in virt-manager and in libvirt-python. When implementation of node device lifecycle events is done I've added support for udev drivers.

Storage pool lifecycle events

libvirt implementation

Storage pools can transition through several states during their lifecycle.

  1. Undefined: storage pool is not defined or created
  2. Defined: defined storage pool, but its not started
  3. Started: started storage pool
  4. Stopped: stopped storage pool
  5. Refreshed: refreshing storage pool/s

Storage pool lifecycle includes handling: Undefined, Defined, Started and Stopped and Refreshed is a new type of top level event.
I've implemented functions: virStoragePoolEventRegisterAny(), virStoragePoolEventDeregisterAny(), virStoragePoolLifeCycleEventNew(), introduced STARTED, STOPPED, DEFINE and UNDEFINE (virStoragePoolEventLifecycleType).
When defining a storage pool with storagePoolDefineXML() a DEFINED signal is emitted. STARTED signal is emitted when creating a storage pool with storagePoolCreateXML() or storagePoolCreate(). STOPPED signal is emitted when calling storagePoolDestroy() and storagePoolRefresh() and UNDEFINED is called on undefining a storage pool with storagePoolUndefine(). I've also implemented REFRESH signal which is emitted on storagePoolRefresh(), but afterwards we saw the need of implementing REFRESH as a top level event.
The series of patches also included unit tests for the implemented functions, as well as introducing virsh-pool command.
Implemented functions:
-virConnectStoragePoolEventLifecycleCallback
This callback is called when a pool lifecycle action is performed, like start or stop.
-virConnectStoragePoolEventRegisterAny
Adds a callback to receive notifications of arbitrary storage pool events occurring on a storage pool.
-virConnectStoragePoolEventDeregisterAny
Removes an event callback

Links to libvirt commits:
Introduce storage lifecycle event APIs
conf: add storage_event handling
test: implement storage lifecycle event APIs
remote: implement storage lifecycle event APIs
storage: implement storage lifecycle event APIs
event-test: support storage lifecycle event APIs
virsh: Introduce pool-event command

Link to patches on libvirt list: Introducing storage pool lifecycle events

libvirt-python implementation

After introducing the storage pool lifecycle events I've added support for them to libvirt-python library. This included extending event-test.py for printing messages about storage pool lifecycle events.

Links to libvirt-python commits:
Python binding for storage pool lifecycle events API
event-test: Add storage pool lifecycle event tests

Link to patches on libvirt list: Bindings for storage pool lifecycle events

virt-manager implementation

After implementing changes to libvirt-python I've added support for storage pool lifecycle events to virt-manager.

Links to virt-manager commits:
Use storage pool lifecycle events

Link to patches on virt-manager list: Use storage pool lifecycle events

Node device lifecycle events

The nodedev object API are for listing physical host hardware. It is used to get information about host PCI devices, USB devices, scsi devices and others. Virt-manager uses it for listing of hardware for attaching to a VM.

libvirt implementation

As mentioned above nodedev object API is used for listing physical resources on the host, there's not really much of a lifecycle notion for these devices. They don't have an on/off state like a VM/pool/network does, they are either listed/present, or not listed. This implementation only included lifecycle events for creating and deleting node devices. I've implemented virNodeDeviceEventRegisterAny(), virNodeDeviceEventDeregisterAny(), virNodeDeviceLifeCycleEventNew(), and introduced CREATED and DELETED(virNodeDeviceEventLifecycleType).
Code changes include unit tests and introducing virsh nodedev-evnet command.
Implemented functions:
-virConnectNodeDeviceEventLifecycleCallback
This callback is called when a node device lifecycle action is performed, like added or removed.
-virConnectNodeDeviceEventRegisterAny
Adds a callback to receive notifications of arbitrary node device events occurring on a node device.
-virConnectNodeDeviceEventDeregisterAny
Removes an event callback

Links to libvirt commits:
Introduce node device lifecycle event APIs
conf: add node_device_event handling
test: implement node device lifecycle event APIs
remote: implement node device lifecycle event APIs
node_device: implement node device lifecycle event APIs
node_device: Implement event queue in udev
event-test: support node device lifecycle event APIs
virsh: Introduce nodedev-event command

Link to patches on libvirt: Introducing node device lifecycle events

libvirt-python implementation

In libvirt-python I've added support for node device lifecycle events and extended event-python for printing messages for events for node devices.

Links to commits:
Python binding for node poll lifecycle events API
event-test: Add node device lifecycle event tests

Link to patches on libvirt Bindings for node device lifecycle events

virt-manager implementation

After adding support for libvirt-python, I've made changes to virt-manager and added support for using node device lifecycle events.

Links to commits:
Use node device lifecycle events

Link to patches: Use node device lifecycle events

Node device UPDATED as top level event

After the implementation of node devices CREATED and DELETED lifecycle events, there was a need of implementing UPDATED as a top level event.

libvirt implementation

Series include API implementation, udev driver implementation, event-test implementation and virsh nodedev-event changes. Added UPDATE to virNodeDeviceEventID
Implemented functions:
-virConnectNodeDeviceEventGenericCallback

Links to libvirt commits:
Introduce node device update event as top level event

Link to patches on libvirt list: Node device update lifecycle event as top level event

libvirt-python implementation

Added python bindings for node device update event, and extended event-test for printing messages for update events of node devices.

Links to commits:
Add support for node device update callback
event-test: support node device update callback

Link to patches on libvirt-list: Python binding for node device update event API

virt-manager implementation

Added support for node device update events to virt-managet.

Links to commits:
Use node device update event support

Link to patches on list: https://www.redhat.com/archives/virt-tools-list/2016-August/msg00056.html

Other work

More usage of virGetLastErrorMessage()

Relace virGetLastError() with virGetLastErrorMessage() in drivers and unit tests.

Link to libvirt commits:
More usage of virGetLastErrorMessage
tests: More usage of virGetLastErrorMessage()

Link to patches on list: https://www.redhat.com/archives/libvir-list/2016-May/msg01523.html

Replace VIR_ERROR with standard vir*Error in state driver init

Replace VIR_ERROR logging macros for error reporting in driver startup routines with vir*Error function.

Links to libvirt code:
bhyve: Replace VIR_ERROR with standard vir*Error in state driver init
libxl: Replace VIR_ERROR with standard vir*Error in state driver init
nwfilter: Replace VIR_ERROR with standard vir*Error in state driver init
storage: Replace VIR_ERROR with standard vir*Error in state driver init
uml: Replace VIR_ERROR with standard vir*Error in state driver init
xen: Replace VIR_ERROR with standard vir*Error in state driver init
qemu: Replace VIR_ERROR with standard vir*Error in state driver init
node_device: Replace VIR_ERROR with standard vir*Error in state driver init

Link to patches on list: https://www.redhat.com/archives/libvir-list/2016-June/msg00353.html

conf: events: handle NULL uuid

Not all objects have a uuid, such as nodedevs. When we add events support for them, NULL will be passed here, so handle it.

Link to libvirt commits:
conf: events: handle NULL uuid

Fix storage pool refresh event signal

When creating a storage volume, it wouldn't show up in the list since the refresh signaling was busted. Code was using VIR_STORAGE_POOL_EVENT_REFRESHED which was dropped from lifecycle event type, refresh is introduced as top level event.

Links to commits:
Fix storage pool refresh event signal

Link to patches: https://www.redhat.com/archives/virt-tools-list/2016-August/msg00068.html

Pending work

There is work that is done and not sent to the list. If we not achieve to push it till August 23rd, it will be pushed afterwards.

Interface lifecycle event

I've started implementation of interface lifecycle events API. Code needs to be checked and sent to list. Implementation can be found here.

Call refreshPool on volume creation/deletion

Implementation can be found here.

Summary of commits:

  1. libvirt github repository
  2. libvirt-python
  3. virt-manager