OpenStack: Perform Consistent Snapshots With Qemu Guest Agent

重温战场

　　http://www.sebastien-han.fr/blog/2015/02/09/openstack-perform-consistent-snapshots-with-qemu-guest-agent/
　　

A while back, I wrote
an article about taking consistent snapshots of your virtual machines in your OpenStack environment. However this method was really intrusive since
it required to be inside the virtual machine and to manually summon a filesystem freeze. In this article, I will use a different approach to achieve the same goal without the need to be inside
the virtual machine. The only requirement is to have a virtual machine running the qemu-guest-agent.






OpenStack Nova and QEMU guest agent


The QEMU guest support landed in Nova during the
Havana cycle, so basically we are two release ahead. This functionality is based on Glance properties.

But how does that work?

The principle is quite easy. A virtual machine boots with a new virtio device attached pointing to unix socket on the hypervisor. Inside the virtual machine this socket will appear as a new character device, commonly under /dev/virtio-ports/.

A picture is always better:








Configure the QEMU agent


For Ubuntu, you need to apply this fix since AppArmor will not allow the creation of the socket:



[color=#586E75!important]1
[color=#586E75!important]2
[color=#586E75!important]3
[color=#586E75!important]4


[color=#268BD2!important]$ sudo [color=#859900!important]echo [color=#2AA198!important]"/var/lib/libvirt/qemu/*.sock rw," | sudo tee -a /etc/apparmor.d/abstractions/libvirt-qemu
[color=#268BD2!important]$ sudo service libvirt-bin restart
[color=#268BD2!important]$ sudo service nova-compute restart
[color=#268BD2!important]$ sudo service apparmor reload



Configure your Glance image:



[color=#586E75!important]1
[color=#586E75!important]2
[color=#586E75!important]3
[color=#586E75!important]4
[color=#586E75!important]5
[color=#586E75!important]6
[color=#586E75!important]7


[color=#268BD2!important]$ glance image-create --name cirros [color=#DC322F!important]\
--disk-format raw [color=#DC322F!important]\
--container-format bare [color=#DC322F!important]\
--file cirros-0.3.3-x86_64-disk.raw [color=#DC322F!important]\
--is-public True [color=#DC322F!important]\
--property [color=#268BD2!important]hw_qemu_guest_agent=yes [color=#DC322F!important]\
--progress



Boot your virtual machine:



[color=#586E75!important]1


[color=#268BD2!important]$ nova boot ...



Verify that the agent is in the virtual machine:



[color=#586E75!important]1
[color=#586E75!important]2


ubuntu@agent:~[color=#268BD2!important]$ file /dev/virtio-ports/org.qemu.guest_agent.0
/dev/virtio-ports/org.qemu.guest_agent.0: symbolic link to `../vport2p1[color=#DC322F!important]'



Install the QEMU agent inside your VM:



[color=#586E75!important]1
[color=#586E75!important]2
[color=#586E75!important]3
[color=#586E75!important]4
[color=#586E75!important]5
[color=#586E75!important]6
[color=#586E75!important]7
[color=#586E75!important]8
[color=#586E75!important]9
[color=#586E75!important]10


ubuntu@agent:~[color=#268BD2!important]$ sudo apt-get install -y qemu-guest-agent
ubuntu@agent:~[color=#268BD2!important]$ sudo mkdir /var/log/qemu-agent
ubuntu@agent:~[color=#268BD2!important]$ sudo tee /etc/default/qemu-guest-agent > /dev/null [color=#2AA198!important]<<EOF
[color=#2AA198!important]DAEMON_ARGS=&quot;--logfile /var/log/qemu-agent/org.qemu.guest_agent.0.log --fsfreeze-hook --verbose&quot;
[color=#2AA198!important]EOF
ubuntu@agent:~[color=#268BD2!important]$ sudo service qemu-guest-agent restart
* Restarting QEMU Guest Agent qemu-qa
   ...done
ubuntu@agent:~[color=#268BD2!important]$ sudo ls /var/log/qemu-agent/
org.qemu.guest_agent.0.log



Now go back on the hypervisor and check that the socket file is present (it must here since we have the character device inside the virtual machine):



[color=#586E75!important]1
[color=#586E75!important]2
[color=#586E75!important]3
[color=#586E75!important]4
[color=#586E75!important]5


[color=#268BD2!important]$ sudo bash -c  [color=#2AA198!important]&quot;ls /var/lib/libvirt/qemu/*.sock&quot;
/var/lib/libvirt/qemu/capabilities.monitor.sock  /var/lib/libvirt/qemu/org.qemu.guest_agent.0.instance-00000007.sock

[color=#268BD2!important]$ sudo file /var/lib/libvirt/qemu/org.qemu.guest_agent.0.instance-00000007.sock
/var/lib/libvirt/qemu/org.qemu.guest_agent.0.instance-00000007.sock: socket



Test if the QEMU agent responds:



[color=#586E75!important]1
[color=#586E75!important]2


[color=#268BD2!important]$ sudo virsh qemu-agent-command instance-00000007 [color=#2AA198!important]'{&quot;execute&quot;:&quot;guest-ping&quot;}'
{[color=#2AA198!important]&quot;return&quot;:{}}








Iced that guy!




Full reference of the image available here :D.





Setup the fsfreeze hook mechanism, on Red Hat systems the file already exists:



[color=#586E75!important]1
[color=#586E75!important]2


ubuntu@agent:~[color=#268BD2!important]$ sudo wget -O /etc/qemu/fsfreeze-hook https://raw.githubusercontent.com/qemu/qemu/master/scripts/qemu-guest-agent/fsfreeze-hook
ubuntu@agent:~[color=#268BD2!important]$ sudo chmod &#43;x /etc/qemu/fsfreeze-hook



Configure a basic hook, it will be executed either during the freeze or thaw operation:



[color=#586E75!important]1
[color=#586E75!important]2
[color=#586E75!important]3
[color=#586E75!important]4
[color=#586E75!important]5
[color=#586E75!important]6
[color=#586E75!important]7
[color=#586E75!important]8
[color=#586E75!important]9
[color=#586E75!important]10
[color=#586E75!important]11
[color=#586E75!important]12
[color=#586E75!important]13
[color=#586E75!important]14
[color=#586E75!important]15
[color=#586E75!important]16
[color=#586E75!important]17


ubuntu@agent:~[color=#268BD2!important]$ sudo mkdir /etc/qemu/fsfreeze-hook.d
ubuntu@agent:~[color=#268BD2!important]$ sudo tee > /etc/qemu/fsfreeze-hook.d/foo.sh > /dev/null [color=#2AA198!important]<<EOF
[color=#2AA198!important]#!/bin/bash

[color=#2AA198!important]case &quot;$1&quot; in
[color=#2AA198!important]    freeze)
[color=#2AA198!important]        echo &quot;I'm frozen&quot; > /tmp/freeze
[color=#2AA198!important]        ;;
[color=#2AA198!important]    thaw)
[color=#2AA198!important]        echo &quot;I'm thawed&quot; >> /tmp/freeze
[color=#2AA198!important]        ;;
[color=#2AA198!important]    *)
[color=#2AA198!important]        exit 1
[color=#2AA198!important]        ;;
[color=#2AA198!important]esac
[color=#2AA198!important]EOF
ubuntu@agent:~[color=#268BD2!important]$ sudo chmod &#43;x /etc/qemu/fsfreeze-hook.d/foo.sh



Now let’s freeze ad thaw the filesystem:



[color=#586E75!important]1
[color=#586E75!important]2


[color=#268BD2!important]$ sudo virsh qemu-agent-command instance-00000008 [color=#2AA198!important]'{&quot;execute&quot;:&quot;guest-fsfreeze-freeze&quot;}'
[color=#268BD2!important]$ sudo virsh qemu-agent-command instance-00000008 [color=#2AA198!important]'{&quot;execute&quot;:&quot;guest-fsfreeze-thaw&quot;}'



Did the hook work as expected? Yes!



[color=#586E75!important]1
[color=#586E75!important]2
[color=#586E75!important]3


ubuntu@agent:~[color=#268BD2!important]$ sudo cat /tmp/freeze
I[color=#2AA198!important]'m frozen
[color=#2AA198!important]I'm thawed



For a more concrete example, have a look
at this work for a database from Chromium.






OpenStack Nova and Snapshots


Several patches have been submitted to quiesce the filesystem prior to run the snapshot. The initial work to support fs-freeze while performing a snapshot of an instance was introduced in Juno with a spec.
However the commit only got merged in
December… This will be available in Kilo. Another effort to support this feature while booting from a volume is currently under review.

The original blueprint can be found here.
Ultimately this option will be available via a Glance property:



[color=#586E75!important]1


[color=#268BD2!important]$ glance image-update 53bd9dbe-23db-412b-81d5-9743aabdfeb5 --property [color=#268BD2!important]os_require_quiesce=yes



When this option will be set and the virtual machine running the QEMU guest agent, when a user will snapshot an instance, the filesystem will get frozen and thawed after the operation.