cinder - 备份glusterfs卷到nfs server
cinder backup就是针对cinder volume的备份，后端可以有不同的bakcup driver，如nfs、ceph、swift、glusterfs等。这里演示volume backend为glusterfs，backup backend为nfs。（centos7 环境）
一、如何安装配置glusterfs
二、如何安装配置nfs server
1
2
3
4
5
6
7
8
9
10
11
12
# nfs server端配置
yum install nfs-utils# 安装nfs包
mkdir -p /backup # 创建nfs共享目录
vim /etc/exports # 编译exports，配置一些访问权限
/backup 172.16.40.0/24(rw,sync,no_root_squash,no_all_squash)
# 启动服务
systemctl enable rpcbind
systemctl enable nfs-server
systemctl restart rpcbind
systemctl restart nfs-server
1
2
3
4
# nfs client端配置
yum install nfs-utils
showmount -e # 验证/backup共享目录是否被发现
三、下面就是OpenStack Cinder配置
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
stack@yxb-devstack:~$ cat /etc/cinder/cinder.conf
enabled_backends = glusterfs
os_region_name = RegionOne
backup_driver = cinder.backup.drivers.nfs
backup_share=172.16.40.189:/backup
#backup_mount_options = "vers=3" #
backup_mount_point_base = $state_path/backup_mount# 备份目录挂载点
volume_driver=cinder.volume.drivers.glusterfs.GlusterfsDriver
glusterfs_shares_config = /etc/cinder/glusterfs_shares
volume_backend_name = glusterfs
# 重启cinder-volume、cinder-backup服务
systemctl restart openstack-cinder-volumeopenstack-cinder-backup
# horizon启动cinder backup功能
vim /etc/openstack-dashboard/local_settings
OPENSTACK_CINDER_FEATURES = {
'enable_backup': True,
}
参考链接：
http://docs.openstack.org/admin-guide-cloud/blockstorage_volume_backups.html
http://docs.openstack.org/liberty/config-reference/content/nfs-backup-driver.html
https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3/html/Administration_Guide/sect-NFS.html#Manually_Mounting_Volumes_Using_NFS (glusterfs-nfs)
http://docs.openstack.org/liberty/config-reference/content/GlusterFS-driver.html# 也支持glusterfs backup
四、cinder backup cli使用
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
stack@yxb-devstack:~$ cinder help backup-create# help下就懂怎么用了
usage: cinder backup-create [--container <container>] [--name <name>]
[--description <description>] [--incremental]
[--force]
<volume>
Creates a volume backup.
Positional arguments:
<volume> Name or ID of volume to backup.
Optional arguments:
--container <container>
Backup container name. Default=None.
--name <name> Backup name. Default=None.
--description <description>
Backup description. Default=None.
--incremental Incremental backup. Default=False.
--force Allows or disallows backup of a volume when the volume
is attached to an instance. If set to True, backs up
the volume whether its status is "available" or "in-
use". The backup of an "in-use" volume means your data
is crash consistent. Default=False.
小结：
1、cinder backup目前是支持全量、增量、在线、离线备份；
2、增量备份必须建立在全量备份的基础上；
3、删除备份，如果有基于全量备份的增量备份，应先删除增量备份；删除增量备份也有顺序之分，后面做的增量备份先删除
4、目前不支持在线备份还原（L版和最新master都是）；
5、第一次从全量备份还原的时候目前不能指定卷名字（master分支已有patch修复 https://review.openstack.org/#/c/275910/）
6、备份glusterfs卷有限制，该卷是raw格式、不能有snapshot，
备份的volume必须是raw格式,相关配置项nas_volume_prov_type = thick(thin|thick)； （L版存在这种问题，最新master已不存在）
五、cinder backup代码调用（最新master分支）
cinderclient/v2/volume_backups.py # cinder client端发起请求到api端
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
def create(self, volume_id, container=None,
name=None, description=None,
incremental=False, force=False,
snapshot_id=None):
"""Creates a volume backup.
:param volume_id: The ID of the volume to backup.
:param container: The name of the backup service container.
:param name: The name of the backup.
:param description: The description of the backup.
:param incremental: Incremental backup.
:param force: If True, allows an in-use volume to be backed up.
:rtype: :class:`VolumeBackup`
"""
body = {'backup': {'volume_id': volume_id,
'container': container,
'name': name,
'description': description,
'incremental': incremental,
'force': force,
'snapshot_id': snapshot_id, }}
return self._create('/backups', body, 'backup')
cinder/api/contrib/backup.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
@wsgi.response(202)
@wsgi.serializers(xml=BackupTemplate)
@wsgi.deserializers(xml=CreateDeserializer)
def create(self, req, body):
"""Create a new backup."""
LOG.debug('Creating new backup %s', body)
self.assert_valid_body(body, 'backup')
context = req.environ['cinder.context']
backup = body['backup']
try:
volume_id = backup['volume_id']
except KeyError:
msg = _("Incorrect request body format")
raise exc.HTTPBadRequest(explanation=msg)
container = backup.get('container', None)
self.validate_name_and_description(backup)
name = backup.get('name', None)
description = backup.get('description', None)
incremental = backup.get('incremental', False)
force = backup.get('force', False)
snapshot_id = backup.get('snapshot_id', None)
LOG.info(_LI("Creating backup of volume %(volume_id)s in container"
" %(container)s"),
{'volume_id': volume_id, 'container': container},
context=context)
try:
new_backup = self.backup_api.create(context, name, description, # 转发请求到backup api
volume_id, container,
incremental, None, force,
snapshot_id)
except (exception.InvalidVolume,
exception.InvalidSnapshot) as error:
raise exc.HTTPBadRequest(explanation=error.msg)
except (exception.VolumeNotFound,
exception.SnapshotNotFound) as error:
raise exc.HTTPNotFound(explanation=error.msg)
except exception.ServiceNotFound as error:
raise exc.HTTPInternalServerError(explanation=error.msg)
retval = self._view_builder.summary(req, dict(new_backup))
return retval
cinder/backup/api.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
def create(self, context, name, description, volume_id,
container, incremental=False, availability_zone=None,
force=False, snapshot_id=None):
"""Make the RPC call to create a volume backup."""
check_policy(context, 'create') # 校验policy权限
volume = self.volume_api.get(context, volume_id)
snapshot = None
if snapshot_id:
snapshot = self.volume_api.get_snapshot(context, snapshot_id)
if volume['status'] not in ["available", "in-use"]:
msg = (_('Volume to be backed up must be available '
'or in-use, but the current status is "%s".')
% volume['status'])
raise exception.InvalidVolume(reason=msg)
elif volume['status'] in ["in-use"] and not snapshot_id and not force:
msg = _('Backing up an in-use volume must use '
'the force flag.')
raise exception.InvalidVolume(reason=msg)
elif snapshot_id and snapshot['status'] not in ["available"]:
msg = (_('Snapshot to be backed up must be available, '
'but the current status is "%s".')
% snapshot['status'])
raise exception.InvalidSnapshot(reason=msg)
previous_status = volume['status']
host = self._get_available_backup_service_host(
None, volume.availability_zone,
volume_utils.extract_host(volume.host, 'host'))
# Reserve a quota before setting volume status and backup status
try:
reserve_opts = {'backups': 1,
'backup_gigabytes': volume['size']}
reservations = QUOTAS.reserve(context, **reserve_opts)
except exception.OverQuota as e:
overs = e.kwargs['overs']
usages = e.kwargs['usages']
quotas = e.kwargs['quotas']
def _consumed(resource_name):
return (usages['reserved'] +
usages['in_use'])
for over in overs:
if 'gigabytes' in over:
msg = _LW("Quota exceeded for %(s_pid)s, tried to create "
"%(s_size)sG backup (%(d_consumed)dG of "
"%(d_quota)dG already consumed)")
LOG.warning(msg, {'s_pid': context.project_id,
's_size': volume['size'],
'd_consumed': _consumed(over),
'd_quota': quotas})
raise exception.VolumeBackupSizeExceedsAvailableQuota(
requested=volume['size'],
consumed=_consumed('backup_gigabytes'),
quota=quotas['backup_gigabytes'])
elif 'backups' in over:
msg = _LW("Quota exceeded for %(s_pid)s, tried to create "
"backups (%(d_consumed)d backups "
"already consumed)")
LOG.warning(msg, {'s_pid': context.project_id,
'd_consumed': _consumed(over)})
raise exception.BackupLimitExceeded(
allowed=quotas)
# Find the latest backup and use it as the parent backup to do an
# incremental backup.
latest_backup = None
if incremental:
backups = objects.BackupList.get_all_by_volume(context.elevated(),
volume_id)
if backups.objects:
# NOTE(xyang): The 'data_timestamp' field records the time
# when the data on the volume was first saved. If it is
# a backup from volume, 'data_timestamp' will be the same
# as 'created_at' for a backup. If it is a backup from a
# snapshot, 'data_timestamp' will be the same as
# 'created_at' for a snapshot.
# If not backing up from snapshot, the backup with the latest
# 'data_timestamp' will be the parent; If backing up from
# snapshot, the backup with the latest 'data_timestamp' will
# be chosen only if 'data_timestamp' is earlier than the
# 'created_at' timestamp of the snapshot; Otherwise, the
# backup will not be chosen as the parent.
# For example, a volume has a backup taken at 8:00, then
# a snapshot taken at 8:10, and then a backup at 8:20.
# When taking an incremental backup of the snapshot, the
# parent should be the backup at 8:00, not 8:20, and the
# 'data_timestamp' of this new backup will be 8:10.
latest_backup = max(
backups.objects,
key=lambda x: x['data_timestamp']
if (not snapshot or (snapshot and x['data_timestamp']
< snapshot['created_at']))
else datetime(1, 1, 1, 1, 1, 1, tzinfo=timezone('UTC')))
else:
msg = _('No backups available to do an incremental backup.')
raise exception.InvalidBackup(reason=msg)
parent_id = None
if latest_backup:
parent_id = latest_backup.id
if latest_backup['status'] != fields.BackupStatus.AVAILABLE:
msg = _('The parent backup must be available for '
'incremental backup.')
raise exception.InvalidBackup(reason=msg)
data_timestamp = None
if snapshot_id:
snapshot = objects.Snapshot.get_by_id(context, snapshot_id)
data_timestamp = snapshot.created_at
self.db.volume_update(context, volume_id,
{'status': 'backing-up',
'previous_status': previous_status})
backup = None
try:
kwargs = {
'user_id': context.user_id,
'project_id': context.project_id,
'display_name': name,
'display_description': description,
'volume_id': volume_id,
'status': fields.BackupStatus.CREATING,
'container': container,
'parent_id': parent_id,
'size': volume['size'],
'host': host,
'snapshot_id': snapshot_id,
'data_timestamp': data_timestamp,
}
backup = objects.Backup(context=context, **kwargs)
backup.create()
if not snapshot_id:
backup.data_timestamp = backup.created_at
backup.save()
QUOTAS.commit(context, reservations)
except Exception:
with excutils.save_and_reraise_exception():
try:
if backup and 'id' in backup:
backup.destroy()
finally:
QUOTAS.rollback(context, reservations)
# TODO(DuncanT): In future, when we have a generic local attach,
# this can go via the scheduler, which enables
# better load balancing and isolation of services
self.backup_rpcapi.create_backup(context, backup) # rpc请求通过消息队列这里不贴出来了
return backup
cinder/backup/manager.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
def create_backup(self, context, backup):
"""Create volume backups using configured backup service."""
volume_id = backup.volume_id
volume = objects.Volume.get_by_id(context, volume_id)
previous_status = volume.get('previous_status', None)
LOG.info(_LI('Create backup started, backup: %(backup_id)s '
'volume: %(volume_id)s.'),
{'backup_id': backup.id, 'volume_id': volume_id})
self._notify_about_backup_usage(context, backup, "create.start")
backup.host = self.host
backup.service = self.driver_name
backup.availability_zone = self.az
backup.save()
expected_status = 'backing-up'
actual_status = volume['status']
if actual_status != expected_status:
err = _('Create backup aborted, expected volume status '
'%(expected_status)s but got %(actual_status)s.') % {
'expected_status': expected_status,
'actual_status': actual_status,
}
self._update_backup_error(backup, context, err)
raise exception.InvalidVolume(reason=err)
expected_status = fields.BackupStatus.CREATING
actual_status = backup.status
if actual_status != expected_status:
err = _('Create backup aborted, expected backup status '
'%(expected_status)s but got %(actual_status)s.') % {
'expected_status': expected_status,
'actual_status': actual_status,
}
self._update_backup_error(backup, context, err)
backup.save()
raise exception.InvalidBackup(reason=err)
try:
self._run_backup(context, backup, volume) # 跟进去
except Exception as err:
with excutils.save_and_reraise_exception():
self.db.volume_update(context, volume_id,
{'status': previous_status,
'previous_status': 'error_backing-up'})
self._update_backup_error(backup, context, six.text_type(err))
# Restore the original status.
self.db.volume_update(context, volume_id,
{'status': previous_status,
'previous_status': 'backing-up'})
backup.status = fields.BackupStatus.AVAILABLE
backup.size = volume['size']
backup.save()
# Handle the num_dependent_backups of parent backup when child backup
# has created successfully.
if backup.parent_id:
parent_backup = objects.Backup.get_by_id(context,
backup.parent_id)
parent_backup.num_dependent_backups += 1
parent_backup.save()
LOG.info(_LI('Create backup finished. backup: %s.'), backup.id)
self._notify_about_backup_usage(context, backup, "create.end")
def _run_backup(self, context, backup, volume):
backup_service = self.service.get_backup_driver(context)
properties = utils.brick_get_connector_properties()
backup_dic = self.volume_rpcapi.get_backup_device(context,
backup, volume)
try:
backup_device = backup_dic.get('backup_device')
is_snapshot = backup_dic.get('is_snapshot')
attach_info = self._attach_device(context, backup_device,
properties, is_snapshot)
try:
device_path = attach_info['device']['path']
if isinstance(device_path, six.string_types):
if backup_dic.get('secure_enabled', False):
with open(device_path) as device_file:
backup_service.backup(backup, device_file)
else:
with utils.temporary_chown(device_path):
with open(device_path) as device_file:
backup_service.backup(backup, device_file)
else:
backup_service.backup(backup, device_path)
finally:
self._detach_device(context, attach_info,
backup_device, properties,
is_snapshot)
finally:
backup = objects.Backup.get_by_id(context, backup.id)
self._cleanup_temp_volumes_snapshots_when_backup_created(
context, backup)
cinder/backup/drivers/nfs.py (def get_backup_driver) --》 cinder/volume/rpcapi.py(def get_backup_device)--》cinder/volume/manager.py(def get_backup_device)
--》cinder/volume/driver.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
def get_backup_device(self, context, backup):
"""Get a backup device from an existing volume.
The function returns a volume or snapshot to backup service,
and then backup service attaches the device and does backup.
"""
backup_device = None
is_snapshot = False
if (self.backup_use_temp_snapshot() and
self.snapshot_remote_attachable()):
(backup_device, is_snapshot) = (
self._get_backup_volume_temp_snapshot(context, backup))
else:
backup_device = self._get_backup_volume_temp_volume(
context, backup)
is_snapshot = False
return (backup_device, is_snapshot)
def _get_backup_volume_temp_volume(self, context, backup):
"""Return a volume to do backup.
To backup a snapshot, create a temp volume from the snapshot and
back it up.
Otherwise to backup an in-use volume, create a temp volume and
back it up.
"""
volume = objects.Volume.get_by_id(context, backup.volume_id)
snapshot = None
if backup.snapshot_id:
snapshot = objects.Snapshot.get_by_id(context, backup.snapshot_id)
LOG.debug('Creating a new backup for volume %s.', volume['name'])
temp_vol_ref = None
device_to_backup = volume
# NOTE(xyang): If it is to backup from snapshot, create a temp
# volume from the source snapshot, backup the temp volume, and
# then clean up the temp volume.
if snapshot:
temp_vol_ref = self._create_temp_volume_from_snapshot(
context, volume, snapshot)
backup.temp_volume_id = temp_vol_ref['id']
backup.save()
device_to_backup = temp_vol_ref
else:
# NOTE(xyang): Check volume status if it is not to backup from
# snapshot; if 'in-use', create a temp volume from the source
# volume, backup the temp volume, and then clean up the temp
# volume; if 'available', just backup the volume.
previous_status = volume.get('previous_status')
if previous_status == "in-use":
temp_vol_ref = self._create_temp_cloned_volume(
context, volume)
backup.temp_volume_id = temp_vol_ref['id']
backup.save()
device_to_backup = temp_vol_ref
return device_to_backup
def _create_temp_cloned_volume(self, context, volume):
temp_volume = {
'size': volume['size'],
'display_name': 'backup-vol-%s' % volume['id'],
'host': volume['host'],
'user_id': context.user_id,
'project_id': context.project_id,
'status': 'creating',
'attach_status': 'detached',
'availability_zone': volume.availability_zone,
}
temp_vol_ref = self.db.volume_create(context, temp_volume)
try:
self.create_cloned_volume(temp_vol_ref, volume)
except Exception:
with excutils.save_and_reraise_exception():
self.db.volume_destroy(context.elevated(),
temp_vol_ref['id'])
self.db.volume_update(context, temp_vol_ref['id'],
{'status': 'available'})
return temp_vol_ref
--》cinder/volume/drivers/remotefs.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
@locked_volume_id_operation
def create_cloned_volume(self, volume, src_vref):
"""Creates a clone of the specified volume."""
return self._create_cloned_volume(volume, src_vref)
def _create_cloned_volume(self, volume, src_vref):
LOG.info(_LI('Cloning volume %(src)s to volume %(dst)s'),
{'src': src_vref['id'],
'dst': volume['id']})
if src_vref['status'] != 'available':
msg = _("Volume status must be 'available'.")
raise exception.InvalidVolume(msg)
volume_name = CONF.volume_name_template % volume['id']
volume_info = {'provider_location': src_vref['provider_location'],
'size': src_vref['size'],
'id': volume['id'],
'name': volume_name,
'status': src_vref['status']}
temp_snapshot = {'volume_name': volume_name,
'size': src_vref['size'],
'volume_size': src_vref['size'],
'name': 'clone-snap-%s' % src_vref['id'],
'volume_id': src_vref['id'],
'id': 'tmp-snap-%s' % src_vref['id'],
'volume': src_vref}
self._create_snapshot(temp_snapshot) # 做快照
try:
self._copy_volume_from_snapshot(temp_snapshot, # 备份glusterfs卷实际上通过qemu快照方式来备份
volume_info,
volume['size'])
finally:
self._delete_snapshot(temp_snapshot) # 删除这个过渡的快照
return {'provider_location': src_vref['provider_location']}
这里备份glusterfs卷的时候，原来的卷会处于backing-up的状态，这是个社区bug，详情见 https://review.openstack.org/#/c/288875/
页:
[1]