opentsack
openstack 调用ceph-rbd块存储环境描述
openstack 版本 QUEENS
ceph 版本 Luminous 12.2.5
openstack和ceph集群安装部署步骤，后续文章介绍；
此环境建立在已有一套ceph-rbd和openstack运行正常的环境；
块存储调用关系逻辑图如下：
http://i2.运维网.com/images/blog/201808/22/251e3437f40179aa385847a189bf07b8.png
openstack-cinder
openstack需要先安装一套cinder服务，可以选择安装在controller节点或者额外选择一台机器单独作为cinder服务；
在此，额外选择一台服务器作为cinder服务；
1)配置yum源，安装openstack-cinder；
yum install centos-release-openstack-queens
yum install https://rdoproject.org/repos/rdo-release.rpm
yum upgrade
yum install python-openstackclient
yum install openstack-cinder targetcli python-keystone
2)安装完成后，直接修改cinder配置文件；
vim /etc/cinder/cinder.conf
配置项目，keystone/rabbitmq/controller指向controller节点；
注意账户和口令修改，uuid修改；
将conrtoller主机地址解析加入本地hosts文件，若有本地DNS解析，则忽略此项；
###去除注释行和空行###
transport_url = rabbit://openstack:openstack_password@controller
auth_strategy = keystone
my_ip = 192.168.17.8
enabled_backends = ceph
glance_api_servers = http://controller:9292
glance_api_version = 2
volume_driver = cinder.volume.drivers.rbd.RBDDriver
rbd_pool = volumes
rbd_ceph_conf = /etc/ceph/ceph.conf
rbd_flatten_volume_from_snapshot = false
volume_backend_name = ceph
rbd_max_clone_depth = 5
rbd_store_chunk_size = 4
rados_connect_timeout = -1
glance_api_version = 2
rbd_user = cinder
rbd_secret_uuid = *****
backup_driver = cinder.backup.drivers.ceph
backup_ceph_conf = /etc/ceph/ceph.conf
backup_ceph_pool = backups
backup_ceph_user = cinder-backup
backup_ceph_chunk_size = 134217728
backup_ceph_stripe_unit = 0
backup_ceph_stripe_count = 0
restore_discard_excess_bytes = true
log_dir = /var/log/cinder/cinder.log
connection = mysql+pymysql://cinder:cinder_pass@controller/cinder
auth_uri = http://controller:5000
auth_url = http://controller:35357
memcached_servers = controller:11211
auth_type = password
project_domain_id = default
user_domain_id = default
project_name = service
username = cinder
password = cinder_password
lock_path = /var/lib/cinder/tmp
3)controller节点配置
###数据库创建账户，授权###
mysql -u root -p
CREATE DATABASE cinder;
GRANT ALL PRIVILEGES ON cinder. TO 'cinder'@'localhost' IDENTIFIED BY 'stack';
GRANT ALL PRIVILEGES ON cinder. TO 'cinder'@'%' IDENTIFIED BY 'stack';
###创建openstak接口，创建账户###
.admin-openrc
openstack user create --domain default --password-prompt cinder
openstack role add --project service --user cinder admin
openstack service create --name cinderv2 --description "OpenStack Block Storage" volumev2
openstack service create --name cinderv3 --description "OpenStack Block Storage" volumev3
openstack endpoint create --region RegionOne \
volumev2 public http://controller:8776/v2/%\(project_id\)s
openstack endpoint create --region RegionOne \
volumev2 internal http://controller:8776/v2/%\(project_id\)s
openstack endpoint create --region RegionOne \
volumev2 admin http://controller:8776/v2/%\(project_id\)s
openstack endpoint create --region RegionOne \
volumev3 public http://controller:8776/v3/%\(project_id\)s
openstack endpoint create --region RegionOne \
volumev3 internal http://controller:8776/v3/%\(project_id\)s
openstack endpoint create --region RegionOne \
volumev3 admin http://controller:8776/v3/%\(project_id\)s
###安装openstack-cinder服务###
yum install openstack-cinder
###修改cinder.conf配置文件###
vim /etc/cinder/cinder.conf
my_ip = 172.16.188.9
auth_strategy = keystone
enabled_backends = lvm
transport_url = rabbit://openstack:openstack_password@controller
connection = mysql+pymysql://cinder:cinder_password@controller/cinder
auth_uri = http://controller:5000
auth_url = http://controller:35357
memcached_servers = controller:11211
auth_type = password
project_domain_id = default
user_domain_id = default
project_name = service
username = cinder
password = cinder_password
lock_path = /var/lib/cinder/tmp
4)调用ceph-rbd块存储配置
配置思路：
登陆ceph集群服务器；
安装ceph驱动；
创建存储池；
授权账户；
拷贝密钥文件；
配置libvirt调用ceph-rbd；
###安装ceph驱动###
yum install python-rbd
yum install ceph-common
###创建存储池###
ceph osd pool create volumes 128
ceph osd pool create backups 128
ceph osd pool create images 128
ceph osd pool create vms 128
###创建授权账户###
ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rx pool=images'
ceph auth get-or-create client.cinder-backup mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=backups'
ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images'
###拷贝密钥文件###
ceph auth get-or-create client.cinder | ssh {your-volume-server} sudo tee /etc/ceph/ceph.client.cinder.keyring
ssh {your-cinder-volume-server} chown cinder:cinder /etc/ceph/ceph.client.cinder.keyring
ceph auth get-or-create client.cinder-backup | ssh {your-cinder-backup-server} tee /etc/ceph/ceph.client.cinder-backup.keyring
ssh {your-cinder-backup-server} chown cinder:cinder /etc/ceph/ceph.client.cinder-backup.keyring
ceph auth get-or-create client.glance | ssh {your-glance-api-server} sudo tee /etc/ceph/ceph.client.glance.keyring
ssh {your-glance-api-server} chown glance:glance /etc/ceph/ceph.client.glance.keyring
ceph auth get-or-create client.cinder | ssh {your-nova-compute-server} tee /etc/ceph/ceph.client.cinder.keyring
5)配置libvrit调用ceph-rbd
uuidgen > uuid-secret.txt
cat > secret.xml
页:
[1]