Opestack Juno for CentOS7 架构安装笔记（swift篇）

121thre

1、安装配置swift存储节点
(1)安装swift软件

1	yum -y install openstack-utils openstack-swift-account openstack-swift-container openstack-swift-object xfsprogs python-keystone

(2)新建硬盘分区，这里其余2块分区一样的

1 2	fdisk /dev/sdb mkfs.xfs /dev/sdb1

(3)建立挂载目录，写进开机启动挂载

1 2 3 4

mkdir -pv  /srv/node/sd{b,c,d}1 echo "/dev/sdb1 /srv/node/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab echo "/dev/sdc1 /srv/node/sdc1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab echo "/dev/sdd1 /srv/node/sdd1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab

(4)下载并修改/etc/swift/account-server.conf文件

1 2	curl -o /etc/swift/account-server.conf \   https://raw.githubusercontent.co ... -server.conf-sample

1 2 3 4 5 6 7 8 9 10 11 12

[DEFAULT] ... bind_ip = 10.90.2.100 bind_port = 6002 user = swift swift_dir = /etc/swift devices = /srv/node [pipeline:main] pipeline = healthcheck recon account-server [filter:recon] ... recon_cache_path = /var/cache/swift

(5)下载并修改/etc/swift/container-server.conf 文件  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

curl -o /etc/swift/container-server.conf \   https://raw.githubusercontent.co ... -server.conf-sample     [DEFAULT] ... bind_ip = 10.90.2.100 bind_port = 6001 user = swift swift_dir = /etc/swift devices = /srv/node   [pipeline:main] pipeline = healthcheck recon container-server [filter:recon] ... recon_cache_path = /var/cache/swift

(6)下载并修改/etc/swift/object-server.conf 文件
c

1 2 3 4 5 6 7 8 9 10 11 12 13 14

url -o /etc/swift/object-server.conf \   https://raw.githubusercontent.co ... -server.conf-sample   [DEFAULT] ... bind_ip = 10.90.2.100 bind_port = 6000 user = swift swift_dir = /etc/swift devices = /srv/node [pipeline:main] pipeline = healthcheck recon object-server [filter:recon] ... recon_cache_path = /var/cache/swift

(7)建立目录添加权限  

1 2 3

chown -R swift:swift /srv/node mkdir -p /var/cache/swift chown -R swift:swift /var/cache/swift

(8)创建所需ring，通过“swift-ring-builder”命令创建account、container和object所用的ring文件，注意要在/etc/swift目录下执行

1 2 3 4

cd /etc/swift swift-ring-builder account.builder create 10 3 1 swift-ring-builder container.builder create 10 3 1 swift-ring-builder object.builder create 10 3 1

上面的命令后面三个参数中，18表示构建一致hash环时其环的大小，即使用2^18大小的环。实际使用中，可以根据节点数规模进行调整，最大为32；3表示为每个对象存储3个副本；最后的1表示多长时间可以移动一次某partiton。

(9)将存储设备关系至ring

在ring文件创建完成后，还需要将每个存储设备添加至每个ring上，而且通常需要以account、container和object的顺序进行。其格式为：

swift-ring-builder account.builder add z<ZONE>-<STORAGE_LOCAL_NET_IP>:6002/<DEVICE> 100
swift-ring-builder container.builder add z<ZONE>-<STORAGE_LOCAL_NET_IP_1>:6001/<DEVICE> 100
swift-ring-builder object.builder add z<ZONE>-<STORAGE_LOCAL_NET_IP_1>:6000/<DEVICE> 100

其中<ZONE>为zone编号，如z1表示第一个zone；<STORAGE_LOCAL_NET_IP>表示存储节点自己的IP地址；<DEVICE>表示对应设备的设备名称或挂载点名称(二者通常相同)，如/dev/sdb1设备的挂载点为/srv/node/sdb1时，其<DEVICE>即为sdb1。因此，将sdb1、sdc1和sdd1三个设备添加至ring文件可通过如下命令实现。

1 2 3 4 5 6 7 8 9

swift-ring-builder account.builder add z1-10.90.2.100:6002/sdb1 100 swift-ring-builder container.builder add z1-10.90.2.100:6001/sdb1 100 swift-ring-builder object.builder add z1-10.90.2.100:6000/sdb1 100 swift-ring-builder account.builder add z2-10.90.2.100:6002/sdc1 100 swift-ring-builder container.builder add z2-10.90.2.100:6001/sdc1 100 swift-ring-builder object.builder add z2-10.90.2.100:6000/sdc1 100 swift-ring-builder account.builder add z3-10.90.2.100:6002/sdd1 100 swift-ring-builder container.builder add z3-10.90.2.100:6001/sdd1 100 swift-ring-builder object.builder add z3-10.90.2.100:6000/sdd1 100

(10)查看每个ring文件的内容，验正添加结果：

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

# swift-ring-builder account.builder account.builder, build version 3 262144 partitions, 3 replicas, 3 zones, 3 devices, 100.00 balance The minimum number of hours before a partition can be reassigned is 1 Devices:    id  zone     ip address  port      name weight partitions balance meta              0     1    10.90.2.100  6002      sdb1 100.00          0 -100.00              1     2    10.90.2.100  6002      sdc1 100.00          0 -100.00              2     3    10.90.2.100  6002      sdd1 100.00          0 -100.00 # swift-ring-builder container.builder container.builder, build version 3 262144 partitions, 3 replicas, 3 zones, 3 devices, 100.00 balance The minimum number of hours before a partition can be reassigned is 1 Devices:    id  zone     ip address  port      name weight partitions balance meta              0     1    10.90.2.100  6001      sdb1 100.00          0 -100.00              1     2    10.90.2.100  6001      sdc1 100.00          0 -100.00              2     3    10.90.2.100  6001      sdd1 100.00          0 -100.00 # swift-ring-builder object.builder object.builder, build version 3 262144 partitions, 3 replicas, 3 zones, 3 devices, 100.00 balance The minimum number of hours before a partition can be reassigned is 1 Devices:    id  zone     ip address  port      name weight partitions balance meta              0     1    10.90.2.100  6000      sdb1 100.00          0 -100.00              1     2    10.90.2.100  6000      sdc1 100.00          0 -100.00              2     3    10.90.2.100  6000      sdd1 100.00          0 -100.00

(11)使用如下命令重新平衡三个环，即将对应的设备分布式环中的节点上。不过，执行这些命令可能会需要一些时间。

1 2 3

swift-ring-builder account.builder rebalance swift-ring-builder container.builder rebalance swift-ring-builder object.builder rebalance

(12)复制 account.ring.gz、container.ring.gz 和 object.ring.gz 文件到每个存储节点和其他运行了代理服务的额外节点的 /etc/swift 目录下
确保每个节点的/etc/swift目录及其内部子目录和文件的属主为swift用户，属组为swift组。

1	chown -R swift:swift /etc/swift

(13)配置hash和默认的存储策略

1 2 3 4 5 6 7 8 9 10 11 12 13 14

#curl -o /etc/swift/swift.conf \   https://raw.githubusercontent.co ... c/swift.conf-sample #openssl rand -hex 10 89e49ff5a001520f4150 [swift-hash] ... swift_hash_path_suffix = 89e49ff5a001520f4150 swift_hash_path_prefix = 89e49ff5a001520f4150 [storage-policy:0] ... name = Policy-0 default = yes

(14)复制 swift.conf 文件到每个存储节点和其他运行了代理服务的额外节点的 /etc/swift 目录下,在所有节点上，确认配置文件目录是否有合适的所有权：

1 2	scp /etc/swift/*.gz 10.90.2.1:/etc/swift ssh 10.90.2.1 "chown -R swift:swift"

(15)在存储节点上，启动对象存储服务，并将其设置为随系统启动：

1 2 3 4 5 6 7 8 9 10 11 12 13

systemctl enable openstack-swift-account.service openstack-swift-account-auditor.service \   openstack-swift-account-reaper.service openstack-swift-account-replicator.service systemctl enable openstack-swift-container.service openstack-swift-container-auditor.service \   openstack-swift-container-replicator.service openstack-swift-container-updater.service systemctl enable openstack-swift-object.service openstack-swift-object-auditor.service \   openstack-swift-object-replicator.service openstack-swift-object-updater.service       systemctl status openstack-swift-account.service openstack-swift-account-auditor.service \   openstack-swift-account-reaper.service openstack-swift-account-replicator.service systemctl status openstack-swift-container.service openstack-swift-container-auditor.service \   openstack-swift-container-replicator.service openstack-swift-container-updater.service systemctl status openstack-swift-object.service openstack-swift-object-auditor.service \   openstack-swift-object-replicator.service openstack-swift-object-updater.service

2、安装配置proxy_swift代理节点,在控制节点
(1)安装proxy基本软件

1	yum -y install openstack-swift-proxy memcached python-keystone openstack-utils python-keystoneclient

(2)启动memcached服务,开机自起(这个如果控制节点装了就不需要在执行了)

1 2	systemctl start memcached systemctl enable memcached

(3)修改配置swift-proxy配置文件/etc/swift/proxy-server.conf

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

curl -o /etc/swift/proxy-server.conf \   https://raw.githubusercontent.co ... -server.conf-sample     [DEFAULT] ... bind_port = 8080 user = swift swift_dir = /etc/swift [pipeline:main] pipeline = authtoken cache healthcheck  proxy-logging proxy-server keystoneauth(注意这个keystoneauth有问题可以不需要填写) [app:proxy-server] ... allow_account_management = true account_autocreate = true [filter:keystoneauth] use = egg:swift#keystoneauth ... operator_roles = admin,_member_     [filter:cache] ... memcache_servers = 127.0.0.1:11211 [filter:authtoken] paste.filter_factory = keystonemiddleware.auth_token:filter_factory ... auth_host = 10.90.2.1 auth_port = 35357 auth_protocol = http auth_uri = http://10.90.2.1:5000/ admin_tenant_name = service admin_user = swift admin_password = Service

(4)在控制节点和其他运行了代理服务的节点上，启动对象存储代理服务及其依赖服务，并将它们配置为随系统启动

1 2	systemctl enable openstack-swift-proxy.service systemctl start openstack-swift-proxy.service

(5)swift基本验证

1 2 3 4 5 6 7 8 9 10 11 12 13 14

# swift stat                         Account: AUTH_06536a37d6434cbb85834e6a1d9eb421                      Containers: 0                         Objects: 0                           Bytes: 0 Containers in policy "policy-0": 0    Objects in policy "policy-0": 0      Bytes in policy "policy-0": 0     X-Account-Project-Domain-Id: default                      Connection: keep-alive                     X-Timestamp: 1445927026.21389                      X-Trans-Id: txd6da3be6a251407f8f786-00562f275a                    Content-Type: text/plain; charset=utf-8                   Accept-Ranges: bytes

(6)上传文件，查看

1 2 3

#swift upload testfile localfile   #swift list testfile

(7)配置glance+swift，修改/etc/glance/glance-api.conf

1 2 3 4 5 6 7 8 9 10 11

[DEFAULT] default_store = swift swift_store_auth_address=http://10.90.2.1:5000/v2.0/ swift_store_create_container_on_put=True swift_store_user=service:swift swift_store_auth_version=2 swift_store_key=Service123 enable_v2_api=True …… [glance_store] stores=glance.store.swift.Store

(8)重新启动glance服务：

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

systemctl restart openstack-glance-api openstack-glance-registry # glance image-create --name "cirros-3.3.3-x86_64x1" --file cirros-0.3.3-x86_64-disk.img   --disk-format qcow2 --container-format bare --is-public True --progress [=============================>] 100% +------------------+--------------------------------------+ | Property         | Value                                | +------------------+--------------------------------------+ | checksum         | 133eae9fb1c98f45894a4e60d8736619     | | container_format | bare                                 | | created_at       | 2015-11-26T02:17:17                  | | deleted          | False                                | | deleted_at       | None                                 | | disk_format      | qcow2                                | | id               | a6185a87-8291-4053-9ac1-6fb6fe1997f6 | | is_public        | True                                 | | min_disk         | 0                                    | | min_ram          | 0                                    | | name             | cirros-3.3.3-x86_64x1                | | owner            | 5cd48f15e3514988870611ab2eeee536     | | protected        | False                                | | size             | 13200896                             | | status           | active                               | | updated_at       | 2015-11-26T02:17:19                  | | virtual_size     | None                                 | +------------------+--------------------------------------+

(9)登录openstack dashboard可以查看刚刚上传的情况：