openstack octavia 手工安装

yutrew

openstack octavia 是 openstack lbaas的支持的一种后台程序，提供为虚拟机流量的负载均衡。实质是类似于trove，调用 nove 以及neutron的api生成一台安装好haproxy软件的虚拟机，并连接到目标网路。具体原理和devstack安装方法 参考
http://lingxiankong.github.io/bl ... utm_medium=referral
http://docs.openstack.org/develo ... -with-lbaas-v2.html

目前官方不提供安装文档。谷歌了下似乎也没人写过具体的安装步骤，只推荐用devstack来进行安装。本人尝试根据devstack的安装脚本总结了下安装octavia的步骤，不当之处请各位指正。
1、创建数据库

1 2	mysql> CREATE DATABASE octavia; mysql> GRANT ALL PRIVILEGES ON octavia.* TO 'octavia'@'localhost'  IDENTIFIED BY 'OCTAVIA_DBPASS';mysql> GRANT ALL PRIVILEGES ON octavia.* TO 'octavia'@'%' \  IDENTIFIED BY 'OCTAVIA_DBPASS';

2 创建用户 角色 endpoint

1 2 3 4 5

openstack user create --domain default --password-prompt octavia openstack role add --project service --user cinder admin openstack endpoint create octavia public http://10.1.65.58:9876/ --region RegionOne openstack endpoint create octavia admin http://10.1.65.58:9876/ --region RegionOne openstack endpoint create octavia internal http://10.1.65.58:9876/ --region RegionOne

3 安装软件包

1	yum install openstack-octavia-worker openstack-octavia-api python-octavia openstack-octavia openstack-octavia openstack-octavia

4 导入镜像 镜像是从devstack 生成的系统中导出来的

1	openstack  image create amphora-x64-haproxy --public --container-format=bare --disk-format qcow2

5 创建管理网络，并在主机创建ovs端口，使octavia-worker，octavia-housekeeping，octavia-health-manager能和生成的虚拟机实例通讯
5.1 生成管理网络，网段

1 2	openstack network create lb-mgmt-net openstack subnet create --subnet-range 192.168.0.0/24 --allocation-pool       start=192.168.0.2,end=192.168.0.200 --network lb-mgmt-net lb-mgmt-subnet

5.2 生成管理端口防火墙规则
5555端口是管理网络，考虑到octavia组件尚不成熟，开启了22端口，镜像本身也是开启了22端口，这点吐槽下trove，同样是不成熟的模块，默认不开启22端口，还得去改源码。

1 2 3

openstack security group create lb-mgmt-sec-grp openstack security group rule create --protocol udp --dst-port 5555 lb-health-mgr-sec-grp openstack security group rule create --protocol tcp --dst-port 22 lb-mgmt-sec-grp

5.3 在管理网络创建一个端口用于连接宿主机中的octavia health_manager

1	neutron port-create --name octavia-health-manager-standalone-listen-port --security-group lb-health-mgr-sec-grp --device-owner Octavia:health-mgr --binding:host_id=controller lb-mgmt-net

5.4 创建宿主机的ovs端口 并连接至5.1生成的网络

1	ovs-vsctl  --may-exist add-port br-int o-hm0 -- set Interface o-hm0 type=internal -- set Interface o-hm0 external-ids:iface-status=active -- set Interface o-hm0 external-ids:attached-mac=fa:16:3e:6f:9f:9a -- set Interface o-hm0 external-ids:iface-id=457e4953-b2d6-49ee-908b-2991506602b2

其中iface-id 和attached-mac 为 5.3生成的port的 属性

1	ip link set dev o-hm0 address fa:16:3e:6f:9f:9a

5.5 在宿主机上创建dhcp （为啥不用传统的dnsmasq呢？）

1	dhclient -v o-hm0 -cf /etc/octavia/dhcp/dhclient.conf

6 配置修改，和其他openstack组件设置差不多
  6.1 设置数据库

1 2	[database] connection = mysql+pymysql://octavia:octavia@controller/octavia

  6.2 设置消息队列

1 2 3 4

[oslo_messaging_rabbit] rabbit_host = controller rabbit_userid = openstack rabbit_password = RABBIT_PASS

6.3 设置 keystone的认证信息

1 2 3 4 5 6

[keystone_authtoken] auth_version = 2 admin_password = OCTAVIA_PASS admin_tenant_name = octavia admin_user = octavia auth_uri = http://controller:5000/v2.0

6.4 设置health_manager组件监听地址，此ip地址等于5.3中创建的io地址

1 2 3 4

[health_manager] bind_port = 5555 bind_ip = 192.168.0.7 controller_ip_port_list = 192.168.0.7:5555

6.5 设置和虚拟机通信的 公钥私钥
[haproxy_amphora]
server_ca = /etc/octavia/certs/ca_01.pem
client_cert = /etc/octavia/certs/client.pem
key_path = /etc/octavia/.ssh/octavia_ssh_key
base_path = /var/lib/octavia
base_cert_dir = /var/lib/octavia/certs
connection_max_retries = 1500
connection_retry_interval = 1
6.6 设置 用于开启虚拟机实例的信息

1 2 3 4 5 6 7 8 9 10 11 12

[controller_worker] amp_boot_network_list = 826be4f4-a23d-4c5c-bff5-7739936fac76 # 步骤5.1中生成的id amp_image_tag = amphora # 步骤4 中已经定义这个metadata amp_secgroup_list = d949202b-ba09-4003-962f-746ae75809f7 # 步骤5.2 生成的安全组id amp_flavor_id = dd49b3d5-4693-4407-a76e-2ca95e00a9ec amp_image_id = b23dda5f-210f-40e6-9c2c-c40e9daa661a #步骤4中生成的镜像id amp_ssh_key_name = 155 # amp_active_wait_sec = 1 amp_active_retries = 100 network_driver = allowed_address_pairs_driver compute_driver = compute_nova_driver amphora_driver = amphora_haproxy_rest_driver

7 修改neutron配置
  7.1 修改 /etc/neutron/neutron.conf 增加lbaas服务

1	service_plugins = [existing service plugins],neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPluginv2

7.2 在[service_providers] 章节 设置lbaas 的服务提供者为octavia

1	service_provider = LOADBALANCERV2:Octavia:neutron_lbaas.drivers.octavia.driver.OctaviaDriver:default

8 启动服务
如果之前 开启了 LBaaS v2 with an agent 服务 请关闭，并清理下neutron数据库下 lbaas_loadbalancers lbaas_loadbalancer_statistics 不然会报错
重启neutron

1	systemctl restart neutron-server

启动octavia

1	systemctl restart  octavia-housekeeping  octavia-worker octavia-api octavia-health-manager

9验证操作
9.1 创建loadbalancer

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

[iyunv@controller ~]# neutron lbaas-loadbalancer-create --name test-lb-1 lbtest Created a new loadbalancer: +---------------------+--------------------------------------+ | Field               | Value                                | +---------------------+--------------------------------------+ | admin_state_up      | True                                 | | description         |                                      | | id                  | 5af472bb-2068-4b96-bcb3-bef7ff7abc56 | | listeners           |                                      | | name                | test-lb-1                            | | operating_status    | OFFLINE                              | | pools               |                                      | | provider            | octavia                              | | provisioning_status | PENDING_CREATE                       | | tenant_id           | 9a4b2de78c2d45cfbf6880dd34877f7b     | | vip_address         | 192.168.123.10                       | | vip_port_id         | d163b73c-258a-4e03-90ad-5db31cfe23ac | | vip_subnet_id       | 74aea53a-014a-4f9c-86f9-805a2a772a27 | +---------------------+--------------------------------------+

9.2 查看虚拟机，值得注意的地方，loadbalancer的地址是vip，和虚拟机的地址是不相同的

1 2	[iyunv@controller ~]# openstack server list |grep 82b59e85-29f2-46ce-ae0b-045b7fceb5ca | 82b59e85-29f2-46ce-ae0b-045b7fceb5ca | amphora-734da57c-e444-4b8e-a706-455230ae0803 | ACTIVE  | lbtest=192.168.123.9; lb-mgmt-net=192.168.0.6        | amphora-x64-haproxy 201610131607    |

9.3 创建linstener

1	neutron lbaas-listener-create --name test-lb-tcp --loadbalancer test-lb-1 --protocol TCP  --protocol-port 22

9.4 设置安全组

1	neutron port-update  --security-group default d163b73c-258a-4e03-90ad-5db31cfe23ac

9.5 创建pool,新建三台虚拟机，并加入pool

1 2 3 4 5 6 7 8 9 10 11 12 13

openstack server create  --flavor m1.small --nic net-id=22525640-297e-40eb-bd77-0a9afd861f8c --image "cirros for kvm raw"  --min 3 --max 3 test [iyunv@controller ~]# openstack server list |grep test- | d8dc22d4-e657-4c54-96f9-3a53ca67533d | test-3                                       | ACTIVE  | lbtest=192.168.123.8                                 | cirros for kvm raw                  | | c7926665-84c5-48a5-9de5-5e15e71baa5d | test-2                                       | ACTIVE  | lbtest=192.168.123.13                                | cirros for kvm raw                  | | fcf60c23-b799-4d08-a5a7-2b0fc9f1905e | test-1                                       | ACTIVE  | lbtest=192.168.123.11                                | cirros for kvm raw                  | neutron lbaas-pool-create   --name test-lb-pool-tcp  --lb-algorithm ROUND_ROBIN --listener test-lb-tcp --protocol TCP for i in {8,13,11} do neutron lbaas-member-create --subnet lbtest  --address 192.168.123.${i}  --protocol-port 22  test-lb-pool-tcp done

9.6 验证

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

[iyunv@controller ~]# >/root/.ssh/known_hosts;ip netns exec qrouter-4718cc34-68cc-47a7-9201-405d1fc09213 ssh cirros@192.168.123.10 "hostname" The authenticity of host '192.168.123.10 (192.168.123.10)' can't be established. RSA key fingerprint is 72:c4:11:41:53:51:f2:1b:b5:e6:1b:69:a8:c2:5b:d4. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.123.10' (RSA) to the list of known hosts. cirros@192.168.123.10's password: test-3 [iyunv@controller ~]# >/root/.ssh/known_hosts;ip netns exec qrouter-4718cc34-68cc-47a7-9201-405d1fc09213 ssh cirros@192.168.123.10 "hostname" The authenticity of host '192.168.123.10 (192.168.123.10)' can't be established. RSA key fingerprint is 3d:88:0f:4a:b1:77:c9:6a:fd:82:4d:31:0c:ca:82:d8. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.123.10' (RSA) to the list of known hosts. cirros@192.168.123.10's password: test-1 [iyunv@controller ~]# >/root/.ssh/known_hosts;ip netns exec qrouter-4718cc34-68cc-47a7-9201-405d1fc09213 ssh cirros@192.168.123.10 "hostname" The authenticity of host '192.168.123.10 (192.168.123.10)' can't be established. RSA key fingerprint is 1c:03:f0:f9:92:a7:0f:5d:9d:09:22:14:94:62:e4:c4. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.123.10' (RSA) to the list of known hosts. cirros@192.168.123.10's password: test-2