CentOS 6.5 Openstack Havana 多节点安装（Flat网络模式）

maxc1017

　　有5种网络模式
　　Local模式：一般测试时使用，只需一台物理机即可。
GRE模式：隧道数量没有限制，性能有点问题。
Vlan模式：vlan数量有4096的限制
VXlan模式：vlan数量没有限制，性能比GRE好。
Flat模式：管理员创建租户直接到外网，不需要NAT了。
　　目前的几个项目正好用到了这种Flat模式的网络结构，我觉得小项目中还是比较方便的，Openstack的体系和框架越来越稳定，项目中用用虚拟 化和存储，直接借助Openstack还是很方便。这时网络结构可能就不需要用租户、或者在细分其它，Flat模式反而更简洁直观，毕竟外网IP直接到了 instance。
　　下面以我实验环境配置为例，存储部分配置省略。
　　环境：controller node一台       compute node一台       OS：Centos6.4
　　系统逻辑结构如下

　　
　　一、controller node 配置
　　1、网络

[iyunv@controller ~]# more /etc/hosts
192.168.3.98    controller
192.168.3.95    compute
[iyunv@controller ~]# more /etc/sysconfig/network-scripts/ifcfg-eth0   #外部网络
DEVICE=eth0
HWADDR=10:BF:48:B7:8F:A0
TYPE=Ethernet
ONBOOT=yes
BOOTPROTO=none
IPADDR=192.168.3.98
NETMASK=255.255.255.0
GATEWAY=192.168.3.1
[iyunv@controller ~]# more /etc/sysconfig/network-scripts/ifcfg-eth1  #vm 及node之间通讯使用
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
[iyunv@controller ~]#  more /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=controller
GATEWAY=192.168.3.1
[iyunv@controller ~]#  more /etc/resolv.conf
nameserver 202.102.134.68
nameserver 202.102.128.68
　　
　　2、添加源 yum install wget cd /etc/yum.repos.d/ wget http://mirrors.163.com/.help/CentOS6-Base-163.repo yum list
　　3、LVM配置 [我单独拿出一个分区给cinder使用，同时我的控制和计算节点也都是cinder节点]
　　[iyunv@controller ~]# umount /dev/sda5 [iyunv@controller ~]# pvcreate /dev/sda5 Physical volume "/dev/sda5" successfully created [iyunv@controller ~]# vgcreate cinder-volumes /dev/sda5 Volume group "cinder-volumes" successfully created
　　4、安装packstack
　　yum install -y http://rdo.fedorapeople.org/openstack/openstack-havana/rdo-release-havana.rpm yum install -y openstack-packstack yum -y update 重启机器
　　5、Packstack配置
　　创建一个应答文件 packstack --gen-answer-file=multi-node.txt 修改应答文件，由于我已经有专门的cinder volume, 所以不需要创建。如果没有手工创建cinder volume，那么这就不修改就可以。 CONFIG_CINDER_VOLUMES_CREATE=n #不在创建volume CONFIG_HEAT_INSTALL=y CONFIG_NAGIOS_INSTALL=y CONFIG_KEYSTONE_ADMIN_PW=admin CONFIG_NOVA_COMPUTE_HOSTS=192.168.3.98 #此处可以添加compute node IP ,号分割 CONFIG_NEUTRON_OVS_TENANT_NETWORK_TYPE=vlan CONFIG_NEUTRON_OVS_VLAN_RANGES=physnet1:100:200 CONFIG_NEUTRON_OVS_BRIDGE_MAPPINGS=physnet1:br-eth1 CONFIG_NEUTRON_OVS_BRIDGE_IFACES=br-eth1:eth1 CONFIG_HEAT_CLOUDWATCH_INSTALL=y CONFIG_HEAT_CFN_INSTALL=y CONFIG_NAGIOS_PW=admin
　　6、安装 [RDO的方式部署简直太简单了，我值截取部分]
　　[iyunv@controller ~]# packstack --answer-file=multi-node.txt #安装过程，开始会输入密码，多节点的话会依此类推。安装过程有可能因为源的原因中断，可以重复运行。 Welcome to Installer setup utility Packstack changed given value to required value /root/.ssh/id_rsa.pub Installing: Clean Up... [ DONE ]
　　7、 装完后，有可能由于RDO的问题，没有创建br-ex，需要我们手工来创建。
【目前看2014年后，redhat已经修复这个bug，剩下的只是我们对应的创建网卡配置文件和启动port】
　　[iyunv@controller network-scripts]# more ifcfg-br-ex DEVICE=br-ex IPADDR=192.168.3.98 PREFIX=24 GATEWAY=192.168.3.1 DNS1=202.102.134.68 ONBOOT=yes 修改 /etc/sysconfig/network-scripts/ifcfg-eth0, 注意：一定要加上eth0 的mac地址，不然会出错。 [iyunv@controller network-scripts]# more ifcfg-eth0 DEVICE=eth0 HWADDR=00:90:27:E0:61:14 #TYPE=Ethernet ONBOOT=yes #BOOTPROTO=none #IPADDR=192.168.3.98 #NETMASK=255.255.255.0 ovs-vsctl add-port br-ex eth0; service network restart 当运行 ovs-vsctl add-port的时候，网络就会中断，所以你两条命令需要一起执行。
　　8、改变桥接网络配置，此处主要修改neutron配置，我们的flat网络模式是依赖在vlan的基础上改进的，也是此文重点之处。
　　编辑 vi /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini 原安装完初始内容如下 [OVS] network_vlan_ranges=physnet1:100:200 tenant_network_type=vlan enable_tunneling=False integration_bridge=br-int bridge_mappings=physnet1:br-eth1 修改成 [OVS] network_vlan_ranges=physnet1,physnet2:100:200 tenant_network_type=vlan enable_tunneling=False integration_bridge=br-int bridge_mappings=physnet1:br-ex,physnet2:br-eth1 然后重启相关服务，如果为省事，重启机器 之后检查网络和桥接情况查询 [iyunv@controller ~]# ovs-vsctl show
　　9、创建Flat模式网络，目前都可以在web界面操作
　　[iyunv@controller ~(keystone_admin)]# neutron net-create Public --provider:network_type flat --provider:physical_network physnet1 Created a new network: +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | id | 3c5eca6f-aeb4-451b-9fa2-ed70d8bab9e9 | | name | Public | | provider:network_type | flat | | provider:physical_network | physnet1 | | provider:segmentation_id | | | shared | False | | status | ACTIVE | | subnets | | | tenant_id | 2cfb66b8a9524cc68c2e1bf424bda943 | +---------------------------+--------------------------------------+ [iyunv@controller ~(keystone_admin)]# neutron subnet-create Public 192.168.3.0/24 --disable-dhcp --allocation-pool start=192.168.3.100,end=192.168.3.110 Created a new subnet: +------------------+----------------------------------------------------+ | Field | Value | +------------------+----------------------------------------------------+ | allocation_pools | {"start": "192.168.3.100", "end": "192.168.3.110"} | | cidr | 192.168.3.0/24 | | dns_nameservers | | | enable_dhcp | False | | gateway_ip | 192.168.3.1 | | host_routes | | | id | 8d089754-3764-4249-98f2-402f62a872bc | | ip_version | 4 | | name | | | network_id | 3c5eca6f-aeb4-451b-9fa2-ed70d8bab9e9 | | tenant_id | 2cfb66b8a9524cc68c2e1bf424bda943 | +------------------+----------------------------------------------------+
　　10、cinder配置
　　查询部署的服务是否正常 [iyunv@controller ~(keystone_admin)]# cinder-manage service list Binary Host Zone Status State Updated At cinder-volume controller nova enabled 2014-02-10 07:42:22 cinder-scheduler controller nova enabled 2014-02-10 07:42:30 [iyunv@controller ~(keystone_admin)]# cinder-manage host list host zone controller nova 然后查询论证是否成功 [iyunv@controller ~(keystone_admin)]# lvdisplay --- Logical volume --- LV Path /dev/cinder-volumes/volume-0a9cc8fc-d3fd-4dd1-9501-9c3813056415 LV Name volume-0a9cc8fc-d3fd-4dd1-9501-9c3813056415 VG Name cinder-volumes LV UUID zbGuJa-3bdk-V6UZ-ZBsx-FWeA-dlys-qMphas LV Write Access read/write LV Creation host, time controller, 2014-02-10 15:46:44 +0800 LV Status available # open 1 LV Size 5.00 GiB Current LE 1280 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 4096 Block device 253:0 [iyunv@controller ~(keystone_admin)]# cinder list +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+ | ID | Status | Display Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+ | 0a9cc8fc-d3fd-4dd1-9501-9c3813056415 | available | oak | 5 | SSD | false | | +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+ [iyunv@controller ~(keystone_admin)]# 此时，启动一个instance 可以在云硬盘中，选择挂载，挂载到相应的instance上。
　　二、compute node 配置
　　1、网络，参考控制节点，其次检查ntp，时间是否与控制节点同步
2、添加源和Openstack软件环境都参考控制节点的前期准备
3、RDO应答文件，填写了compute node IP后，一般nova-compute、openstack-neutron-openvswitch和MySQL-python都已经安装
4、因为我的环境是Flat模式，所以此处neutron配置文件还需修改，然后创建相应的桥
修改neutron
　　vi /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini [OVS] network_vlan_ranges=physnet1,physnet2:100:200 tenant_network_type=vlan enable_tunneling=False integration_bridge=br-int bridge_mappings=physnet1:br-ex,physnet2:br-eth1
　　创建桥
　　[iyunv@compute neutron]# cd /etc/sysconfig/network-scripts/ [iyunv@compute network-scripts]# more ifcfg-br-ex DEVICE=br-ex IPADDR=192.168.3.95 PREFIX=24 GATEWAY=192.168.3.1 DNS1=202.102.134.68 ONBOOT=yes [iyunv@compute network-scripts]# more ifcfg-eth0 DEVICE=eth0 HWADDR=00:90:27:E0:61:14 #TYPE=Ethernet ONBOOT=yes #BOOTPROTO=none #IPADDR=192.168.3.95 #NETMASK=255.255.255.0 #GATEWAY=192.168.3.1 [iyunv@compute network-scripts]# ovs-vsctl add-br br-ex [iyunv@compute network-scripts]# ovs-vsctl add-port br-ex eth0; service network restart [iyunv@compute network-scripts]# ovs-vsctl show 3875869a-c249-4514-aadc-b95b92e7f304 Bridge br-ex Port br-ex Interface br-ex type: internal Port "eth0" Interface "eth0" Bridge br-int Port br-int Interface br-int type: internal ovs_version: "1.11.0"
　　5、cinder 配置
　　[iyunv@compute ~]# umount /dev/sda4 [iyunv@compute ~]# pvcreate /dev/sda4 Physical volume "/dev/sda4" successfully created [iyunv@compute ~]# vgcreate cinder-volumes /dev/sda4 Volume group "cinder-volumes" successfully created yum install openstack-cinder openstack-utils vi /etc/cinder/api-paste.ini 参考拷贝控制节点内容 [filter:authtoken] paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory # signing_dir is configurable, but the default behavior of the authtoken # middleware should be sufficient. It will create a temporary directory # in the home directory for the user the cinder process is running as. #signing_dir = /var/lib/cinder/keystone-signing admin_tenant_name=services auth_host=192.168.3.98 service_port=5000 auth_uri=http://192.168.3.98:5000/ auth_port=35357 service_host=192.168.3.98 service_protocol=http admin_user=cinder auth_protocol=http admin_password=570fe4a9f7bd4e2d # openstack-config --set /etc/cinder/cinder.conf DEFAULT rpc_backend cinder.openstack.common.rpc.impl_qpid # openstack-config --set /etc/cinder/cinder.conf DEFAULT qpid_hostname 192.168.3.98 vi /etc/cinder/api-paste.ini 参考拷贝控制节点内容 sql_connection=mysql://cinder:0ba1a335f2834e1a@192.168.3.98/cinder qpid_reconnect_timeout=0 qpid_reconnect_limit=0 qpid_reconnect=True qpid_reconnect_interval_max=0 qpid_reconnect_interval_min=0 sql_idle_timeout=3600 qpid_reconnect_interval=0 notification_driver=cinder.openstack.common.notifier.rpc_notifier vi /etc/tgt/targets.conf 增加如下内容 include /etc/cinder/volumes/* # service openstack-cinder-volume start # service tgtd start # chkconfig openstack-cinder-volume on # chkconfig tgtd on [iyunv@controller ~(keystone_admin)]# cinder-manage service list Binary Host Zone Status State Updated At cinder-volume controller nova enabled 2014-02-11 06:31:08 cinder-scheduler controller nova enabled 2014-02-11 06:31:06 cinder-volume compute nova enabled 2014-02-11 06:30:40 [iyunv@controller ~(keystone_admin)]#