OpenStack-FlatDHCP(multi_host)实施方案

sky

服务器规划：
控制节点：eth0 : 10.10.128.11  ,  eth2: 10.10.64.11
计算节点：eth2 : 10.10.128.12  ,  eth4:10.10.64.12
计算节点：eth2 : 10.10.128.13  ,  eth4:10.10.64.13

网段规划:
公网：163.167.28.70
内网：10.10.128.0/24
私网：10.10.64.0/24
虚拟机网段：10.11.11.0/24

操作系统：
Linux：ubuntu-12.04.1-server-amd64



root@ubuntu-11:~# nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-scheduler   ubuntu-11                            nova             enabled    :-)   2012-11-20 03:54:48
nova-consoleauth ubuntu-11                            nova             enabled    :-)   2012-11-20 03:54:47
nova-compute     ubuntu-13                            nova             enabled    :-)   2012-11-20 03:54:50
nova-network     ubuntu-13                            nova             enabled    :-)   2012-11-20 03:54:43
nova-cert        ubuntu-11                            nova             enabled    :-)   2012-11-20 03:54:46
nova-compute     ubuntu-12                            nova             enabled    :-)   2012-11-20 03:54:43
nova-network     ubuntu-12                            nova             enabled    :-)   2012-11-20 03:54:50

一、安装OpenStack的控制节点
1、开启root权限


sudo passwd root
设置新密码

2、网卡设置


# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet static
    address 10.10.128.11
    netmask 255.255.128.0
    network 10.10.128.0
    broadcast 10.10.255.255
    gateway 10.10.128.254
    # dns-* options are implemented by the resolvconf package, if installed
    dns-nameservers 10.10.128.254
auto eth2
iface eth2 inet static
    address 10.10.64.11
    netmask 255.255.255.0
    network 10.10.64.0
重启网络：

sudo /etc/init.d/networking restart
3、Install Base OS & bridge-utils


sudo apt-get update
sudo apt-get upgrade
sudo apt-get install bridge-utils
4、设置NTP Server
sudo apt-get install ntp
修改/etc/ntp.conf


server ntp.ubuntu.com
server 127.127.1.0
fudge 127.127.1.0 stratum 10
5、Install mysql-server and python-mysqldb package


sudo apt-get install mysql-server python-mysqldb
密码为: mygreatsecret
修改/etc/mysql/my.cnf

bind-address = 0.0.0.0
重启mysql

sudo restart mysql
Create MySQL databases to be used with nova, glance and keystone.
以下脚本粘贴到doMysql.sh,修改权限chmod +x doMsql.sh后执行，按提示输入主机名字


#!/bin/bash
read -p "Enter HostName(For example: computer-node1):" HOSTNAME
echo "hostname is :$HOSTNAME"
sudo mysql -uroot -pmygreatsecret -e 'CREATE DATABASE nova;'
sudo mysql -uroot -pmygreatsecret -e 'CREATE USER novadbadmin;'
sudo mysql -uroot -pmygreatsecret -e "GRANT ALL PRIVILEGES ON nova.* TO 'novadbadmin'@'%';"
sudo mysql -uroot -pmygreatsecret -e "SET PASSWORD FOR 'novadbadmin'@'%' = PASSWORD('novasecret');"
sudo mysql -uroot -pmygreatsecret -e 'CREATE DATABASE glance;'
sudo mysql -uroot -pmygreatsecret -e 'CREATE USER glancedbadmin;'
sudo mysql -uroot -pmygreatsecret -e "GRANT ALL PRIVILEGES ON glance.* TO 'glancedbadmin'@'%';"
sudo mysql -uroot -pmygreatsecret -e "SET PASSWORD FOR 'glancedbadmin'@'%' = PASSWORD('glancesecret');"
sudo mysql -uroot -pmygreatsecret -e 'CREATE DATABASE keystone;'
sudo mysql -uroot -pmygreatsecret -e 'CREATE USER keystonedbadmin;'
sudo mysql -uroot -pmygreatsecret -e "GRANT ALL PRIVILEGES ON keystone.* TO 'keystonedbadmin'@'%';"
sudo mysql -uroot -pmygreatsecret -e "SET PASSWORD FOR 'keystonedbadmin'@'%' = PASSWORD('keystonesecret');"
sudo mysql -uroot -pmygreatsecret -e "GRANT ALL PRIVILEGES ON nova.* TO 'novadbadmin'@'localhost';"
sudo mysql -uroot -pmygreatsecret -e "SET PASSWORD FOR 'novadbadmin'@'localhost' = PASSWORD('novasecret');"
sudo mysql -uroot -pmygreatsecret -e "GRANT ALL PRIVILEGES ON glance.* TO 'glancedbadmin'@'localhost';"
sudo mysql -uroot -pmygreatsecret -e "SET PASSWORD FOR 'glancedbadmin'@'localhost' = PASSWORD('glancesecret');"
sudo mysql -uroot -pmygreatsecret -e "GRANT ALL PRIVILEGES ON keystone.* TO 'keystonedbadmin'@'localhost';"
sudo mysql -uroot -pmygreatsecret -e "SET PASSWORD FOR 'keystonedbadmin'@'localhost' = PASSWORD('keystonesecret');"
sudo mysql -uroot -pmygreatsecret -e "GRANT ALL PRIVILEGES ON nova.* TO 'novadbadmin'@'$HOSTNAME';"
sudo mysql -uroot -pmygreatsecret -e "SET PASSWORD FOR 'novadbadmin'@'$HOSTNAME' = PASSWORD('novasecret');"
sudo mysql -uroot -pmygreatsecret -e "GRANT ALL PRIVILEGES ON glance.* TO 'glancedbadmin'@'$HOSTNAME';"
sudo mysql -uroot -pmygreatsecret -e "SET PASSWORD FOR 'glancedbadmin'@'$HOSTNAME' = PASSWORD('glancesecret');"
sudo mysql -uroot -pmygreatsecret -e "GRANT ALL PRIVILEGES ON keystone.* TO 'keystonedbadmin'@'$HOSTNAME';"
sudo mysql -uroot -pmygreatsecret -e "SET PASSWORD FOR 'keystonedbadmin'@'$HOSTNAME' = PASSWORD('keystonesecret');"
6、Install Keystone

sudo apt-get install keystone python-keystone python-keystoneclient
修改文件/etc/keystone/keystone.conf

admin_token = admin



connection = sqlite:////var/lib/keystone/keystone.db
改为
connection = mysql://keystonedbadmin:keystonesecret@10.10.64.11/keystone
重启keystone

sudo service keystone restart

同步数据库

sudo keystone-manage db_sync
添加环境变量到/root/.bashrc


export SERVICE_ENDPOINT="http://localhost:35357/v2.0"
export SERVICE_TOKEN=admin
使环境变量生效

source /root/.bashrc
Creating Tenants,Creating Users,Creating Roles,Listing Tenants, Users and Roles,Adding Roles to Users in Tenants,Creating Services,Creating Endpoints
把以下内容保存到create_keystone_data.sh，然后执行./ create_keystone_data.sh


#!/bin/bash
#easy to run Keystone
#Creating Tenants
#Creating Users
#Creating Roles
#Listing Tenants, Users and Roles
#Adding Roles to Users in Tenants
#Creating Services
#Creating Endpoints
#
function get_id () {
    echo `$@ | awk '/ id / { print $4 }'`
}
# Tenants
echo "-----------------------------Tenants-----------------------------------------"
ADMIN_TENANT=$(get_id keystone tenant-create --name=admin)
SERVICE_TENANT=$(get_id keystone tenant-create --name=service)
echo "ADMIN_TENANT=$ADMIN_TENANT"
echo "SERVICE_TENANT=$SERVICE_TENANT"
echo "-----------------------------Create Tenants Ending-----------------------------"
read -p "Enter your host(For example: xxx@.qq.com):" MAIL
echo "send mail to:$MAIL"
# Users
echo "-----------------------------Users-----------------------------------------"
ADMIN_USER=$(get_id keystone user-create --name=admin --pass=admin --email=$MAIL)
NOVA_USER=$(get_id keystone user-create --name=nova --pass=nova   --email=$MAIL)
GLANCE_USER=$(get_id keystone user-create --name=glance --pass=glance   --email=$MAIL)
SWIFT_USER=$(get_id keystone user-create --name=swift --pass=swift   --email=$MAIL)
echo "ADMIN_USER=$ADMIN_USER"
echo "NOVA_USER=$NOVA_USER"
echo "GLANCE_USER=$GLANCE_USER"
echo "SWIFT_USER=$SWIFT_USER"
echo "-----------------------------Create User Ending-----------------------------"
# Roles
echo "-----------------------------Roles-----------------------------------------"
ADMIN_ROLE=$(get_id keystone role-create --name=admin)
MEMBER_ROLE=$(get_id keystone role-create --name=Member)
echo "ADMIN_ROLE=$ADMIN_ROLE"
echo "MEMBER_ROLE=$MEMBER_ROLE"
echo "-----------------------------Create Roles Ending-----------------------------"
# Add Roles to Users in Tenants
echo "-----------------------------Add Roles to Users in Tenants-----------------------------------------"
echo "keystone user-role-add --user=$ADMIN_USER --role=$ADMIN_ROLE --tenant_id=$ADMIN_TENANT"
keystone user-role-add --user=$ADMIN_USER --role=$ADMIN_ROLE --tenant_id=$ADMIN_TENANT
echo "-----------------------------Add Roles to Users in Tenants Ending-----------------------------"
# TODO(termie): these two might be dubious
echo "-----------------------------TODO(termie): these two might be dubious-----------------------------------------"
echo "keystone user-role-add --user=$NOVA_USER --role=$ADMIN_ROLE --tenant_id=$SERVICE_TENANT"
echo "keystone user-role-add --user=$GLANCE_USER --role=$ADMIN_ROLE --tenant_id=$SERVICE_TENANT"
echo "keystone user-role-add --user=$SWIFT_USER --role=$ADMIN_ROLE --tenant_id=$SERVICE_TENANT"
keystone user-role-add --user=$NOVA_USER --role=$ADMIN_ROLE --tenant_id=$SERVICE_TENANT
keystone user-role-add --user=$GLANCE_USER --role=$ADMIN_ROLE --tenant_id=$SERVICE_TENANT
keystone user-role-add --user=$SWIFT_USER --role=$ADMIN_ROLE --tenant_id=$SERVICE_TENANT
echo "-----------------------------these two might be dubious Ending-----------------------------"
# The Member role is used by Horizon and Swift so we need to keep it:
echo "------------------------The Member role is used by Horizon and Swift so we need to keep it---------------------------------"
keystone user-role-add --user=$ADMIN_USER --role=$MEMBER_ROLE --tenant_id=$ADMIN_TENANT
echo "keystone user-role-add --user=$ADMIN_USER --role=$MEMBER_ROLE --tenant_id=$ADMIN_TENANT"
echo "-----------------------------The Member role is used by Horizon and Swift so we need to keep it Ending-----------------------------"
read -p "Enter your host(For example: 10.10.128.11):" HOSTADDR
echo "Your host is:$HOSTADDR"
#Creating Services
echo "------------------------Creating Services---------------------------------"
COMPUTESERVICE=$(get_id keystone service-create --name=nova --type=compute --description='OpenStack_Compute_Service')
VOLUMESERVICE=$(get_id keystone service-create --name=volume --type=volume --description='OpenStack_Volume_Service')
IMAGESERVICE=$(get_id keystone service-create --name=glance --type=image --description='OpenStack_Image_Service')
STORAGESERVICE=$(get_id keystone service-create --name=swift --type=object-store --description='OpenStack_Storage_Service')
IDENTITYSERVICE=$(get_id keystone service-create --name=keystone --type=identity --description='OpenStack_Identity_Service')
EC2SERVICE=$(get_id keystone service-create --name=ec2 --type=ec2 --description='EC2_Service')
echo "COMPUTESERVICE=$COMPUTESERVICE"
echo "VOLUMESERVICE=$VOLUMESERVICE"
echo "IMAGESERVICE=$IMAGESERVICE"
echo "STORAGESERVICE=$STORAGESERVICE"
echo "IDENTITYSERVICE=$IDENTITYSERVICE"
echo "EC2SERVICE=$EC2SERVICE"
echo "-----------------------------Creating Services Ending-----------------------------"
echo "-----------------------------Creating Endpoints-----------------------------"
keystone endpoint-create --region myregion --service_id=$COMPUTESERVICE --publicurl "http://$HOSTADDR:8774/v2/%(tenant_id)s" --adminurl "http://$HOSTADDR:8774/v2/%(tenant_id)s" --internalurl "http://$HOSTADDR:8774/v2/%(tenant_id)s"
keystone endpoint-create --region myregion --service_id=$VOLUMESERVICE --publicurl "http://$HOSTADDR:8776/v1/%(tenant_id)s" --adminurl "http://$HOSTADDR:8776/v1/%(tenant_id)s" --internalurl "http://$HOSTADDR:8776/v1/%(tenant_id)s"
keystone endpoint-create --region myregion --service_id=$IMAGESERVICE --publicurl "http://$HOSTADDR:9292/v1" --adminurl "http://$HOSTADDR:9292/v1" --internalurl "http://$HOSTADDR:9292/v1"
keystone endpoint-create --region myregion --service_id=$STORAGESERVICE --publicurl "http://$HOSTADDR:8080/v1/AUTH_%(tenant_id)s" --adminurl "http://$HOSTADDR:8080/v1" --internalurl "http://$HOSTADDR:8080/v1/AUTH_%(tenant_id)s"
keystone endpoint-create --region myregion --service_id=$IDENTITYSERVICE --publicurl http://$HOSTADDR:5000/v2.0 --adminurl http://$HOSTADDR:35357/v2.0 --internalurl http://$HOSTADDR:5000/v2.0
keystone endpoint-create --region myregion --service_id=$EC2SERVICE --publicurl http://$HOSTADDR:8773/services/Cloud --adminurl http://$HOSTADDR:8773/services/Admin --internalurl http://$HOSTADDR:8773/services/Cloud
echo "-----------------------------Creating Endpoints Ending-----------------------------"
执行脚本


chmod +x create_keystone_data.sh
./ create_keystone_data.sh
7、Install glance

sudo apt-get install glance glance-api glance-client glance-common glance-registry python-glance
修改/etc/glance/glance-api-paste.ini 和 /etc/glance/glance-registry-paste.ini



admin_tenant_name = %SERVICE_TENANT_NAME%
admin_user = %SERVICE_USER%
admin_password = %SERVICE_PASSWORD%
改为
admin_tenant_name = service
admin_user = glance
admin_password = glance
修改/etc/glance/glance-registry.conf


sql_connection = mysql://glancedbadmin:glancesecret@10.10.64.11/glance
#末尾追加
[paste_deploy]
flavor = keystone
末尾追加以下内容到/etc/glance/glance-api.conf
rabbit_host = 10.10.64.11
Create glance schema in the MySQL database.:


sudo glance-manage version_control 0
sudo glance-manage db_sync
重启glance-api 和glance-registry


sudo restart glance-api
sudo restart glance-registry
添加环境变量到/root/.bashrc


export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin
export OS_AUTH_URL="http://localhost:5000/v2.0/"
使环境变量生效

source /root/.bashrc
打印镜像列表

glance index
8、Install nova 除了nova-compute和nova-network

sudo apt-get install nova-api nova-cert nova-doc nova-objectstore nova-scheduler rabbitmq-server novnc nova-consoleauth
修改/etc/nova/nova.conf


#common
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--s3_host=10.10.64.11
--ec2_host=10.10.64.11
--rabbit_host=10.10.64.11
--glance_api_servers=10.10.64.11:9292
--image_service=nova.image.glance.GlanceImageService
--sql_connection=mysql://novadbadmin:novasecret@10.10.64.11/nova
--ec2_path=http://10.10.128.11:8773/services/Cloud
--api_paste_config=/etc/nova/api-paste.ini
--start_guests_on_host_boot=true
--resume_guests_state_on_host_boot=true
--network_manager=nova.network.manager.FlatDHCPManager
--connection_type=libvirt
--root_helper=sudo nova-rootwrap
--verbose=false
--use-syslog=false
--enabled_apis=ec2,osapi_compute,osapi_volume,metadata
--metadata_host=10.10.64.11
#hypervisors
--libvirt_type=kvm
--libvirt_use_virtio_for_bridges=true
#--libvirt_vif_driver =nova.virt.libvirt.vif.LibvirtBridgeDriver
--use_usb_tablet=true
--libvirt_ovs_bridge=br-int
#keystone
--auth_strategy=keystone
--keystone_ec2_url=http://10.10.128.11:5000/v2.0/ec2tokens
#Scheduler
--scheduler_driver=nova.scheduler.multi.MultiScheduler
#network
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--multi_host=true
--public_interface=eth0
--flat_interface=eth2
--flat_network_bridge=br100
--fixed_range=10.11.11.0/24
--flat_injected=false
--force_dhcp_release=true
--network_size=256
--allow_same_net_traffic=true
#vnc
--vnc_enabled=true
--novncproxy_base_url= http://163.167.28.70:6080/vnc_auto.html
#--vncserver_proxyclient_address=
#--vncserver_listen=
--vnc_keymap=en-us
#log
#--logging_exception_prefix="%(asctime)s TRACE %(name)s %(instance)s"
#--publish_errors=true
#compute
#--compute_driver=nova.virt.connection.get_connection
#--instances_path= /mnt/nova/instances/
修改权限
sudo chown -R nova:nova /etc/nova
sudo chmod 644 /etc/nova/nova.conf
修改/etc/nova/api-paste.ini


admin_tenant_name = %SERVICE_TENANT_NAME%
admin_user = %SERVICE_USER%
admin_password = %SERVICE_PASSWORD%
改为
admin_tenant_name = service
admin_user = nova
admin_password = nova
数据同步

sudo nova-manage db sync
重启nova服务

sudo restart nova-api; sudo restart nova-objectstore; sudo restart nova-scheduler; sudo restart nova-consoleauth;
查看服务


root@ubuntu-50:~# sudo nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-scheduler   ubuntu-50                            nova             enabled    :-)   2012-10-28 04:46:52
nova-consoleauth ubuntu-50                            nova             enabled    :-)   2012-10-28 04:46:52
9、Install OpenStack Dashboard

sudo apt-get install openstack-dashboard
遇到
Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName
添加内容到/etc/apache2/httpd.conf

ServerName 127.0.1.1
重启apache

sudo service apache2 restart
打开浏览器，输入http://10.10.128.11,输入admin@admin登录。

二、安装计算节点
1、安装网桥

sudo apt-get install qemu-kvm libvirt-bin virt-manager bridge-utils
2、修改/etc/network/interfaces


# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth2
iface eth2 inet static
    address 10.10.128.12
    netmask 255.255.128.0
    network 10.10.128.0
    broadcast 10.10.255.255
    gateway 10.10.128.254
auto eth4
iface eth4 inet manual
auto br12
iface br12 inet static
address 10.10.64.12
netmask 255.255.255.0
#network 10.10.64.0
#broadcast 10.10.64.255
bridge_ports eth4
重启网络：


sudo /etc/init.d/networking restart
3、安装NTP Client

sudo apt-get install ntp

重启NTP

sudo service ntp restart
4、设置ipv4转发

sysctl -w net.ipv4.ip_forward=1
5、安装nova-network,nova-compute

sudo apt-get install nova-compute nova-compute-kvm nova-network
修改/etc/nova/nova.conf ，与控制节点一致


#common
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--s3_host=10.10.64.11
--ec2_host=10.10.64.11
--rabbit_host=10.10.64.11
--glance_api_servers=10.10.64.11:9292
--image_service=nova.image.glance.GlanceImageService
--sql_connection=mysql://novadbadmin:novasecret@10.10.64.11/nova
--ec2_path=http://10.10.128.11:8773/services/Cloud
--api_paste_config=/etc/nova/api-paste.ini
--start_guests_on_host_boot=true
--resume_guests_state_on_host_boot=true
--network_manager=nova.network.manager.FlatDHCPManager
--connection_type=libvirt
--root_helper=sudo nova-rootwrap
--verbose=false
--use-syslog=false
--enabled_apis=ec2,osapi_compute,osapi_volume,metadata
--metadata_host=10.10.64.11
#hypervisors
--libvirt_type=kvm
--libvirt_use_virtio_for_bridges=true
#--libvirt_vif_driver =nova.virt.libvirt.vif.LibvirtBridgeDriver
--use_usb_tablet=true
--libvirt_ovs_bridge=br-int
#keystone
--auth_strategy=keystone
--keystone_ec2_url=http://10.10.128.11:5000/v2.0/ec2tokens
#Scheduler
--scheduler_driver=nova.scheduler.multi.MultiScheduler
#network
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--multi_host=true
--public_interface=eth2
--flat_interface=eth4
--flat_network_bridge=br100
--fixed_range=10.11.11.0/24
--flat_injected=false
--force_dhcp_release=true
--network_size=256
--allow_same_net_traffic=true
#vnc
--vnc_enabled=true
--novncproxy_base_url= http://163.167.28.70:6080/vnc_auto.html
--vncserver_proxyclient_address=10.10.64.12
--vncserver_listen=10.10.64.12
--vnc_keymap=en-us
#log
#--logging_exception_prefix="%(asctime)s TRACE %(name)s %(instance)s"
#--publish_errors=true
#compute
#--compute_driver=nova.virt.connection.get_connection
--instances_path=/mnt/nova/instances/
修改权限


sudo chown -R nova:nova /mnt/nova/
sudo chown -R nova:nova /mnt/nova/instances/
重启服务

sudo restart nova-compute; sudo restart nova-network; sudo restart libvirt-bin;

三、初始化系统
1、上传镜像


glance add name="qcow2_win08_35G" is_public=true container_format=ovf disk_format=qcow2 < qcow2_win08_35G.img
glance add name="qcow2_ubuntu64" is_public=true container_format=ovf disk_format=qcow2 < qcow2_ubuntu64.img
## nova image-list
2、创建网络


nova-manage network create --label=br100 --fixed_range_v4=10.11.11.0/24 --num_network=1  --multi_host=T --bridge=br100
#ip addr
#brctl show
3、安全组(防火墙规则)


nova secgroup-add-rule default tcp 1 65535 0.0.0.0/0
nova secgroup-add-rule default udp 1 65535 0.0.0.0/0
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
Create the win-server security group to allow ping (icmp), ssh (tcp/22), and http (tcp/80) from everywhere (0.0.0.0/0):


nova secgroup-create win-server "win server running on default port"
nova secgroup-add-rule win-server icmp -1 -1 0.0.0.0/0
nova secgroup-add-rule win-server tcp 22 22 0.0.0.0/0
nova secgroup-add-rule win-server tcp 3389 3389 0.0.0.0/0
nova secgroup-add-rule win-server tcp 80 80 0.0.0.0/0
nova secgroup-add-rule win-server tcp 20 20 0.0.0.0/0
nova secgroup-add-rule win-server tcp 21 21 0.0.0.0/0
4、创建浮动IP

nova-manage floating create --pool=pool1 --ip_range=10.10.128.192/26

5、使用镜像创建实例(虚拟机)

6、绑定浮动IP

8、VNC控制台