OpenStack使用Bosh部署CloudFoundry(四)—使用Micro Bosh部署Bosh

wdcsx

目录(?)[-]

• 上传Bosh Stemcell到Micro Bosh中
  
• 下载Bosh Release代码并打包
  
• 上传Bosh Release包到Micro Bosh中
  
• 确认已经上传的stemcell和release
  
• 创建部署Bosh的Manifest文件
  
• 部署Bosh
  
• 验证部署是否成功
  

版本说明：
CloudFoundry：V2版本-v149+（本文后续章节中的Manifest适用于149+版本的CF Release）

OpenStack：Grizzly版本+nova-network
CloudFoundry部署交流QQ群：176302388

本文介绍如何使用Micro Bosh来部署Bosh，部署需要创建8个虚拟机来部署8个Bosh组件，所以，至少得准备好8个Fixed IP，后边使用Bosh的时候需要访问Director组件及配置DNS，所以准备至少2个Floating IP，本文中为（10.68.19.131和10.68.19.132），Bosh组件的详细介绍可以参看下边连接。
BOSH组件介绍：http://docs.cloudfoundry.com/docs/running/bosh/components/


下面是部署步骤：
1、上传Bosh Stemcell到Micro Bosh中
[plain] viewplaincopy

• root@bosh-cli:~/bosh-workspace/stemcells# bosh upload stemcell bosh-stemcell-latest-openstack-kvm-ubuntu.tgz  
  
•   
  
• Verifying stemcell...  
  
• File exists and readable                                     OK  
  
• Using cached manifest...  
  
• Stemcell properties                                          OK  
  
•   
  
• Stemcell info  
  
• -------------  
  
• Name:    bosh-openstack-kvm-ubuntu  
  
• Version: 1478  
  
•   
  
• Checking if stemcell already exists...  
  
• No  
  
•   
  
• Uploading stemcell...  
  
•   
  
• latest-bosh-s: 100% |ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo| 251.3MB  11.3MB/s Time: 00:00:22  
  
•   
  
• Director task 1  
  
•   
  
• Update stemcell  
  
•   extracting stemcell archive (00:00:06)  
  
•   verifying stemcell manifest (00:00:00)  
  
•   checking if this stemcell already exists (00:00:00)  
  
•   uploading stemcell bosh-stemcell/776 to the cloud (00:01:53)  
  
•   save stemcell bosh-stemcell/776 (f207bfcd-f0cb-4ea2-b753-022e6234ebfa) (00:00:00)  
  
• Done                    5/5 00:01:59  
  
•   
  
• Task 1 done  
  
• Started         2013-12-16 09:51:59 UTC  
  
• Finished        2013-12-16 09:53:58 UTC  
  
• Duration        00:01:59  
  
•   
  
• Stemcell uploaded and created  
  

2、下载Bosh Release代码并打包
[plain] viewplaincopy

• root@bosh-cli:~# cd ~/bosh-workspace  
  
• root@bosh-cli:~# git clone git://github.com/cloudfoundry/bosh.git  
  

耗费时间较长，请耐心等待。
[plain] viewplaincopy

• root@bosh-cli:~# cd ~/bosh-workspace/bosh  
  
• root@bosh-cli:~# apt-get install libxslt-dev libxml2-dev libmysql-ruby libmysqlclient-dev libpq-dev  
  
• root@bosh-cli:~# bundle install --local  
  
• root@bosh-cli:~# bundle exec rake release:create_dev_release  
  

可以先修改~/bosh-workspace/bosh/Gemfile的source为淘宝源：

[plain] viewplaincopy

• source 'http://ruby.taobao.org'  
  

打包结束得到以下输出：

[plain] viewplaincopy

• bosh create release --force  
  
• Syncing blobs...  
  
•   
  
• Building DEV release  
  
• ---------------------------------  
  
•   
  
• ....  
  
•   
  
• Release version: 13.1-dev  
  
• Release manifest: /root/bosh-workspace/bosh/release/dev_releases/bosh-13.1-dev.yml  
  

3、上传Bosh Release包到Micro Bosh中
[plain] viewplaincopy

• root@bosh-cli:~/bosh-workspace# bosh upload release ~/bosh-workspace/bosh/release/dev_releases/bosh-13.1-dev.yml  
  

成功后得到以下输出：

[plain] viewplaincopy

• ....  
  
•   
  
• Creating new jobs  
  
•   blobstore/3.1-dev (00:00:00)  
  
•   powerdns/3.1-dev (00:00:00)  
  
•   redis/3 (00:00:00)  
  
•   nats/5 (00:00:00)  
  
•   postgres/4.1-dev (00:00:00)  
  
•   director/11.2-dev (00:00:00)  
  
•   health_monitor/5.1-dev (00:00:00)  
  
•   registry/0.1-dev (00:00:00)  
  
• Done                    8/8 00:00:00  
  
•   
  
• Release has been created  
  
•   bosh/13.1-dev (00:00:00)  
  
• Done                    1/1 00:00:00  
  
•   
  
• Task 2 done  
  
• Started         2013-12-16 10:06:26 UTC  
  
• Finished        2013-12-16 10:06:46 UTC  
  
• Duration        00:00:20  
  
•   
  
• Release uploaded  
  

4、确认已经上传的stemcell和release
[plain] viewplaincopy

• root@bosh-cli:~# bosh stemcells  
  
•   
  
• +---------------------------+---------+--------------------------------------+  
  
• | Name                      | Version | CID                                  |  
  
• +---------------------------+---------+--------------------------------------+  
  
• | bosh-openstack-kvm-ubuntu | 1478    | e64c6da5-f2ab-48f2-b03d-79c29d1acc43 |  
  
• +---------------------------+---------+--------------------------------------+  
  
•   
  
• Stemcells total: 1  
  
• root@bosh-cli:~# bosh releases  
  
•   
  
• +------+-----------+-------------+  
  
• | Name | Versions  | Commit Hash |  
  
• +------+-----------+-------------+  
  
• | bosh | 13.1-dev* | a8e43f41+   |  
  
• +------+-----------+-------------+  
  
• (*) Currently deployed  
  
• (+) Uncommitted changes  
  
•   
  
• Releases total: 1  
  

5、创建部署Bosh的Manifest文件
创建目录

[plain] viewplaincopy

• root@bosh-cli:~# mkdir -p ~/bosh-workspace/deployments/bosh-openstack  
  
• root@bosh-cli:~# cd ~/bosh-workspace/deployments/bosh-openstack  
  

复制模板文件

[plain] viewplaincopy

• root@bosh-cli:~# cp ~/bosh-workspace/bosh/release/examples/bosh-openstack-dynamic.yml bosh-openstack.yml  
  

如果OpenStack中采用的是quantum网络组件，则使用下边的模板文件

[plain] viewplaincopy

• root@bosh-cli:~# cp ~/bosh-workspace/bosh/release/examples/bosh-openstack-manual.yml bosh-openstack.yml  
  

本次安装中使用nova-network网络组件，所以采用bosh-openstack-dynamic.yml，修改其中标记有# CHANGE 的配置项，下边是本环境中的配置，大家可参考：

[plain] viewplaincopy

• ---  
  
• name: bosh-openstack  
  
• director_uuid: 5d8c682c-290a-4194-bfc6-60248567ceab # CHANGE  
  
•   
  
• release:  
  
•   name: bosh  
  
•   version: latest  
  
•   
  
• compilation:  
  
•   workers: 3  
  
•   network: default  
  
•   reuse_compilation_vms: true  
  
•   cloud_properties:  
  
•     instance_type: v1.compile # CHANGE  
  
•   
  
• update:  
  
•   canaries: 1  
  
•   canary_watch_time: 3000-120000  
  
•   update_watch_time: 3000-120000  
  
•   max_in_flight: 4  
  
•   max_errors: 1  
  
•   
  
• networks:  
  
•   - name: floating  
  
•     type: vip  
  
•     cloud_properties: {}  
  
•   - name: default  
  
•     type: dynamic  
  
•     cloud_properties: {}  
  
•   
  
• resource_pools:  
  
•   - name: common  
  
•     network: default  
  
•     size: 8  
  
•     stemcell:  
  
•       name: bosh-openstack-kvm-ubuntu  
  
•       version: latest  
  
•     cloud_properties:  
  
•       instance_type: v1.micro # CHANGE  
  
•   
  
• jobs:  
  
•   - name: nats  
  
•     template: nats  
  
•     instances: 1  
  
•     resource_pool: common  
  
•     networks:  
  
•       - name: default  
  
•         default: [dns, gateway]  
  
•   
  
•   - name: redis  
  
•     template: redis  
  
•     instances: 1  
  
•     resource_pool: common  
  
•     networks:  
  
•       - name: default  
  
•         default: [dns, gateway]  
  
•   
  
•   - name: postgres  
  
•     template: postgres  
  
•     instances: 1  
  
•     resource_pool: common  
  
•     persistent_disk: 16384  
  
•     networks:  
  
•       - name: default  
  
•         default: [dns, gateway]  
  
•   
  
•   - name: powerdns  
  
•     template: powerdns  
  
•     instances: 1  
  
•     resource_pool: common  
  
•     networks:  
  
•       - name: default  
  
•         default: [dns, gateway]  
  
•       - name: floating  
  
•         static_ips:  
  
•           - 10.68.19.131 # CHANGE  
  
•   
  
•   - name: blobstore  
  
•     template: blobstore  
  
•     instances: 1  
  
•     resource_pool: common  
  
•     networks:  
  
•       - name: default  
  
•         default: [dns, gateway]  
  
•   
  
•   - name: director  
  
•     template: director  
  
•     instances: 1  
  
•     resource_pool: common  
  
•     persistent_disk: 16384  
  
•     networks:  
  
•       - name: default  
  
•         default: [dns, gateway]  
  
•       - name: floating  
  
•         static_ips:  
  
•           - 10.68.19.132 # CHANGE  
  
•   
  
•   - name: registry  
  
•     template: registry  
  
•     instances: 1  
  
•     resource_pool: common  
  
•     networks:  
  
•       - name: default  
  
•         default: [dns, gateway]  
  
•   
  
•   - name: health_monitor  
  
•     template: health_monitor  
  
•     instances: 1  
  
•     resource_pool: common  
  
•     networks:  
  
•       - name: default  
  
•         default: [dns, gateway]  
  
•   
  
• properties:  
  
•   nats:  
  
•     address: 0.nats.default.bosh-openstack.microbosh  
  
•     user: nats  
  
•     password: nats  
  
•   
  
•   redis:  
  
•     address: 0.redis.default.bosh-openstack.microbosh  
  
•     password: redis  
  
•   
  
•   postgres: &bosh_db  
  
•     host: 0.postgres.default.bosh-openstack.microbosh  
  
•     user: postgres  
  
•     password: postgres  
  
•     database: bosh  
  
•   
  
•   dns:  
  
•     address: 10.68.19.131 # CHANGE  
  
•     db: *bosh_db  
  
•     recursor: 10.68.19.130 # CHANGE  
  
•   
  
•   blobstore:  
  
•     address: 0.blobstore.default.bosh-openstack.microbosh  
  
•     agent:  
  
•       user: agent  
  
•       password: agent  
  
•     director:  
  
•       user: director  
  
•       password: director  
  
•   
  
•   director:  
  
•     name: bosh  
  
•     address: 0.director.default.bosh-openstack.microbosh  
  
•     db: *bosh_db  
  
•   
  
•   registry:  
  
•     address: 0.registry.default.bosh-openstack.microbosh  
  
•     db: *bosh_db  
  
•     http:  
  
•       user: registry  
  
•       password: registry  
  
•   
  
•   hm:  
  
•     http:  
  
•       user: hm  
  
•       password: hm  
  
•     director_account:  
  
•       user: admin  
  
•       password: admin  
  
•     resurrector_enabled: true  
  
•   
  
•   ntp:  
  
•     - 0.north-america.pool.ntp.org  
  
•     - 1.north-america.pool.ntp.org  
  
•   
  
•   openstack:  
  
•     auth_url: http://10.68.19.61:5000/v2.0 # CHANGE   
  
•     username: VF # CHANGE   
  
•     api_key: 1 # CHANGE   
  
•     tenant: VF_CF # CHANGE   
  
•     region: myregion # CHANGE   
  
•     default_security_groups: ["default"] # CHANGE   
  
•     default_key_name: vkey # CHANGE  
  

注意：director_uuid是Micro Bosh的Director的UUID，可以使用bosh status查看并复制到Manifest文件中。
6、部署Bosh
设置Bosh的部署文件

[plain] viewplaincopy

• root@bosh-cli:~/bosh-workspace/deployments# bosh deployment ~/bosh-workspace/deployments/bosh-openstack/bosh-openstack.yml  
  
• Deployment set to `/root/bosh-workspace/deployments/bosh-openstack/bosh-openstack.yml'  
  

执行部署命令

[plain] viewplaincopy

• root@bosh-cli:~/bosh-workspace/deployments# bosh deploy  
  

部署时可能会遇到错误RateLimit报错，报错内容如下：

[plain] viewplaincopy

• E, [2013-12-15T06:13:32.599434 #6477] [task:3] ERROR -- : OpenStack API Request Entity Too Large error. Check task debug log for details.  
  
• /var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.1478/lib/cloud/openstack/helpers.rb:20:in `cloud_error'  
  
• /var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.1478/lib/cloud/openstack/helpers.rb:39:in `rescue in with_openstack'  
  
• /var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.1478/lib/cloud/openstack/helpers.rb:25:in `with_openstack'  
  
• /var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.1478/lib/cloud/openstack/cloud.rb:512:in `block in set_vm_metadata'  
  
• /var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.1478/lib/common/thread_formatter.rb:46:in `with_thread_name'  
  
• /var/vcap/packages/director/gem_home/gems/bosh_openstack_cpi-1.5.0.pre.1478/lib/cloud/openstack/cloud.rb:511:in `set_vm_metadata'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/metadata_helper.rb:14:in `update_vm_metadata'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:225:in `block (2 levels) in compile_package'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:290:in `prepare_vm'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:224:in `block in compile_package'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/lock_helper.rb:89:in `block in with_compile_lock'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/lock.rb:58:in `lock'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/lock_helper.rb:88:in `with_compile_lock'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:217:in `compile_package'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:205:in `block (2 levels) in process_task'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/event_log.rb:58:in `track'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:203:in `block in process_task'  
  
• /var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.1478/lib/common/thread_formatter.rb:46:in `with_thread_name'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:199:in `process_task'  
  
• /var/vcap/packages/director/gem_home/gems/director-1.5.0.pre.1478/lib/director/package_compiler.rb:174:in `block (4 levels) in compile_packages'  
  
• /var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.1478/lib/common/thread_pool.rb:83:in `call'  
  
• /var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.1478/lib/common/thread_pool.rb:83:in `block (2 levels) in create_thread'  
  
• /var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.1478/lib/common/thread_pool.rb:67:in `loop'  
  
• /var/vcap/packages/director/gem_home/gems/bosh_common-1.5.0.pre.1478/lib/common/thread_pool.rb:67:in `block in create_thread'  
  

解决办法：请查看本人博客：http://blog.iyunv.com/tiger435/article/details/8808623#t5，部署成功反馈：

[plain] viewplaincopy

• root@bosh-cli:~/bosh-workspace/deployments# bosh deploy  
  
• Getting deployment properties from director...  
  
• Unable to get properties list from director, trying without it...  
  
• Compiling deployment manifest...  
  
• Cannot get current deployment information from director, possibly a new deployment  
  
• Please review all changes carefully  
  
• Deploying `bosh-openstack.yml' to `microbosh-openstack' (type 'yes' to continue): yes  
  
•   
  
• Director task 89  
  
•   
  
• Preparing deployment  
  
•   binding deployment (00:00:00)  
  
•   binding releases (00:00:00)  
  
•    
  
• ....  
  
•   
  
• Updating job nats  
  
•   nats/0 (canary) (00:00:29)  
  
• Done                    1/1 00:00:29  
  
•   
  
• Updating job redis  
  
•   redis/0 (canary) (00:00:24)  
  
• Done                    1/1 00:00:24  
  
•   
  
• Updating job postgres  
  
•   postgres/0 (canary) (00:00:44)  
  
• Done                    1/1 00:00:44  
  
•   
  
• Updating job powerdns  
  
•   powerdns/0 (canary) (00:00:25)  
  
• Done                    1/1 00:00:25  
  
•   
  
• Updating job blobstore  
  
•   blobstore/0 (canary) (00:00:25)  
  
• Done                    1/1 00:00:25  
  
•   
  
• Updating job director  
  
•   director/0 (canary) (00:00:58)  
  
• Done                    1/1 00:00:58  
  
•   
  
• Updating job registry  
  
•   registry/0 (canary) (00:00:33)  
  
• Done                    1/1 00:00:33  
  
•   
  
• Updating job health_monitor  
  
•   health_monitor/0 (canary) (00:00:28)  
  
• Done                    1/1 00:00:28  
  
•   
  
• Task 3 done  
  
• Started         2013-12-17 02:41:15 UTC  
  
• Finished        2013-12-17 02:57:08 UTC  
  
• Duration        00:15:53  
  
•   
  
• Deployed `bosh-openstack.yml' to `microbosh-openstack'  
  

8、验证部署是否成功

[plain] viewplaincopy

• root@bosh-cli:~/bosh-workspace/deployments/bosh-openstack# bosh vms  
  
• Deployment `bosh-openstack'  
  
•   
  
• Director task 90  
  
•   
  
• Task 90 done  
  
•   
  
• +------------------+---------+---------------+--------------------------+  
  
• | Job/index        | State   | Resource Pool | IPs                      |  
  
• +------------------+---------+---------------+--------------------------+  
  
• | blobstore/0      | running | common        | 50.50.0.11               |  
  
• | director/0       | running | common        | 50.50.0.10, 10.68.19.132 |  
  
• | health_monitor/0 | running | common        | 50.50.0.13               |  
  
• | nats/0           | running | common        | 50.50.0.5                |  
  
• | postgres/0       | running | common        | 50.50.0.7                |  
  
• | powerdns/0       | running | common        | 50.50.0.9, 10.68.19.131  |  
  
• | redis/0          | running | common        | 50.50.0.6                |  
  
• | registry/0       | running | common        | 50.50.0.12               |  
  
• +------------------+---------+---------------+--------------------------+  
  
•   
  
• VMs total: 8  
  

如上，所有Job的状态都是running，则部署成功。