基于openstack上的MICRO BOSH 安装cf

(安西)

bosh部署3要素
1. manifest
2. release包
3. stemcell
如何获得？
创建manifest
用spiff生成manifest
spiff下载地址：
https://github.com/cloudfoundry-incubator/spiff/releases
在cf-release根目录
修改spec/fixtures/openstack/cf-stub.yml中的配置
运行./generate_deployment_manifest openstack cf-stub.yml > cf.yml
资源
cf-release下载
https://github.com/cloudfoundry-community/community-shared-boshreleases
stemcell下载
http://bosh-artifacts.cfapps.io/file_collections?type=stemcells
cf-release版本和stemcell，bosh版本对应关系查询
https://github.com/cloudfoundry-community/cf-docs-contrib/wiki/v187
部署
1. 上传release
$ bosh upload release cf-release
验证：
$ bosh releases

+---------------------+------------+-------------+ | Name | Versions | Commit Hash | +---------------------+------------+-------------+ | cf | 169 | ae04d8ba+ | +---------------------+------------+-------------+

 2. 上传stemcell
bosh upload stemcell 
验证：
$ bosh stemcells
+------------------------------------+---------+--------------------------------------+
| Name | Version | CID |
+------------------------------------+---------+--------------------------------------+
| bosh-openstack-kvm-centos-go_agent | 2748 | 0a4da153-4878-4cd4-8451-6ddefe7f6e34 |
3.部署
$ bosh deployment cf.yml
$ bosh deploy
部署完成后
1. 所有DEA节点的cc api域名需解析到内部网络
例如：/etc/hosts 加入：10.60.31.112 api.ae2.demo
不然，dea上传droplet会报错。
问题排查
上传stemcell不能调用CPI
Director task 2
Error 100: Unable to connect to the OpenStack Compute API. Check task debug log for details.
原因：microbosh不能访问openstack cc
解决：
在microbosh的机器里，/etc/hosts 加上：
172.16.40.14    cc14.ae2
超过cpu核配置限制
E, [2014-10-10T03:34:37.945276 #6683] [create_vm(decb786a-0cdc-4eca-8827-17e5bb1d1fcb, ...)] ERROR -- : OpenStack API Request Entity Too Large error. Check task debug log f
or details.
E, [2014-10-10T03:34:37.945342 #6683] [create_vm(decb786a-0cdc-4eca-8827-17e5bb1d1fcb, ...)] ERROR -- : Expected([200, 202]) <=> Actual(413 Request Entity Too Large)
response => #<Excon::Response:0x007f7e942384e8 @data={:body=>"{\"overLimit\": {\"message\": \"Quota exceeded for cores: Requested 2, but already used 49 of 50 cores\", \"
code\": 413, \"retryAfter\": \"0\"}}", :headers=>{"Retry-After"=>"0", "Content-Length"=>"132", "Content-Type"=>"application/json; charset=UTF-8", "X-Compute-Request-Id"=>"r
eq-fd2272b6-d749-4b9f-bd8d-e2b6792ed988", "Date"=>"Fri, 10 Oct 2014 03:34:39 GMT"}, :status=>413, :remote_ip=>"172.16.40.14"}, @body="{\"overLimit\": {\"message\": \"Quota 
exceeded for cores: Requested 2, but already used 49 of 50 cores\", \"code\": 413, \"retryAfter\": \"0\"}}", @headers={"Retry-After"=>"0", "Content-Length"=>"132", "Content
-Type"=>"application/json; charset=UTF-8", "X-Compute-Request-Id"=>"req-fd2272b6-d749-4b9f-bd8d-e2b6792ed988", "Date"=>"Fri, 10 Oct 2014 03:34:39 GMT"}, @status=413, @remot
e_ip="172.16.40.14"> (Excon::Errors::RequestEntityTooLarge)
修改/etc/nova.conf
quota_cores=-1 # 无限制
get_state超时
Director task 29
Started preparing deployment
Started preparing deployment > Binding deployment. Done (00:00:00)
Started preparing deployment > Binding releases. Done (00:00:00)
Started preparing deployment > Binding existing deployment. Failed: Timed out sending `get_state' to 65941315-33fb-446c-af36-cef368e7e62d after 45 seconds (00:02:15)
Error 450002: Timed out sending `get_state' to 65941315-33fb-446c-af36-cef368e7e62d after 45 seconds
用bosh cck命令删除vm reference，每次都选3
3. Delete VM reference (DANGEROUS!)
创建虚机失败
| fault | {"message": "No valid host was found. ", "code": 500, "details": " File \"/usr/lib/python2.6/site-packages/nova/scheduler/filter_scheduler.py\", line 107, in schedule_run_instance |
| | raise exception.NoValidHost(reason=\"\") |
| | ", "created": "2014-10-11T06:16:43Z"}
nova日志/var/log/nova/scheduler.log：
raise exception.PortLimitExceeded()
网卡超过限制
neutron port-list
修改/etc/neutron/neutron.conf
quota_port = -1
编译失败
错误：
Compile failed
解决办法：
a. 编译
用stemcell创建一个虚机，上传release包到该虚机，解压，运行编译失败模块包里的packging脚本，查看具体错误，解决编译错误，一般是第三方依赖不满足。
b.重新打包
重新打包依赖包和模块包，计算该包的sha1sum，并用base64编码，替换release.MF中该模块的sha1sum，重新打包整个release包：cf-xxx.tgz
c. 重新上传release包
bosh upload release cf-xxx.tgz
 
上传stemcell时报SchemaValidationError错
Director task 1
Error 100: Invalid OpenStack cloud properties: #<Membrane::SchemaValidationError: { openstack => { api_key => Expected instance of String, given an instance of Fixnum } }>
解决办法：
进入micro bosh虚拟机
a. 修改director配置
vi /var/vcap/jobs/director/config/director.yml.erb
api_key: 123456  =>  api_key: '123456'
注意是修改director.yml.erb文件，
b. 重启director服务
bosh/bin/monit restart director
你会发现/var/vcap/jobs/director/config/director.yml已经被修改。
c. 重新上传stemcell。