Puppet master/agent模型详解
本帖最后由 sky 于 2014-4-15 10:04 编辑一、Puppetmaster/agent模型puppet通过在master端启动puppetmaster服务来接受agent客户端的请，在/etc/puppet/manifest/site.pp中通过客户端的FQDN来定义每个agent所有应用的class,首次启动puppet守护进程时，其会自动进行运行环境的初始化，创建一个本地CA及服务器端相关的证书和密钥等。初始化操作完成后，puppet就会监听指定的套接字并等待客户端的连接请求。默认情况下，其证书和密钥等文件位于/var/lib/puppet/ssl/目录中。
MCollective简介MCollective是一个调度器，可以解决多个puppet agent同时向master提出请求造成性能，速度下降的问题，它可以根据不同的属性对节点进行分类，对不同的分类执行不同的任务；它是一个控制终端，可以使用它控制客户端和服务器，因此不需要puppet agent定时运行了。MCollective也是C/S架构，而且client和server使用Midware(中间件)进行通信
Puppet架构与集群Puppet通常部署为C/S架构，当agent过多时会面临性能问题常见的集群方案：puppet + nginxpuppet + passenger + apachePuppet集群的构建机制puppetmaster集群：Active/Active模式高可用集群，分摊puppetmaster上来自于agent的请求压力反向代理模式，将针对于8140端口请求分散到多台puppetmaster
下面是master/agent模型的原理图：
二、实验环境192.168.30.116OS：CentOS 6.4 x86_64 node1.luojianlong.com192.168.30.117OS：CentOS 6.4 x86_64 node2.luojianlong.com192.168.30.119OS：CentOS 6.4 x86_64 node3.luojianlong.com
需要的软件包：puppet-2.7.23-1.el6.noarch.rpmpuppet-server-2.7.23-1.el6.noarch.rpmfacter-1.7.3-1.el6.x86_64.rpmpuppet-dashboard-1.2.23-1.el6.noarch.rpmmysql-5.5.33-linux2.6-x86_64.tar.gz首先在node1安装master端
#设置各节点的hosts文件# cat /etc/hosts192.168.30.116 node1.luojianlong.com192.168.30.117 node2.luojianlong.com192.168.30.119 node3.luojianlong.com# 更新facter# rpm -Uvh facter-1.7.3-1.el6.x86_64.rpm# 配置epel源# cat /etc/yum.repos.d/epel.reponame=epelbaseurl=http://mirrors.sohu.com/fedora-epel/6/$basearch/gpgcheck=1gpgkey=http://mirrors.sohu.com/fedora-epel/RPM-GPG-KEY-EPEL-6# 安装puppet，puppet-server# yum -y localinstall puppet-2.7.23-1.el6.noarch.rpm# yum -y localinstall puppet-server-2.7.23-1.el6.noarch.rpm在node2，node3安装puppet-agent
# rpm -Uvh facter-1.7.3-1.el6.x86_64.rpm# yum -y localinstall puppet-2.7.23-1.el6.noarch.rpm# rpm -Uvh facter-1.7.3-1.el6.x86_64.rpm# yum -y localinstall puppet-2.7.23-1.el6.noarch.rpm
在node1创建并配置模块
# mkdir -pv /etc/puppet/modules/nginx/{manifests,files,lib,templates,tests,spec}mkdir: created directory `/etc/puppet/modules/nginx'mkdir: created directory `/etc/puppet/modules/nginx/manifests'mkdir: created directory `/etc/puppet/modules/nginx/files'mkdir: created directory `/etc/puppet/modules/nginx/lib'mkdir: created directory `/etc/puppet/modules/nginx/templates'mkdir: created directory `/etc/puppet/modules/nginx/tests'mkdir: created directory `/etc/puppet/modules/nginx/spec'# puppet module list/etc/puppet/modules└── nginx (???)/usr/share/puppet/modules (no modules installed)# 在nginx模块中定义init.pp# vi /etc/puppet/modules/nginx/manifests/init.ppclass nginx { package {'nginx': ensure => installed, }}# 定义nginx_web.pp文件# vi /etc/puppet/modules/nginx/manifests/nginx_web.ppclass nginx::nginx_web inherits nginx { file {'/etc/nginx/nginx.conf': ensure => file, source => 'puppet:///modules/nginx/nginx-web.conf', mode => '0644', owner => 'root', group => 'root', notify => Service['nginx'], require => Package['nginx'], } service {'nginx': ensure => running,}}# 准备source文件# cp /tmp/nginx.conf /etc/puppet/modules/nginx/files/nginx-web.conf# 创建site.pp文件调用前面定义的class# vi /etc/puppet/manifests/site.ppnode 'node2.luojianlong' { include nginx::nginx_web}node 'node3.luojianlong' { include nginx::nginx_web}
首次启动puppet服务进程可以以非守护进程方式进行，并让其输出详解信息以便于观察初始化过程。如下所示过程，其逐步展示了创建本地CA、作为puppet服务器的本地主机向CA申请证书、获得证书以及CA移出证书签署请求的过程等，而后启动服务进程并准备接受各agent的连接请求。为下面的命令额外使用--debug选项，还可以获得更为详细的输出信息。
# puppet master --verbose --no-daemonizeinfo: Creating a new SSL key for cainfo: Creating a new SSL certificate request for cainfo: Certificate Request fingerprint (md5): E0:74:ED:BA:83:EC:6E:A7:1A:1F:89:B1:CC:81:C3:CEnotice: Signed certificate request for canotice: Rebuilding inventory fileinfo: Creating a new certificate revocation listinfo: Creating a new SSL key for node1.luojianlong.cominfo: Creating a new SSL certificate request for node1.luojianlong.cominfo: Certificate Request fingerprint (md5): 05:F1:37:DE:6E:13:CA:32:46:5B:07:2A:05:DE:D1:12notice: node1.luojianlong.com has a waiting certificate requestnotice: Signed certificate request for node1.luojianlong.comnotice: Removing file Puppet::SSL::CertificateRequest node1.luojianlong.com at '/var/lib/puppet/ssl/ca/requests/node1.luojianlong.com.pem'notice: Removing file Puppet::SSL::CertificateRequest node1.luojianlong.com at '/var/lib/puppet/ssl/certificate_requests/node1.luojianlong.com.pem'notice: Starting Puppet master version 2.7.23
注意：如果此前曾以其它主机名或各种原因启动过puppet客户端进程并完成过初始化，其证书文件将无法符合本此启动的需要；此时，需要先清空/var/lib/puppet/ssl/目录方可完成后续的初始化操作。
如果上述的测试启动没有问题，可中止当前的启动后将之启动守护进程了，在CentOS6上，通常会使用如下命令进行。
# service puppetmaster startStarting puppetmaster: # chkconfig puppetmaster on
启动puppet客户端puppet agent在首次启动时，会向为其指定的puppet server申请证书，并完成后续连接请求。同样地理由，出于测试的目的，接入当前puppet集群中的首个agent节点可以以非守护进程的方式进行，以观察其初始化过程，如下面的命令所示
# puppet agent --server=node1.luojianlong.com --no-daemonize --verboseinfo: Creating a new SSL key for node2.luojianlong.cominfo: Caching certificate for cainfo: Creating a new SSL certificate request for node2.luojianlong.cominfo: Certificate Request fingerprint (md5): 11:56:36:0D:A5:92:11:69:AC:66:46:1B:86:D9:B4:ED# puppet agent --server=node1.luojianlong.com --no-daemonize --verboseinfo: Creating a new SSL key for node3.luojianlong.cominfo: Caching certificate for cainfo: Creating a new SSL certificate request for node3.luojianlong.cominfo: Certificate Request fingerprint (md5): A3:70:BF:52:F9:11:DA:0F:09:8B:35:C6:FC:EB:87:14
此时，在puppet服务器端使用puppet cert命令管理客户端的证书请求，其--list选项能够查看等待签署证书的客户端列表，而--sign选项可用于为指定指定节点签署证书，如果要一次性地为多个节点的证书申请进行签署可同时使用--all选项。
# puppet cert --list"node2.luojianlong.com" (11:56:36:0D:A5:92:11:69:AC:66:46:1B:86:D9:B4:ED)"node3.luojianlong.com" (A3:70:BF:52:F9:11:DA:0F:09:8B:35:C6:FC:EB:87:14)# puppet cert --sign node2.luojianlong.comnotice: Signed certificate request for node2.luojianlong.comnotice: Removing file Puppet::SSL::CertificateRequest node2.luojianlong.com at '/var/lib/puppet/ssl/ca/requests/node2.luojianlong.com.pem'# puppet cert --sign node3.luojianlong.comnotice: Signed certificate request for node3.luojianlong.comnotice: Removing file Puppet::SSL::CertificateRequest node3.luojianlong.com at '/var/lib/puppet/ssl/ca/requests/node3.luojianlong.com.pem'
一旦agent节点收到签署过的证书时，其就会显示类似如下信息。
# puppet agent --server=node1.luojianlong.com --no-daemonize --verboseinfo: Creating a new SSL key for node2.luojianlong.cominfo: Caching certificate for cainfo: Creating a new SSL certificate request for node2.luojianlong.cominfo: Certificate Request fingerprint (md5): 11:56:36:0D:A5:92:11:69:AC:66:46:1B:86:D9:B4:EDinfo: Caching certificate for node2.luojianlong.comnotice: Starting Puppet client version 2.7.23info: Caching certificate_revocation_list for cainfo: Caching catalog for node2.luojianlong.cominfo: Applying configuration version '1389325340'notice: /Stage/Nginx/Package/ensure: creatednotice: /Stage/Nginx::Nginx_web/Service/ensure: ensure changed 'stopped' to 'running'info: Creating state file /var/lib/puppet/state/state.yamlnotice: Finished catalog run in 10.22 seconds# puppet agent --server=node1.luojianlong.com --no-daemonize --verboseinfo: Creating a new SSL key for node3.luojianlong.cominfo: Caching certificate for cainfo: Creating a new SSL certificate request for node3.luojianlong.cominfo: Certificate Request fingerprint (md5): A3:70:BF:52:F9:11:DA:0F:09:8B:35:C6:FC:EB:87:14info: Caching certificate for node3.luojianlong.comnotice: Starting Puppet client version 2.7.23info: Caching certificate_revocation_list for cainfo: Caching catalog for node3.luojianlong.cominfo: Applying configuration version '1389325340'notice: /Stage/Nginx/Package/ensure: creatednotice: /Stage/Nginx::Nginx_web/Service/ensure: ensure changed 'stopped' to 'running'info: Creating state file /var/lib/puppet/state/state.yamlnotice: Finished catalog run in 17.83 seconds
确保上述agent相关的操作不存在问题后，便可以将--server选项指定的信息存储于agent的配置文件中，并以服务进程的方式启动puppet agent了。其配置文件为/etc/puppet/puppet.conf，server指令定义于段中。配置完成，即可以服务方式启动puppet。
# vi /etc/puppet/puppet.confserver = node1.luojianlong.com# vi /etc/puppet/puppet.confserver = node1.luojianlong.com# service puppet startStarting puppet: # service puppet startStarting puppet:
再次通过客户端测试。
# puppet agent --server=node1.luojianlong.com --no-daemonize --verbose --testinfo: Caching catalog for node2.luojianlong.cominfo: Applying configuration version '1389325340'notice: Finished catalog run in 0.97 seconds# puppet agent --server=node1.luojianlong.com --no-daemonize --verbose --testinfo: Caching catalog for node3.luojianlong.cominfo: Applying configuration version '1389325340'notice: Finished catalog run in 0.95 seconds如上的信息显示其已经能正常与master建立连接查看node2，node3 nginx是否安装并启动
# rpm -q nginxnginx-1.0.15-5.el6.x86_64# ps aux | grep nginxroot 192330.00.0964321968 ? Ss 12:18 0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.confnginx 192340.00.0967802612 ? S 12:18 0:00 nginx: worker process root 195150.00.0 103248 820 pts/0 S+ 12:22 0:00 grep nginx# rpm -q nginxnginx-1.0.15-5.el6.x86_64# ps aux | grep nginxroot 30820.00.0964321968 ? Ss 12:18 0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.confnginx 30830.00.0967802612 ? S 12:18 0:00 nginx: worker process root 32420.00.0 103248 824 pts/0 S+ 12:22 0:00 grep nginx
正常安装启动
自动签发证书可以设置master自动签发所有的证书，我们只需要在/etc/puppet 目录下创建 autosign.conf 文件,修改 /etc/puppet/puppet.conf文件
# cat > /etc/puppet/autosign.conf << EOF> *.luojianlong.com> EOF
# vi /etc/puppet/puppet.conf# 添加autosign = /etc/puppet/autosign.conf# service puppetmaster restartStopping puppetmaster: Starting puppetmaster:
这样就会对所有来自luojianlong.com 的机器的请求，都自动签名,puppet每半个小时检查一次更新，如果想修改检查时间，可以修改客户端配置文件/etc/puppet/puppet.conf，在中添加runinterval的值，然后重启puppet默认为600，单位秒。
在node1上安装配置puppet-dashboard:
# yum -y install rubygem-rake ruby-mysql# yum localinstall puppet-dashboard-1.2.23-1.el6.noarch.rpm -y# gem install rake
在node1上安装mysql
# tar zxvf mysql-5.5.33-linux2.6-x86_64.tar.gz -C /usr/local/# ln -s /usr/local/mysql-5.5.33-linux2.6-x86_64 /usr/local/mysql# cd /usr/local/mysql# useradd -r mysql# mkdir /mydata/data -p# chown -R root.mysql ./*# chown -R mysql.mysql /mydata/data/# cp support-files/mysql.server /etc/rc.d/init.d/mysqld# chkconfig --add mysqld# chkconfig mysqld on# cp support-files/my-large.cnf /etc/my.cnf# ./scripts/mysql_install_db --user=mysql --datadir=/mydata/data# vi /etc/profile.d/mysql.shexport PATH=/usr/local/mysql/bin:$PATH# . /etc/profile.d/mysql.sh# vi /etc/my.cnfdatadir = /mydata/datainnodb_file_per_table = 1# service mysqld startStarting MySQL..... SUCCESS!
创建数据库并完成授权
mysql> create database dashboard character set utf8;Query OK, 1 row affected (0.00 sec)mysql> grant all privileges on dashboard.* to 'dashboard'@'localhost' identified by '123456';Query OK, 0 rows affected (0.00 sec)mysql> flush privileges;Query OK, 0 rows affected (0.00 sec)
修改/usr/share/puppet-dashboard/config/database.yml中的production段。
# vi /usr/share/puppet-dashboard/config/database.ymlproduction:host: 127.0.0.1database: dashboardusername: dashboardpassword: 123456encoding: utf8adapter: mysql# cd /usr/share/puppet-dashboard/# rake gems:refresh_specs# 为dashboard依赖的数据库导入所需要的表：# rake RAILS_ENV=production db:migrate
测试服务器是否能正常工作
# /usr/share/puppet-dashboard/script/server -e production=> Booting WEBrick=> Rails 2.3.17 application starting on http://0.0.0.0:3000=> Call with -d to detach=> Ctrl-C to shutdown server INFOWEBrick 1.3.1 INFOruby 1.8.7 (2011-06-30) INFOWEBrick::HTTPServer#start: pid=20641 port=3000
打开浏览器访问http://192.168.30.116:3000
配置puppet服务端和客户端
# vi /etc/puppet/puppet.conf
#在段中添加
reports = store, http
reporturl = http://192.168.30.116:3000/reports/upload
# service puppetmaster restart
Stopping puppetmaster:
Starting puppetmaster:
# vi /etc/puppet/puppet.conf
# 在段中添加
report = true
# service puppet restart
Stopping puppet:
Starting puppet:
# node3也一样，添加并重启puppet
然后启动dashboard
# /usr/share/puppet-dashboard/script/server -e production -d
打开浏览器访问http://192.168.30.116:3000/
看到“# pending task”类的信息，且数字大于0，则表示已经正常接收报告了，一旦有用户任务延迟就会记录在dashboard中。
puppet kick 功能实现puppet客户端默认每30分钟跟服务器通讯一次，但是有时，我们希望服务端能给客户端紧急推送一些任务，于是就有了puppet kick（puppet 2.6以前叫puppetrun）。编辑客户端/etc/puppet/puppet.conf# vi /etc/puppet/puppet.conf# 在段中添加listen = true# 编辑或新建文件/etc/puppet/namespaceauth.conf# vi /etc/puppet/namespaceauth.confallow *.luojianlong.com
编辑文件auth.conf
# vi /etc/puppet/auth.conf# 添加如下几行path /runmethod saveallow node1.luojianlong.com# service puppet restartStopping puppet: Starting puppet: # netstat -anptl | grep rubytcp 0 0 0.0.0.0:8139 0.0.0.0:* LISTEN 27053/ruby
node3做上述一样的操作
在服务端运行命令
# puppet kick -a --host=node2.luojianlong.comTriggering node2.luojianlong.comGetting statusstatus is successnode2.luojianlong.com finished with exit code 0Finished# puppet kick -a --host=node3.luojianlong.comTriggering node3.luojianlong.comGetting statusstatus is successnode3.luojianlong.com finished with exit code 0Finished
发现可以正常推送
页:
[1]