1.1.docker swarm初始化
1.1.docker swarm-manager初始化
[root@docker-2-120 ~]# docker swarm init --advertise-addr 192.168.2.120
Swarm initialized: current node (shyex5019e84dup8kjwgy9esj) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join --token SWMTKN-1-2nioyk01u61y4xju7z13ngr70zglt5w0hj2bpfnn9l7d3lso34-c45yo9ivt30qmai0lp3zyb3ks 192.168.2.120:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions. 1.1.2.docker manager查看节点
[root@docker-2-120 ~]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
shyex5019e84dup8kjwgy9esj * docker-2-120 Ready Active Leader 18.09.0 #暂时只有管理节点
1.2.添加work节点
1.2.1.在dm1上执行:
[root@dm1 /]# docker swarm join --token SWMTKN-1-2nioyk01u61y4xju7z13ngr70zglt5w0hj2bpfnn9l7d3lso34-c45yo9ivt30qmai0lp3zyb3ks 192.168.2.120:2377
This node joined a swarm as a worker. 1.2.2.在dm2上执行:
[root@dm2 ~]# docker swarm join --token SWMTKN-1-2nioyk01u61y4xju7z13ngr70zglt5w0hj2bpfnn9l7d3lso34-c45yo9ivt30qmai0lp3zyb3ks 192.168.2.120:2377
This node joined a swarm as a worker. 1.2.3.在manger节点上查看:
[root@docker-2-120 ~]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
32ak19ny1nib5woq9wfij5cz4 dm1 Ready Active 18.09.0
jhf5lcacfdun78t4tie3dt0zy dm2 Ready Active 18.09.0
shyex5019e84dup8kjwgy9esj * docker-2-120 Ready Active Leader 18.09.0 二、部署service
2.1.在manager上运行nginx镜像的service
[root@docker-2-120 ~]# docker service create --name "ckl_web" nginx 2.1.1.查看service
[root@docker-2-120 ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
v3ifjb3k2yye ckl_web replicated 1/1 nginx:latest #ID:serviceID
#NAME :名称
#REPLICAS :当前的副本数量
#IMAGE :镜像名词
#PORTS:端口
2.1.2.查看副本状态
[root@docker-2-120 ~]# docker service ps ckl_web
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
cb1cv5dgqfrh ckl_web.1 nginx:latest docker-2-120 Running Running 59 seconds ago 查看容器运行:
[root@docker-2-120 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
acb5a7377b4f nginx:latest "nginx -g 'daemon of…" About a minute ago Up About a minute 80/tcp ckl_web.1.cb1cv5dgqfrhrt2bdpk8elzhw 2.2.提升副本,模拟负载均衡
2.2.1.在manager上执行
[root@docker-2-120 ~]# docker service scale ckl_web=6
ckl_web scaled to 6
overall progress: 6 out of 6 tasks
1/6: running [==================================================>]
2/6: running [==================================================>]
3/6: running [==================================================>]
4/6: running [==================================================>]
5/6: running [==================================================>]
6/6: running [==================================================>]
verify: Service converged 2.2.2.查看扩展后副本信息
[root@docker-2-120 ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
v3ifjb3k2yye ckl_web replicated 6/6 nginx:latest 查看容器运行节点:
[root@docker-2-120 ~]# docker service ps ckl_web
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
cb1cv5dgqfrh ckl_web.1 nginx:latest docker-2-120 Running Running 7 minutes ago
wbx0wbr9w8tq ckl_web.2 nginx:latest dm1 Running Running about a minute ago
mviu588mulo2 ckl_web.3 nginx:latest dm2 Running Running about a minute ago
ltnuyjpbj4eo ckl_web.4 nginx:latest dm2 Running Running about a minute ago
lil6pq81fnaa ckl_web.5 nginx:latest docker-2-120 Running Running about a minute ago
ap36bx8k6y65 ckl_web.6 nginx:latest dm1 Running Running about a minute ago #目前正好均匀的运行在三个节点上,每个节点2个容器运行
2.2.3.如果只希望容器运行在worker节点
[root@docker-2-120 ~]# docker node update --availability drain docker-2-120
docker-2-120 2.2.4.查看node节点状态
[root@docker-2-120 ~]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
32ak19ny1nib5woq9wfij5cz4 dm1 Ready Active 18.09.0
jhf5lcacfdun78t4tie3dt0zy dm2 Ready Active 18.09.0
shyex5019e84dup8kjwgy9esj * docker-2-120 Ready Drain Leader 18.09.0 2.2.5.查看service运行
[root@docker-2-120 ~]# docker service ps ckl_web
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
q5d9hbtzfszp ckl_web.1 nginx:latest dm1 Running Running 2 minutes ago
cb1cv5dgqfrh \_ ckl_web.1 nginx:latest docker-2-120 Shutdown Shutdown 2 minutes ago
wbx0wbr9w8tq ckl_web.2 nginx:latest dm1 Running Running 9 minutes ago
mviu588mulo2 ckl_web.3 nginx:latest dm2 Running Running 9 minutes ago
ltnuyjpbj4eo ckl_web.4 nginx:latest dm2 Running Running 9 minutes ago
ij9dt6fjjmhy ckl_web.5 nginx:latest dm2 Running Running 2 minutes ago
lil6pq81fnaa \_ ckl_web.5 nginx:latest docker-2-120 Shutdown Shutdown 2 minutes ago
ap36bx8k6y65 ckl_web.6 nginx:latest dm1 Running Running 9 minutes ago #原来manager节点运行的容器已经被分摊到dm1和dm2上了
2.3.减少副本
2.3.1.减少副本数
[root@docker-2-120 ~]# docker service scale ckl_web=4
ckl_web scaled to 4
overall progress: 4 out of 4 tasks
1/4: running [==================================================>]
2/4:
3/4: running [==================================================>]
4/4: running [==================================================>]
verify: Service converged 2.3.2.查看service
[root@docker-2-120 ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
v3ifjb3k2yye ckl_web replicated 4/4 nginx:latest #副本数为4
2.3.3.查看容器运行节点
[root@docker-2-120 ~]# docker service ps ckl_web
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
q5d9hbtzfszp ckl_web.1 nginx:latest dm1 Running Running 7 minutes ago
cb1cv5dgqfrh \_ ckl_web.1 nginx:latest docker-2-120 Shutdown Shutdown 7 minutes ago
wbx0wbr9w8tq ckl_web.2 nginx:latest dm1 Running Running 14 minutes ago
mviu588mulo2 ckl_web.3 nginx:latest dm2 Running Running 14 minutes ago
ltnuyjpbj4eo ckl_web.4 nginx:latest dm2 Running Running 14 minutes ago
lil6pq81fnaa ckl_web.5 nginx:latest docker-2-120 Shutdown Shutdown 7 minutes ago 三、故障转移
swarm内置failover策略可以实现故障转移
模拟dm1宕机
3.1.查看node节点状态
[root@docker-2-120 ~]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
32ak19ny1nib5woq9wfij5cz4 dm1 Down Active 18.09.0
jhf5lcacfdun78t4tie3dt0zy dm2 Ready Active 18.09.0
shyex5019e84dup8kjwgy9esj * docker-2-120 Ready Drain Leader 18.09.0
#dm1阶段状态为down 3.2.查看service状态
[root@docker-2-120 ~]# docker service ps ckl_web
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
rs3ntbl74en3 ckl_web.1 nginx:latest dm2 Running Running 2 minutes ago
q5d9hbtzfszp \_ ckl_web.1 nginx:latest dm1 Shutdown Running 25 minutes ago
cb1cv5dgqfrh \_ ckl_web.1 nginx:latest docker-2-120 Shutdown Shutdown 25 minutes ago
kwctm7bg4jha ckl_web.2 nginx:latest dm2 Running Running 2 minutes ago
wbx0wbr9w8tq \_ ckl_web.2 nginx:latest dm1 Shutdown Running 33 minutes ago
mviu588mulo2 ckl_web.3 nginx:latest dm2 Running Running 32 minutes ago
ltnuyjpbj4eo ckl_web.4 nginx:latest dm2 Running Running 32 minutes ago
lil6pq81fnaa ckl_web.5 nginx:latest docker-2-120 Shutdown Shutdown 25 minutes ago #运行在dm1上的容器,已都转移到dm2上了,故障节点状态是shutdown
四、访问docker service
4.1.在manager上部署service
[root@docker-2-120 ~]# docker service create --name "ckl_nginx" --replicas=2 nginx 4.2.service运行节点
[root@docker-2-120 ~]# docker service ps ckl_nginx
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
p28o2lobduog ckl_nginx.1 nginx:latest dm2 Running Running 29 seconds ago
38yanzwce36u ckl_nginx.2 nginx:latest dm1 Running Running 28 seconds ago 4.3.在dm1查看容器
[root@dm1 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6a835035f35f nginx:latest "nginx -g 'daemon of…" 51 seconds ago Up 49 seconds 80/tcp ckl_nginx.2.38yanzwce36uf007j0okvctus 查看容器并访问:
[root@dm1 ~]# curl http://10.8.95.2
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
4.4.端口映射,外网访问service
[root@docker-2-120 ~]# docker service update --publish-add 8000:80 ckl_nginx 查看service状态:
[root@docker-2-120 ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
ggjfmz39jzj7 ckl_nginx replicated 2/2 nginx:latest *:8000->80/tcp[root@docker-2-120 ~]# docker service ps ckl_nginx
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
r84wybzu2zez ckl_nginx.1 nginx:latest dm2 Running Running 27 seconds ago
p28o2lobduog \_ ckl_nginx.1 nginx:latest dm2 Shutdown Shutdown 29 seconds ago
t5xln1zllf3i ckl_nginx.2 nginx:latest dm1 Running Running 30 seconds ago
38yanzwce36u \_ ckl_nginx.2 nginx:latest dm1 Shutdown Shutdown 32 seconds ago 外网访问:
查看docker网络:
[root@docker-2-120 ~]# docker network ls
NETWORK ID NAME DRIVER SCOPE
9a316505e6c8 bridge bridge local
bf0464ee4b94 ckl_net bridge local
33a5e13e7f47 ckl_net1 bridge local
aed4e7c4891e host host local
43z63iliw2ok ingress overlay swarm #swarm创建的overlay网络
cf895048ef39 none null local 五、service 之间通信
5.1.创建overlay网络
[root@docker-2-120 ~]# docker network create --driver overlay ckl_ov
p6qmkf3jlbgescuwurwalu5me[root@docker-2-120 ~]# docker network ls
NETWORK ID NAME DRIVER SCOPE
9a316505e6c8 bridge bridge local
bf0464ee4b94 ckl_net bridge local
33a5e13e7f47 ckl_net1 bridge local
p6qmkf3jlbge ckl_ov overlay swarm #创建的新网络
aed4e7c4891e host host local
43z63iliw2ok ingress overlay swarm
cf895048ef39 none null local 5.2.创建web service
[root@docker-2-120 ~]# docker service create --name "ckl_web" --replicas=3 --network ckl_ov nginx
anlm52hs5p12efn5wras7h65v[root@docker-2-120 ~]# docker service ps ckl_web
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
tiqpv81m6icb ckl_web.1 nginx:latest dm2 Running Running about a minute ago
tl3i10w296kk ckl_web.2 nginx:latest dm1 Running Running about a minute ago
zdxxqy8zwnvt ckl_web.3 nginx:latest dm2 Running Running about a minute ago 5.3.创建centos service
[root@docker-2-120 ~]# docker service create --name "ckl_cent" --network ckl_ov centos sleep 10000
xut1xxmsp42mw4thixvd4iqt2
#两个service属于同一overlay网络 查看service状态:
[root@docker-2-120 ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
xut1xxmsp42m ckl_cent replicated 1/1 centos:latest
anlm52hs5p12 ckl_web replicated 3/3 nginx:latest[root@docker-2-120 ~]# docker service ps ckl_cent
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
fzs6ejkhangm ckl_cent.1 centos:latest dm1 Running Running 59 seconds ago
5.4.在dm1上测试连通
[root@dm1 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
710f6b55400d centos:latest "sleep 10000" About a minute ago Up About a minute ckl_cent.1.fzs6ejkhangm3r0eh8v46zljx
5f0c2ba01f88 nginx:latest "nginx -g 'daemon of…" About an hour ago Up About an hour 80/tcp ckl_web.2.tl3i10w296kks0qnh0ri8i0i4[root@dm1 ~]# docker exec -it 710f6b55400d /bin/bash
[root@710f6b55400d /]#
[root@710f6b55400d /]#
[root@710f6b55400d /]# ping -c3 ckl_web
PING ckl_web (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2 (10.0.0.2): icmp_seq=1 ttl=64 time=0.122 ms
64 bytes from 10.0.0.2 (10.0.0.2): icmp_seq=2 ttl=64 time=0.168 ms
64 bytes from 10.0.0.2 (10.0.0.2): icmp_seq=3 ttl=64 time=0.120 ms
--- ckl_web ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.120/0.136/0.168/0.025 ms #10.0.0.2是service 的VIP
在dm1上查看ip:
[root@dm1 ~]# docker inspect 5f0c2ba01f88 | grep IPAddress
"SecondaryIPAddresses": null,
"IPAddress": "",
"IPAddress": "10.0.0.4", 在dm2上查看ip:
[root@dm2 ~]# docker inspect `docker ps | grep ckl_web | awk '{print $1}'` | grep IPAddress
"SecondaryIPAddresses": null,
"IPAddress": "",
"IPAddress": "10.0.0.5",
"SecondaryIPAddresses": null,
"IPAddress": "",
"IPAddress": "10.0.0.3", #对于ckl_cent只需要知道webservice 的名词就可以访问,而不需要知道serviceip
六、swarm 升级副本
6.1.查看当前服务
[root@docker-2-120 ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
xut1xxmsp42m ckl_cent replicated 1/1 centos:latest
anlm52hs5p12 ckl_web replicated 3/3 nginx:latest[root@docker-2-120 ~]# docker service ps ckl_web
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
tiqpv81m6icb ckl_web.1 nginx:latest dm2 Running Running 16 hours ago
tl3i10w296kk ckl_web.2 nginx:latest dm1 Running Running 16 hours ago
zdxxqy8zwnvt ckl_web.3 nginx:latest dm2 Running Running 16 hours ago 6.2.升级副本
部署service,在worker节点上要预先存在的镜像,下面测试以及预先pull过
6.2.1.部署service服务
[root@docker-2-120 ~]# docker service create --name "ckl_web" --replicas=3 nginx:1.2.1
image nginx:1.2.1 could not be accessed on a registry to record
its digest. Each node will access nginx:1.2.1 independently,
possibly leading to different nodes running different
versions of the image.
vnwqr4prkgmqtrpagp3tu9c4n
overall progress: 3 out of 3 tasks
1/3: running [==================================================>]
2/3: running [==================================================>]
3/3: running [==================================================>]
verify: Service converged 查看service状态:
[root@docker-2-120 ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
xut1xxmsp42m ckl_cent replicated 1/1 centos:latest
vnwqr4prkgmq ckl_web replicated 3/3 nginx:1.2.1[root@docker-2-120 ~]# docker service ps ckl_web
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
vma7lxxn29gw ckl_web.1 nginx:1.2.1 dm2 Running Running 2 minutes ago
xiubyfckzete ckl_web.2 nginx:1.2.1 dm2 Running Running 2 minutes ago
gmysmkahj6bz ckl_web.3 nginx:1.2.1 dm1 Running Running 2 minutes ago
6.2.2.将service更新到1.2.2
[root@docker-2-120 ~]#
[root@docker-2-120 ~]# docker service update --image nginx:1.2.2 ckl_web
image nginx:1.2.2 could not be accessed on a registry to record
its digest. Each node will access nginx:1.2.2 independently,
possibly leading to different nodes running different
versions of the image.
ckl_web
overall progress: 3 out of 3 tasks
1/3: running [==================================================>]
2/3: running [==================================================>]
3/3: running [==================================================>]
verify: Service converged 查看service状态:
[root@docker-2-120 ~]# docker service ps ckl_web
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
s9b3pzrzvhgj ckl_web.1 nginx:1.2.2 dm2 Running Running 15 seconds ago
vma7lxxn29gw \_ ckl_web.1 nginx:1.2.1 dm2 Shutdown Shutdown 16 seconds ago
r1cp4b4c2yqb ckl_web.2 nginx:1.2.2 dm2 Running Running 11 seconds ago
xiubyfckzete \_ ckl_web.2 nginx:1.2.1 dm2 Shutdown Shutdown 13 seconds ago
t21saofxe7ai ckl_web.3 nginx:1.2.2 dm1 Running Running 18 seconds ago
gmysmkahj6bz \_ ckl_web.3 nginx:1.2.1 dm1 Shutdown Shutdown 20 seconds ago
#三个副本都已经升级到1.2.2 6.2.3.查看更新配置
[root@docker-2-120 ~]# docker service inspect ckl_web
....
},
"UpdateConfig": {
"Parallelism": 1, #一次并行,更新副本数量
"FailureAction": "pause",
"Monitor": 5000000000,
"MaxFailureRatio": 0,
"Order": "stop-first"
},
.... 6.2.4.设置并行更新副本的数量及更新延迟时间
--update-parallelism 设置并行更新副本的个数
--update-delay 设置更新间隔时长
模拟副本为6个,并行更新数量为2,延迟为50s
[root@docker-2-120 ~]# docker service update --replicas=6 --update-parallelism 2 --update-delay=30s --image nginx:2.2.1 ckl_web
image nginx:2.2.1 could not be accessed on a registry to record
its digest. Each node will access nginx:2.2.1 independently,
possibly leading to different nodes running different
versions of the image.
ckl_web
overall progress: 6 out of 6 tasks
1/6: running [==================================================>]
2/6: running [==================================================>]
3/6: running [==================================================>]
4/6: running [==================================================>]
5/6: running [==================================================>]
6/6: running [==================================================>]
verify: Service converged 6.2.5.查看更新结果
6.2.6.查看当前service配置
[root@docker-2-120 ~]# docker service inspect --pretty ckl_web
ID: mrntpv6h7nv1cm08rt3j4she4
Name: ckl_web
Service Mode: Replicated
Replicas: 6
UpdateStatus:
State: completed
Started: 4 minutes ago
Completed: 3 minutes ago
Message: update completed
Placement:
UpdateConfig:
Parallelism: 2 #并行更新数量
Delay: 30s #更新时间间隔
On failure: pause
Monitoring Period: 5s
Max failure ratio: 0
Update order: stop-first
RollbackConfig:
Parallelism: 1
On failure: pause
Monitoring Period: 5s
Max failure ratio: 0
Rollback order: stop-first
ContainerSpec:
Image: nginx:2.2.1
Init: false
Resources:
Endpoint Mode: vip 6.2.7.更新回退
更新完毕后,如果想回退则只能回退到上一次更新版本使用参数--rollback
[root@docker-2-120 ~]# docker service update --rollback ckl_web
ckl_web
rollback: manually requested rollback
overall progress: rolling back update: 3 out of 3 tasks
1/3: running [> ]
2/3: running [> ]
3/3: running [> ]
verify: Service converged 查看回退结果:
运行了三个副本,版本为1.2.2
七、设置service mode
service 除了replicated mode还有global mode,global mode运行每个host只能运行一个副本
7.1.创建service指定mode
[root@docker-2-120 ~]# docker service create --mode global --name "ckl_http" httpd
29bwhe6v2nltogg059svx6u3j
overall progress: 2 out of 2 tasks
32ak19ny1nib: running [==================================================>]
jhf5lcacfdun: running [==================================================>]
verify: Service converged[root@docker-2-120 ~]# docker service inspect ckl_http --pretty
ID: 29bwhe6v2nltogg059svx6u3j
Name: ckl_http
Service Mode: Global #mode
Placement:
UpdateConfig:
Parallelism: 1
On failure: pause
Monitoring Period: 5s
Max failure ratio: 0
Update order: stop-first
RollbackConfig:
Parallelism: 1
On failure: pause
Monitoring Period: 5s
Max failure ratio: 0
Rollback order: stop-first
ContainerSpec:
Image: httpd:latest@sha256:1a25dda4141b143ca018490fb4f64ce9aa8acb16c2660a7aa395db9fe4ae1793
Init: false
Resources:
Endpoint Mode: vip 八、使用label来控制service运行位置
要想控制service运行位置必须满足:为node定义label;设置service运行的label node
8.1.设置dm1为test环境
[root@docker-2-120 ~]# docker node update --label-add env=test dm1
dm1 查看dm1详情: