Ansible实现自动decommission Hadoop node
介绍如何用ansible的playbook实现自动decommission hadoop 的node ?正文
[*]对于如何部署安装ansible ，请看http://www.iyunv.com/thread-42283-1-1.html
[*]hadoop部署，请看http://www.iyunv.com/thread-42284-1-1.html
[*]使用hadoop用户创建目录ansible_decom,里面的目录结构如下：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
ansible_decom/
ansible_decom/roles
ansible_decom/roles/nn
ansible_decom/roles/nn/tasks
ansible_decom/roles/nn/tasks/main.yml
ansible_decom/roles/decom
ansible_decom/roles/decom/tasks
ansible_decom/roles/decom/tasks/main.yml
ansible_decom/roles/rm
ansible_decom/roles/rm/tasks
ansible_decom/roles/rm/tasks/main.yml
ansible_decom/roles/get-status
ansible_decom/roles/get-status/tasks
ansible_decom/roles/get-status/tasks/main.yml
ansible_decom/group_vars
ansible_decom/group_vars/all
ansible_decom/exclude-iplist
ansible_decom/site.yml
ansible_decom/hosts
以yml结尾的是playbook文件，其中site.yml是主文件，他会调用roles下面的子yml文件。
hosts文件定义的是主机列表。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
10.9.214.151
10.9.214.15
10.9.214.105
10.9.214.113
10.9.214.151
10.9.214.15
10.9.214.105
10.9.214.113
10.9.214.149
10.9.214.149
10.9.214.160
exclude-iplist定义的是要decommission的主机列表。
1
2
10.9.214.160
10.9.214.149
site.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
---
# this playbook displays decommission nodes
- name: add node to mapred-exclude and refreshnode
hosts: rmservers
remote_user: hadoop
roles:
- rm
- name: add node to hdfs-exclude and refreshnodes
hosts: nnservers
remote_user: hadoop
roles:
- nn
- name: stop nodemanager and datanode service
hosts: decomnodes
remote_user: hadoop
roles:
- decom
- name: get status for hosts
hosts: client
roles:
- get-status
roles/decom/tasks/main.yml
1
2
3
4
5
6
---
# stop services
- name: stop nodemanager
command: /opt/hadoop-2.6.0/sbin/yarn-daemon.sh stop nodemanager
- name: stop datanode
command: /opt/hadoop-2.6.0/sbin/hadoop-daemon.sh stop datanode
roles/nn/tasks/main.yml
1
2
3
4
5
6
7
8
---
# step1. add decom nodes to hdfs-excluse file
# step2. refresh nodes
- name: add decom nodes to hdfs-exclude file
copy: src=exclude-iplist dest=/opt/hadoop-2.6.0/etc/hadoop/
- shell: cd /opt/hadoop-2.6.0/etc/hadoop/ ; cat exclude-iplist >> hdfs-exclude ; rm exclude-iplist
- name: refresh nodes
shell: /opt/hadoop-2.6.0/bin/hdfs dfsadmin -refreshNodes
roles/rm/tasks/main.yml
1
2
3
4
5
6
7
8
9
---
# step1. add decom nodes to mapred-excluse file
# step2. refresh nodes
- name: add decom nodes to mapred-exclude file
copy: src=exclude-iplist dest=/opt/hadoop-2.6.0/etc/hadoop/
- shell: cd /opt/hadoop-2.6.0/etc/hadoop/ ; cat exclude-iplist >> mapred-exclude ; rm exclude-iplist
- name: refresh nodes
shell: /opt/hadoop-2.6.0/bin/yarn rmadmin -refreshNodes
4. 执行ansible的playbook
asible-playbook -i hosts site.yml
执行若有失败会报错的。
5. 完成之后查看http://10.9.214.151:50070看Live Node和Dead Node的数目和个数是不是变了。
页:
[1]