kubernetes1.8.9二进制安装并部署简单的tomcat+mysql应用
kubernetes是目前容器编排管理较为活跃的工具，本人最近参考书籍以及网上资料，在内网环境尝试手动安装并记录下来备忘文中部分包可能需要科学上网，请自行解决
感谢该文作者：http://blog.csdn.net/newcrane/article/details/78952987
一：准备工作
[*] 准备3台主机，一台作为master节点，两台作为node节点
192.168.0.44 master
192.168.0.45 node1
192.168.0.46 node2
将上述记录写入三台主机的/etc/hosts文件中
2.关闭3个节点的selinux，swap，firewalld
3.编辑内核参数，写入文件并source
]# cat /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
vm.swappiness=0
]# sysctl -p /etc/sysctl.d/k8s.conf
4.加载所需模块
]# modprobe br_netfilter
]# echo "modprobe br_netfilter" >> /etc/rc.local
5.设置iptables为ACCEPT
]#/sbin/iptables -P FORWARD ACCEPT
]#echo"sleep 60 && /sbin/iptables -P FORWARD ACCEPT" >> /etc/rc.local
6.安装依赖包
yum install -y epel-release
yum install -y yum-utils device-mapper-persistent-data lvm2 net-tools conntrack-tools wget
二.创建CA证书以及秘钥文件
CA证书签名只需要在master节点上进行操作就可以了，完成之后将node所需证书拷贝过去即可。本文采用cfssl进行签名认证
[*] 安装cfssl
1）创建目录并cd进入
mkdir /usr/local/cfssl/
cd /usr/local/cfssl/
2）下载所需二进制文件
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
3）赋予执行权限
chmod+x* 4）修改PATH变量并使其生效
]#cat /etc/profile.d/cfssl.sh
export PATH=$PATH:/usr/local/cfssl
]#source/etc/profile.d/cfssl.sh
2.创建CA配置文件
]#mkdir/etc/kubernetes/cfssl/
]#cd /etc/kubernetes/cfssl/
]# catca-config.json
{
"signing": {
"default": {
"expiry": "8760h"
},
"profiles": {
"kubernetes": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "8760h"
}
}
}
}
3.创建CA证书签名请求
]# cat ca-csr.json
{
"CN": "kubernetes",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "Sichuan",
"L": "Chengdu",
"O": "k8s",
"OU": "System"
}
]
}
4.生成CA 证书和私钥
cfssl gencert -initca ca-csr.json | cfssljson -bare ca 5.创建 kubernetes 证书签名请求文件并生成证书
]#cat kubernetes-csr.json
{
"CN": "kubernetes",
"hosts": [
"127.0.0.1",
"192.168.0.44",
"10.254.0.1",
"kubernetes",
"kubernetes.default",
"kubernetes.default.svc",
"kubernetes.default.svc.cluster",
"kubernetes.default.svc.cluster.local"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "Sichuan",
"L": "Chengdu",
"O": "k8s",
"OU": "System"
}
]
}
]#cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes
上述ip地址自行更改为自身系统中的IP地址10.254.0.1此IP地址为kubernetes服务虚拟地址，对应kube-apiserver服务中定义的地址段第一个可用地址
6.创建并生成admin证书及秘钥
]# cat admin-csr.json
{
"CN": "admin",
"hosts": [],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "Sichuan",
"L": "Chengdu",
"O": "system:masters",
"OU": "System"
}
]
}
]#cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin
7.创建并生成kube-proxy证书秘钥
]# cat kube-proxy-csr.json
{
"CN": "system:kube-proxy",
"hosts": [],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "Sichuan",
"L": "Chengdu",
"O": "k8s",
"OU": "System"
}
]
}
]#cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kuberneteskube-proxy-csr.json | cfssljson -bare kube-proxy
8.将生成的证书秘钥拷贝至node节点，保证3个节点上都有一份
scp*.pem192.168.0.45:/etc/kubernetes/cfssl
scp*.pem192.168.0.46:/etc/kubernetes/cfssl
二.部署ETCD
etcd是kubernetes集群的主数据库，本次架构中只需要在主节点安装即可
[*] 下载并解压etcd
]# wget https://github.com/coreos/etcd/releases/download/v3.3.2/etcd-v3.3.2-linux-amd64.tar.gz
]# tar xzf etcd-v3.3.2-linux-amd64.tar.gz
]# mv etcd-v3.3.2-linux-amd64 /usr/local/etcd
## 添加PATH路径
]# cat /etc/profile.d/etcd.sh
export PATH=$PATH:/usr/local/etcd/
]# source/etc/profile.d/etcd.sh
2.创建工作目录
mkdir /var/lib/etcd 3.创建systemd unit
]# cat /usr/lib/systemd/system/etcd.service
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/local/etcd/etcd \
--name master \
--cert-file=/etc/kubernetes/cfssl/kubernetes.pem \
--key-file=/etc/kubernetes/cfssl/kubernetes-key.pem \
--peer-cert-file=/etc/kubernetes/cfssl/kubernetes.pem \
--peer-key-file=/etc/kubernetes/cfssl/kubernetes-key.pem \
--trusted-ca-file=/etc/kubernetes/cfssl/ca.pem \
--peer-trusted-ca-file=/etc/kubernetes/cfssl/ca.pem \
--initial-advertise-peer-urls https://192.168.0.44:2380 \
--listen-peer-urls https://192.168.0.44:2380 \
--listen-client-urls https://192.168.0.44:2379,http://127.0.0.1:2379 \
--advertise-client-urls https://192.168.0.44:2379 \
--data-dir=/var/lib/etcd
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
WantedBy=multi-user.target
4.重载及启动服务
systemctl daemon-reload
systemctl enable etcd
systemctl start etcd
systemctl status etcd
注：上述unit文件中参数配置也可以使用配置文件形式，只需在一栏中注明即可，具体配置文件配置方式可参考官网，或者自行上网搜索
EnvironmentFile=-/etc/etcd/etcd.conf
三.部署flannel
flannel是CoreOS提供用于解决Dokcer集群跨主机通讯的覆盖网络工具，也可以使用OVS等工具，3个节点均需要部署flannel
1.下载并安装flannel
]# mkdir /usr/local/flannel
]# cd /usr/local/flannel/
]# wget https://github.com/coreos/flannel/releases/download/v0.9.1/flannel-v0.9.1-linux-amd64.tar.gz
]# tar -xzvf flannel-v0.9.1-linux-amd64.tar.gz
]# cat /etc/profile.d/flannel
export PATH=$PATH:/usr/local/flannel/
]#source /etc/profile.d/flannel
2.向 etcd 写入网段信息 ，只需要在master节点操作即可
etcdctl --endpoints=https://192.168.0.44:2379 \
--ca-file=/etc/kubernetes/cfssl/ca.pem \
--cert-file=/etc/kubernetes/cfssl/kubernetes.pem \
--key-file=/etc/kubernetes/cfssl/kubernetes-key.pem \
mkdir /kubernetes/network
etcdctl --endpoints=https://192.168.0.44:2379 \
--ca-file=/etc/kubernetes/cfssl/ca.pem \
--cert-file=/etc/kubernetes/cfssl/kubernetes.pem \
--key-file=/etc/kubernetes/cfssl/kubernetes-key.pem \
mk /kubernetes/network/config '{"Network":"172.30.0.0/16","SubnetLen":24,"Backend":{"Type":"vxlan"}}'
3.创建systemd unit 文件
~]# cat /usr/lib/systemd/system/flanneld.service
Description=Flanneld overlay address etcd agent
After=network.target
After=network-online.target
Wants=network-online.target
After=etcd.service
Before=docker.service
Type=notify
ExecStart=/usr/local/flannel/flanneld \
-etcd-cafile=/etc/kubernetes/cfssl/ca.pem \
-etcd-certfile=/etc/kubernetes/cfssl/kubernetes.pem \
-etcd-keyfile=/etc/kubernetes/cfssl/kubernetes-key.pem \
-etcd-endpoints=https://192.168.0.44:2379 \
-etcd-prefix=/kubernetes/network
ExecStartPost=/usr/local/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker
Restart=on-failure
WantedBy=multi-user.target
RequiredBy=docker.service
4.重载并启动flannel
systemctl daemon-reload
systemctl enable flanneld
systemctl start flanneld
systemctl status flanneld
可以通过以下命令查看flannel服务状态
~]# etcdctl --endpoints=https://192.168.0.44:2379 \
--ca-file=/etc/kubernetes/cfssl/ca.pem \
--cert-file=/etc/kubernetes/cfssl/kubernetes.pem \
--key-file=/etc/kubernetes/cfssl/kubernetes-key.pem \
ls /kubernetes/network/subnets
/kubernetes/network/subnets/172.30.38.0-24
/kubernetes/network/subnets/172.30.37.0-24
/kubernetes/network/subnets/172.30.5.0-24
四.部署 kubectl 工具，创建kubeconfig文件
工具安装需要在3台节点上进行安装，配置文件生成可以在master主机上生成拷贝至node节点
1.下载kubectl并安装
~]# wgethttps://dl.k8s.io/v1.8.9/kubernetes-server-linux-amd64.tar.gz
~]# tar xzf kubernetes-server-linux-amd64.tar.gz
~]# mv kubernetes /usr/local/
~]# cat /etc/profile.d/kubernetes.sh
export PATH=$PATH:/usr/local/kubernetes/server/bin/
2.创建/root/.kube/config
# 设置集群参数,--server指定Master节点ip
kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/cfssl/ca.pem \
--embed-certs=true \
--server=https://192.168.0.44:6443
# 设置客户端认证参数
kubectl config set-credentials admin \
--client-certificate=/etc/kubernetes/cfssl/admin.pem \
--embed-certs=true \
--client-key=/etc/kubernetes/cfssl/admin-key.pem
# 设置上下文参数
kubectl config set-context kubernetes \
--cluster=kubernetes \
--user=admin
# 设置默认上下文
kubectl config use-context kubernetes
3.创建bootstrap.kubeconfig
#生成token 变量export BOOTSTRAP_TOKEN=$(head -c 16 /dev/urandom | od -An -t x | tr -d ' ')
cat > token.csv
页:
[1]