Kubernetes+Etcd-v1.7.0 + CA 分布式集群部署
kubernetes 1.7.0 + flannel 二进制部署kubernetes 1.7.0 + flannel
基于 二进制 文件部署 本地化 kube-apiserver, kube-controller-manager , kube-scheduler
(1).环境说明
k8s-master-1: 192.168.54.12
k8s-node1: 192.168.54.13
k8s-node2: 192.168.54.14
(2).初始化环境
hostnamectl --static set-hostname hostname
192.168.54.12 - k8s-master-1
192.168.54.13 - k8s-node1
192.168.54.14 - k8s-node2
#编辑 /etc/hosts 文件，配置hostname 通信
vi /etc/hosts
192.168.54.12 k8s-master-1
192.168.54.13 k8s-node1
192.168.54.14 k8s-node2
创建 验证
这里使用 CloudFlare 的 PKI 工具集 cfssl 来生成 Certificate Authority (CA) 证书和秘钥文件。
(1).安装 cfssl
mkdir -p /opt/local/cfssl
cd /opt/local/cfssl
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
mv cfssl_linux-amd64 cfssl
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
mv cfssljson_linux-amd64 cfssljson
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
mv cfssl-certinfo_linux-amd64 cfssl-certinfo
chmod +x *
(2).创建 CA 证书配置
mkd.ir /opt/ssl
cd /opt/ssl
/opt/local/cfssl/cfssl print-defaults config > config.json
/opt/local/cfssl/cfssl print-defaults csr > csr.json
# config.json 文件
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"kubernetes": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "87600h"
}
}
}
}
# csr.json 文件
{
"CN": "kubernetes",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "BeiJing",
"L": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
(3).生成 CA 证书和私钥
cd /opt/ssl/
/opt/local/cfssl/cfssl gencert -initca csr.json | /opt/local/cfssl/cfssljson -bare ca
# ls -lt
总用量 20
-rw-r--r-- 1 root root 1005 7月 3 17:26 ca.csr
-rw------- 1 root root 1675 7月 3 17:26 ca-key.pem
-rw-r--r-- 1 root root 1363 7月 3 17:26 ca.pem
-rw-r--r-- 1 root root210 7月 3 17:24 csr.json
-rw-r--r-- 1 root root292 7月 3 17:23 config.json
(4).分发证书
# 创建证书目录
mkdir -p /etc/kubernetes/ssl
# 拷贝所有文件到目录下
cp * /etc/kubernetes/ssl
# 这里要将文件拷贝到所有的k8s 机器上
scp * 192.168.54.13:/etc/kubernetes/ssl/
scp * 192.168.54.14:/etc/kubernetes/ssl/
etcd 集群
etcd 是k8s集群的基础组件，这里感觉没必要创建双向认证。
(1).安装 etcd
yum -y install etcd3
(2).修改 etcd 配置
# etcd-1
# 修改配置文件，/etc/etcd/etcd.conf 需要修改如下参数：
mv /etc/etcd/etcd.conf /etc/etcd/etcd.conf-bak
vi /etc/etcd/etcd.conf
ETCD_NAME=etcd1
ETCD_DATA_DIR="/var/lib/etcd/etcd1.etcd"
ETCD_LISTEN_PEER_URLS="http://192.168.54.12:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.54.12:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.54.12:2380"
ETCD_INITIAL_CLUSTER="etcd1=http://192.168.54.12:2380,etcd2=http://192.168.54.13:2380,etcd3=http://192.168.54.14:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="k8s-etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.54.12:2379"
# etcd-2
# 修改配置文件，/etc/etcd/etcd.conf 需要修改如下参数：
mv /etc/etcd/etcd.conf /etc/etcd/etcd.conf-bak
vi /etc/etcd/etcd.conf
ETCD_NAME=etcd2
ETCD_DATA_DIR="/var/lib/etcd/etcd2.etcd"
ETCD_LISTEN_PEER_URLS="http://192.168.54.13:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.54.13:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.54.13:2380"
ETCD_INITIAL_CLUSTER="etcd1=http://192.168.54.12:2380,etcd2=http://192.168.54.13:2380,etcd3=http://192.168.54.14:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="k8s-etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.54.13:2379"
# etcd-3
# 修改配置文件，/etc/etcd/etcd.conf 需要修改如下参数：
mv /etc/etcd/etcd.conf /etc/etcd/etcd.conf-bak
vi /etc/etcd/etcd.conf
ETCD_NAME=etcd3
ETCD_DATA_DIR="/var/lib/etcd/etcd3.etcd"
ETCD_LISTEN_PEER_URLS="http://192.168.54.14:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.54.14:2379,http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.54.14:2380"
ETCD_INITIAL_CLUSTER="etcd1=http://192.168.54.12:2380,etcd2=http://192.168.54.13:2380,etcd3=http://192.168.54.14:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="k8s-etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.54.14:2379"
修改 etcd 启动文件 /usr/lib/systemd/system/etcd.service
sed -i 's/\\\"${ETCD_LISTEN_CLIENT_URLS}\\\"/\\\"${ETCD_LISTEN_CLIENT_URLS}\\\" --listen-client-urls=\\\"${ETCD_LISTEN_CLIENT_URLS}\\\" --advertise-client-urls=\\\"${ETCD_ADVERTISE_CLIENT_URLS}\\\" --initial-cluster-token=\\\"${ETCD_INITIAL_CLUSTER_TOKEN}\\\" --initial-cluster=\\\"${ETCD_INITIAL_CLUSTER}\\\" --initial-cluster-state=\\\"${ETCD_INITIAL_CLUSTER_STATE}\\\"/g' /usr/lib/systemd/system/etcd.service
(3).启动 etcd
分别启动 所有节点的 etcd 服务
systemctl enable etcd
systemctl start etcd
systemctl status etcd
(4).验证 etcd 集群状态
查看 etcd 集群状态：
etcdctl cluster-health
# 出现 cluster is healthy 表示成功
查看 etcd 集群成员：
etcdctl member list
member 4b622f1d4543c5f7 is healthy: got healthy result from http://192.168.54.13:2379
member 647542be2d7fdef3 is healthy: got healthy result from http://192.168.54.12:2379
member 83464a62a714c625 is healthy: got healthy result from http://192.168.54.14:2379
Flannel 网络
(1).安装 flannel
这边其实由于内网，就没有使用SSL认证，直接使用了
yum -y install flannel
清除网络中遗留的docker 网络 (docker0, flannel0 等)
ifconfig
如果存在 请删除之，以免发生不必要的未知错误
ip link delete docker0
....
(2).配置 flannel
设置 flannel 所用到的IP段
etcdctl --endpoint http://192.168.54.12:2379 set /flannel/network/config '{"Network":"10.233.0.0/16","SubnetLen":25,"Backend":{"Type":"vxlan","VNI":1}}'
接下来修改 flannel 配置文件
vim /etc/sysconfig/flanneld
# 旧版本:
FLANNEL_ETCD="http://192.168.54.12:2379,http://192.168.54.13:2379,http://192.168.54.14:2379" # 修改为 集群地址
FLANNEL_ETCD_KEY="/flannel/network/config" # 修改为 上面导入配置中的/flannel/network
FLANNEL_OPTIONS="--iface=em1" # 修改为 本机物理网卡的名称
# 新版本:
FLANNEL_ETCD="http://192.168.54.12:2379,http://192.168.54.13:2379,http://192.168.54.14:2379" # 修改为 集群地址
FLANNEL_ETCD_PREFIX="/flannel/network" # 修改为 上面导入配置中的/flannel/network
FLANNEL_OPTIONS="--iface=em1" # 修改为 本机物理网卡的名称
(3).启动 flannel
systemctl enable flanneld
systemctl start flanneld
systemctl status flanneld
安装 docker
# 导入 yum 源
# 安装 yum-config-manager
yum -y install yum-utils
# 导入
yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
# 更新 repo
yum makecache
# 安装
yum install docker-ce
(1).更改docker 配置
# 修改配置
vi /usr/lib/systemd/system/docker.service
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target
Type=notify
ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS $DOCKER_OPTS $DOCKER_DNS_OPTIONS
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TimeoutStartSec=0
Delegate=yes
KillMode=process
Restart=on-failure
StartLimitBurst=3
StartLimitInterval=60s
WantedBy=multi-user.target
# 修改其他配置
cat >> /usr/lib/systemd/system/docker.service.d/docker-options.conf
页:
[1]