kubernetes使用ceph
一、有一个ceph cluster，假设已经准备好了，文档网上一大堆二、开始集成ceph和kuberntes
2.1 禁用rbd features
rbd image有4个 features，layering, exclusive-lock, object-map, fast-diff, deep-flatten
因为目前内核仅支持layering，修改默认配置
每个ceph node的/etc/ceph/ceph.conf 添加一行
rbd_default_features = 1
这样之后创建的image 只有这一个feature
验证方式：
ceph --show-config|grep rbd|grep features
rbd_default_features = 1
2.2 创建ceph-secret这个k8s secret对象，这个secret对象用于k8s volume插件访问ceph集群：
获取client.admin的keyring值，并用base64编码：
# ceph auth get-key client.admin
AQBRIaFYqWT8AhAAUtmJgeNFW/o1ylUzssQQhA==
# echo "AQBRIaFYqWT8AhAAUtmJgeNFW/o1ylUzssQQhA=="|base64
QVFCUklhRllxV1Q4QWhBQVV0bUpnZU5GVy9vMXlsVXpzc1FRaEE9PQo=
创建ceph-secret.yaml文件，data下的key字段值即为上面得到的编码值：
apiVersion: v1
kind: Secret
metadata:
name: ceph-secret
data:
key: QVFCUklhRllxV1Q4QWhBQVV0bUpnZU5GVy9vMXlsVXpzc1FRaEE9PQo=
创建ceph-secret:
# kubectl create -f ceph-secret.yaml secret "ceph-secret" created
# kubectl get secret
NAME TYPE DATA AGE
ceph-secretOpaque 1 2d
default-token-5vt3nkubernetes.io/service-account-token 3 106d
三、Kubernetes Persistent Volume和Persistent Volume Claim
概念：PV是集群的资源，PVC请求资源并检查资源是否可用
注意：以下操作设计到name的参数，一定要一致
3.1 创建disk image (以jdk保存到ceph举例)
# rbd create jdk-image -s 1G
# rbd info jdk-image
rbd image ‘jdk-image‘:
size 1024 MB in 256 objects
order 22 (4096 kB objects)
block_name_prefix: rbd_data.37642ae8944a
format: 2
features: layering
flags:
3.2 创建pv
创建mysql-hzb-pv.yaml:
方式一：
monitors: 就是ceph的mon，有几个写几个
apiVersion: v1
kind: PersistentVolume
metadata:
name: mysql-hzb-pv
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
rbd:
monitors:
- 172.16.60.41:6789
- 172.16.60.42:6789
- 172.16.60.43:6789
pool: rbd
image: mysql-hzb
user: admin
keyring: "/etc/ceph/ceph.client.admin.keyring"
fsType: ext4
readOnly: false
persistentVolumeReclaimPolicy: Recycle
方式二(仍然使用之前创建的ceph-secret)：
apiVersion: v1
kind: PersistentVolume
metadata:
name: mysql-hzb-pv
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
rbd:
monitors:
- 172.16.60.41:6789
- 172.16.60.42:6789
- 172.16.60.43:6789
pool: rbd
image: mysql-hzb
user: admin
secretRef:
name: ceph-secret
fsType: ext4
readOnly: false
persistentVolumeReclaimPolicy: Recycle
注意上面两种方式的红色部分的区别
执行创建操作：
# kubectl create -f mysql-hzb-pv.yaml persistentvolume "mysql-hzb-pv" created
#kubectl get pv
NAME CAPACITY ACCESSMODES RECLAIMPOLICY STATUS CLAIM REASON AGE
mysql-hzb-pv 10Gi RWO Recycle Available default/mysql-hzb-pvc 34m
3.3 创建pvc
创建mysql-hzb-pvc.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: mysql-hzb-pvc
spec:
accessModes:
- ReadWriteOnce
volumeName: mysql-hzb-pv
resources:
requests:
storage: 10Gi
执行创建操作：
# kubectl create -f mysql-hzb-pvc.yaml persistentvolumeclaim "mysql-hzb-pv" created
# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESSMODES AGE
mysql-hzb-pvc Bound mysql-hzb-pv 10Gi RWO 39s
3.4 创建挂载ceph rbd的pod:
创建 mysql-hzb-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: ceph-mysql-hzb-pod
spec:
containers:
- name: mysql-hzb
image: 172.16.101.192/common/mysql:5.6
env:
- name: MYSQL_ROOT_PASSWORD
value: "root"
volumeMounts:
- name: ceph-mysql-hzb-vl
mountPath: /var/lib/mysql
readOnly: false
volumes:
- name: ceph-mysql-hzb-vl
persistentVolumeClaim:
claimName: mysql-hzb-pvc
上面的两处红色的名字一定要一样
执行创建操作：
kubectl create -f mysql-hzb-pod.yaml
ceph rbd 持久化 这里描述下：
1、稳定性在于ceph
2、只能同一node挂载，不能跨node
3、读写只能一个pod，其他pod只能读
官方url描述
https://kubernetes.io/docs/user-guide/volumes/#rbd
附官方关于kubernetes的volume的mode
https://kubernetes.io/docs/user-guide/persistent-volumes/
https://s4.51cto.com/wyfs02/M02/8D/CA/wKiom1iqsp7wmQwqAAB6XqjmyUM506.png-wh_500x0-wm_3-wmp_4-s_1481390216.png
本文出自 “银狐” 博客，请务必保留此出处http://foxhound.blog.51cto.com/1167932/1899545
页:
[1]