分布式文件系统--------GlusterFS最佳实战

432232

1. 背景
    GlusterFS 是一个开源的分布式文件系统，具有强大的横向扩展能力，通过扩展能够支持数PB存储容量和处理数千客户端。GlusterFS借助TCP/IP或InfiniBand RDMA(一种支持多并发链接的“转换线缆”技术)网络将物理分布的存储资源聚集在一起，使用单一全局命名空间来管理数据。GlusterFS基于可堆叠的用户空间设计，可为各种不同的数据负载提供优异的性能。
      GlusterFS支持运行在任何标准IP网络上标准应用程序的标准客户端

2. 优势
　* 线性横向扩展和高性能
  * 高可用性
  * 全局统一命名空间
  * 弹性哈希算法和弹性卷管理
  * 基于标准协议
  * 完全软件实现（Software Only）
  * 用户空间实现（User Space）
  * 模块化堆栈式架构（Modular Stackable Architecture）
  * 原始数据格式存储（Data Stored in Native Formats）
  * 无元数据服务设计（No Metadata with the Elastic Hash Algorithm）

3. 环境
      server_1   CentOS 7.2.1511 (Core)  192.168.60.201
      server_2   CentOS 7.2.1511 (Core)  192.168.60.202

4. 安装
   * server_1 安装centos-release-gluster

1	[iyunv@server_1 ~]# yum install centos-release-gluster -y

   * server_1 安装 glusterfs-server

1	[iyunv@server_1 ~]# yum install glusterfs-server -y

   * server_1 启动 glusterfs-server 服务

1	[iyunv@server_1 ~]# systemctl start glusterd

   * server_2 安装centos-release-gluster

1	[iyunv@server_2 ~]# yum install centos-release-gluster -y

   * server_2 安装 glusterfs-server

1	[iyunv@server_2 ~]# yum install glusterfs-server -y

   * server_2 启动 glusterfs-server 服务

1	[iyunv@server_2 ~]# systemctl start glusterd

5. 建立信任池 [ 信任单向建立即可 ]
   * server_1 对 server_2　建立信任

1 2	[iyunv@server_1 ~]# gluster peer probe 192.168.60.202 peer probe: success.

　* 查看信任池建立情况

1 2 3 4 5 6 7 8 9 10 11 12 13

[iyunv@server_1 ~]# gluster peer status Number of Peers: 1 Hostname: 192.168.60.202 Uuid: 84d98fd8-4500-46d3-9d67-8bafacb5898b State: Peer in Cluster (Connected) [iyunv@server_2 ~]# gluster peer status Number of Peers: 1 Hostname: 192.168.60.201 Uuid: 20722daf-35c4-422c-99ff-6b0a41d07eb4 State: Peer in Cluster (Connected)

6. 创建分布式卷
   * server_1 和 server_2 创建数据存放目录

1 2	[iyunv@server_1 ~]# mkdir -p /data/exp1 [iyunv@server_2 ~]# mkdir -p /data/exp2

　 * 使用命令创建分布式卷，命名为test-volume

1 2	[iyunv@server_1 ~]# gluster volume create test-volume 192.168.60.201:/data/exp1 192.168.60.202:/data/exp2 force volume create: test-volume: success: please start the volume to access data

　 * 查看卷信息

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

[iyunv@server_1 ~]# gluster volume info test-volume    Volume Name: test-volume Type: Distribute Volume ID: 457ca1ff-ac55-4d59-b827-fb80fc0f4184 Status: Created Snapshot Count: 0 Number of Bricks: 2 Transport-type: tcp Bricks: Brick1: 192.168.60.201:/data/exp1 Brick2: 192.168.60.202:/data/exp2 Options Reconfigured: transport.address-family: inet nfs.disable: on [iyunv@server_2 ~]# gluster volume info test-volume    Volume Name: test-volume Type: Distribute Volume ID: 457ca1ff-ac55-4d59-b827-fb80fc0f4184 Status: Created Snapshot Count: 0 Number of Bricks: 2 Transport-type: tcp Bricks: Brick1: 192.168.60.201:/data/exp1 Brick2: 192.168.60.202:/data/exp2 Options Reconfigured: transport.address-family: inet nfs.disable: on

　 * 启动卷

1 2	[iyunv@server_1 ~]# gluster volume start test-volume volume start: test-volume: success

7. 创建复制卷　　[ 对比Raid 1 ]

   * server_1 和 server_2 创建数据存放目录

1 2	[iyunv@server_1 ~]# mkdir -p /data/exp3 [iyunv@server_2 ~]# mkdir -p /data/exp4

   * 使用命令创建复制卷，命名为repl-volume

1 2	[iyunv@server_1 ~]# gluster volume create repl-volume replica 2 transport tcp 192.168.60.201:/data/exp3 192.168.60.202:/data/exp4 force volume create: repl-volume: success: please start the volume to access data

   * 查看卷信息

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

[iyunv@server_1 ~]# gluster volume info repl-volume    Volume Name: repl-volume Type: Replicate Volume ID: 1924ed7b-73d4-45a9-af6d-fd19abb384cd Status: Created Snapshot Count: 0 Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 192.168.60.201:/data/exp3 Brick2: 192.168.60.202:/data/exp4 Options Reconfigured: transport.address-family: inet nfs.disable: on [iyunv@server_2 ~]# gluster volume info repl-volume    Volume Name: repl-volume Type: Replicate Volume ID: 1924ed7b-73d4-45a9-af6d-fd19abb384cd Status: Created Snapshot Count: 0 Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 192.168.60.201:/data/exp3 Brick2: 192.168.60.202:/data/exp4 Options Reconfigured: transport.address-family: inet nfs.disable: on

　 * 启动卷

1 2	[iyunv@server_1 ~]# gluster volume start repl-volume volume start: repl-volume: success

8. 创建条带卷  [ 对比Raid 0 ]
   * server_1 和 server_2 创建数据存放目录

1 2	[iyunv@server_1 ~]# mkdir -p /data/exp5 [iyunv@server_2 ~]# mkdir -p /data/exp6

   * 使用命令创建复制卷，命名为raid0-volume

1 2	[iyunv@server_1 ~]# gluster volume create raid0-volume stripe 2 transport tcp 192.168.60.201:/data/exp5 192.168.60.202:/data/exp6 force volume create: raid0-volume: success: please start the volume to access data

     *  查看卷信息

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

[iyunv@server_1 ~]# gluster volume info raid0-volume    Volume Name: raid0-volume Type: Stripe Volume ID: 13b36adb-7e8b-46e2-8949-f54eab5356f6 Status: Created Snapshot Count: 0 Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 192.168.60.201:/data/exp5 Brick2: 192.168.60.202:/data/exp6 Options Reconfigured: transport.address-family: inet nfs.disable: on [iyunv@server_2 ~]# gluster volume info raid0-volume    Volume Name: raid0-volume Type: Stripe Volume ID: 13b36adb-7e8b-46e2-8949-f54eab5356f6 Status: Created Snapshot Count: 0 Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: 192.168.60.201:/data/exp5 Brick2: 192.168.60.202:/data/exp6 Options Reconfigured: transport.address-family: inet nfs.disable: on

　 * 启动卷

1 2	[iyunv@server_1 ~]# gluster volume start raid0-volume volume start: raid0-volume: success

9. 客户端应用
   * 安装glusterfs-cli

1	[iyunv@client ~]# yum install glusterfs-cli -y

   * 创建挂载目录

1	[iyunv@client ~]# mkdir /mnt/g1 /mnt/g2 /mnt/g3

   * 挂载卷

1 2 3

[iyunv@server_1 ~]# mount.glusterfs 192.168.60.201:/test-volume /mnt/g1 [iyunv@server_1 ~]# mount.glusterfs 192.168.60.202:/repl-volume /mnt/g2 [iyunv@server_1 ~]# mount.glusterfs 192.168.60.201:/raid0-volume /mnt/g3

10. 扩展卷
  * 创建存放目录

1	[iyunv@server_1 ~]# mkdir -p /data/exp9

　 * 扩展卷

1 2	[iyunv@server_1 ~]# gluster volume add-brick test-volume 192.168.60.201:/data/exp9 force volume add-brick: success

　 * 重新均衡

1 2 3

[iyunv@server_1 ~]# gluster volume rebalance test-volume start volume rebalance: test-volume: success: Rebalance on test-volume has been started successfully. Use rebalance status command to check status of the rebalance process. ID: 008c3f28-d8a1-4f05-b63c-4543c51050ec

11. 总结

以需求驱动技术，技术本身没有优略之分，只有业务之分。