Azure disk iops的测试

zhaoke0727

　　在Public Cloud中，VM、Storage和Network是IaaS的三大基础。本文将介绍在Azure的VM上测试磁盘IOPS的工具和方法。
　　一、添加磁盘、初始化磁盘
　　1.添加磁盘

　　把相应的信息填写好后，点击OK，加载一块Disk到相应的VM。可以注意到这块Disk的IOPS是500，吞吐量是60MB/s。
　　2.分区和格式化
　　加载好后，可以在VM中看到这块Disk：

　　sda是系统盘，sdb是Azure VM的临时盘，之前加的一块盘是sdc，这块盘是sdd。
　　查看：



[iyunv@hwcentos ~]# fdisk -cul /dev/sdd

Disk /dev/sdd: 1098.4 GB, 1098437885952 bytes
255 heads, 63 sectors/track, 133544 cylinders, total 2145386496 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
　　磁盘容量1TB。



[iyunv@hwcentos ~]# fdisk -cu /dev/sdd
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xabbaf8a0.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): p
Disk /dev/sdd: 1098.4 GB, 1098437885952 bytes
255 heads, 63 sectors/track, 133544 cylinders, total 2145386496 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xabbaf8a0
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First sector (2048-2145386495, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-2145386495, default 2145386495):
Using default value 2145386495
Command (m for help): p
Disk /dev/sdd: 1098.4 GB, 1098437885952 bytes
255 heads, 63 sectors/track, 133544 cylinders, total 2145386496 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xabbaf8a0
Device Boot Start End Blocks Id System
/dev/sdd1 2048 2145386495 1072692224 83 Linux
Command (m for help): w
　　3.格式化硬盘
　　用mkfs.ext4格式化硬盘前，先查找一下帮助文件：



man mkfs.ext4
　　查找lazy，



lazy_itable_init[= <0 to disable, 1 to enable>]
If enabled and the uninit_bg feature is enabled, the inode table will not be fully initialized by mke2fs. This speeds up
filesystem initialization noticeably, but it requires the kernel to finish initializing the filesystem in the background when
the filesystem is first mounted. If the option value is omitted, it defaults to 1 to enable lazy inode table initialization.
　　通过这个选项可以快速对磁盘进行格式化。并且在第一次挂载的时候，系统在后台把系统初始化好。



[iyunv@hwcentos ~]# mkfs.ext4 -E lazy_itable_init /dev/sdd1
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
67043328 inodes, 268173056 blocks
13408652 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
8184 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
    102400000, 214990848
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information:
done
This filesystem will be automatically checked every 31 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
　　1T的硬盘瞬间就格式化好了。
　　挂载并查看：



[iyunv@hwcentos ~]# mkdir /1T
[iyunv@hwcentos ~]# mount /dev/sdd1 /1T
[iyunv@hwcentos ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 30G 2.0G 27G 8% /
tmpfs 841M 0 841M 0% /dev/shm
/dev/sdb1 69G 180M 66G 1% /mnt/resource
/dev/sdd1 1007G 200M 956G 1% /1T
　　二、测试工具IOPS
　　在Github搜索iops，排名第一的是cxcv/iops。
　　这是一个python的程序。
　　https://raw.githubusercontent.com/cxcv/iops/master/iops
　　运行这个程序测试刚刚创建的盘：



[iyunv@hwcentos ~]# ./iops.py /dev/sdd1
/dev/sdd1, 1.10 T, sectorsize=512B, #threads=32, pattern=random:
512 B blocks: 562.5 IO/s, 288.0 kB/s ( 2.3 Mbit/s)
1 kB blocks: 579.3 IO/s, 593.2 kB/s ( 4.7 Mbit/s)
2 kB blocks: 288.9 IO/s, 591.6 kB/s ( 4.7 Mbit/s)
4 kB blocks: 229.6 IO/s, 940.4 kB/s ( 7.5 Mbit/s)
8 kB blocks: 261.9 IO/s, 2.1 MB/s ( 17.2 Mbit/s)
16 kB blocks: 175.8 IO/s, 2.9 MB/s ( 23.0 Mbit/s)
32 kB blocks: 221.9 IO/s, 7.3 MB/s ( 58.2 Mbit/s)
65 kB blocks: 259.7 IO/s, 17.0 MB/s (136.1 Mbit/s)
131 kB blocks: 142.9 IO/s, 18.7 MB/s (149.9 Mbit/s)
262 kB blocks: 106.1 IO/s, 27.8 MB/s (222.5 Mbit/s)
524 kB blocks: 84.1 IO/s, 44.1 MB/s (352.6 Mbit/s)
1 MB blocks: 49.6 IO/s, 52.0 MB/s (416.1 Mbit/s)
2 MB blocks: 28.5 IO/s, 59.8 MB/s (478.2 Mbit/s)
4 MB blocks: 14.8 IO/s, 62.3 MB/s (498.3 Mbit/s)
8 MB blocks: 5.7 IO/s, 48.1 MB/s (384.6 Mbit/s)
　　可以看到IOPS在500左右，吞吐量在60MB/s左右。
　　三、fio测试磁盘性能
　　同样在Github上查找fio，axboe/fio就是fio工具。
　　同样，如果CentOS里设置了epel源，也可以直接yum安装：



[iyunv@hwcentos yum.repos.d]# yum search fio
Loaded plugins: fastestmirror, security
Repository 'epel' is missing name in configuration, using id
Loading mirror speeds from cached hostfile
epel | 4.4 kB 00:00
epel/primary_db | 6.3 MB 00:04
epel/pkgtags | 1.2 MB 00:01
============================================================================ N/S Matched: fio =============================================================================
dpm-rfio-server.x86_64 : DPM RFIO server
gfal2-plugin-rfio.x86_64 : Provide the rfio support for gfal2
fio.x86_64 : Multithreaded IO generation tool
root-io-rfio.x86_64 : Remote File input/output library for ROOT
Name and summary matches only, use "search all" for everything.
[iyunv@hwcentos yum.repos.d]# yum install fio -y
Loaded plugins: fastestmirror, security
Repository 'epel' is missing name in configuration, using id
Loading mirror speeds from cached hostfile
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package fio.x86_64 0:2.0.13-1.el6 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
===========================================================================================================================================================================
Package Arch Version Repository Size
===========================================================================================================================================================================
Installing:
fio x86_64 2.0.13-1.el6 epel 222 k
Transaction Summary
===========================================================================================================================================================================
Install 1 Package(s)
Total download size: 222 k
Installed size: 1.1 M
Downloading Packages:
fio-2.0.13-1.el6.x86_64.rpm | 222 kB 00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : fio-2.0.13-1.el6.x86_64 1/1
Verifying : fio-2.0.13-1.el6.x86_64 1/1
Installed:
fio.x86_64 0:2.0.13-1.el6
Complete!
[iyunv@hwcentos yum.repos.d]# yum info fio
Loaded plugins: fastestmirror, security
Repository 'epel' is missing name in configuration, using id
Loading mirror speeds from cached hostfile
Installed Packages
Name : fio
Arch : x86_64
Version : 2.0.13
Release : 1.el6
Size : 1.1 M
Repo : installed
From repo : epel
Summary : Multithreaded IO generation tool
URL : http://git.kernel.dk/?p=fio.git;a=summary
License : GPLv2
Description : fio is an I/O tool that will spawn a number of threads or processes doing
: a particular type of io action as specified by the user. fio takes a
: number of global parameters, each inherited by the thread unless
: otherwise parameters given to them overriding that setting is given.
: The typical use of fio is to write a job file matching the io load
: one wants to simulate.
　　Yum安装的版本是2.0.13，在Github上的是2.15版本。
　　安装好后，配置fio的测试配置文件：



[global]
ioengine=psync
direct=1
thread=1
norandommap=1
randrepeat=0
runtime=60
ramp_time=6
size=1g
directory=/path/to/test
[read4k-rand]
stonewall
group_reporting
bs=4k
rw=randread
numjobs=8
iodepth=32
[read64k-seq]
stonewall
group_reporting
bs=64k
rw=read
numjobs=4
iodepth=8
[write4k-rand]
stonewall
group_reporting
bs=4k
rw=randwrite
numjobs=2
iodepth=4
[write64k-seq]
stonewall
group_reporting
bs=64k
rw=write
numjobs=2
iodepth=4
　　这个配置文件中有4个测试，分别是顺序读、随机读、顺序写、随机写。同时测试的ioengine采用的是同步模式。
　　具体配置文件中的参数可以通过man fio查看。



fio fio.conf > sync.txt
grep iops sync.txt
　　可以看到不同测试场景下的测试结果



[iyunv@hwcentos ~]# grep iops sync.txt
read : io=113516KB, bw=1887.2KB/s, iops=471 , runt= 60152msec
read : io=1723.8MB, bw=29416KB/s, iops=459 , runt= 60005msec
write: io=109516KB, bw=1825.2KB/s, iops=456 , runt= 60004msec
write: io=926336KB, bw=15409KB/s, iops=240 , runt= 60116msec
　　在配置文件中把测试的ioengine换成libaio：



[global]
ioengine=libaio


fio fio.conf > async.txt
grep iops async.txt
[iyunv@hwcentos ~]# grep iops async.txt
read : io=119896KB, bw=1981.7KB/s, iops=491 , runt= 60503msec
read : io=2225.5MB, bw=37954KB/s, iops=592 , runt= 60043msec
write: io=115040KB, bw=1916.8KB/s, iops=479 , runt= 60019msec
write: io=1725.8MB, bw=29377KB/s, iops=458 , runt= 60155msec
　　iops有所增加，特别是随机写入增加比较大。