Home Iozone - A filesystem benchmark tool
Post
Cancel

Iozone - A filesystem benchmark tool

Introduction

IOzone is a filesystem benchmark tool. The benchmark generates and measures a variety of file operations.

Iozone is useful for performing a broad filesystem analysis of a vendor’s computer platform. The benchmark tests file I/O performance for the following operations:

  • Read, write, re-read, re-write, read backwards, read strided, fread, fwrite, random read, pread ,mmap, aio_read, aio_write

Example for throughput benchmark

In this example, the target is to measure the filesystem throughput(KB/s) for different workloads with 4k iosize. The workload operations include sequential read/write, random read/write and mix random read/write.

We run iozone on 6 mounted filesystems and 1GB files are read and written in each filesystem.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
$ /opt/iozone/bin/iozone -h
    -r #  record size in Kb
    -s #  file size in Kb
    -t #  Number of threads or processes to use in throughput test
    -I  Use VxFS VX_DIRECT, O_DIRECT,or O_DIRECTIO for all file operations    
    -F filenames  for each process/thread in throughput test
    -i #  Test to run (0=write/rewrite, 1=read/re-read, 2=random-read/write
        3=Read-backwards, 4=Re-write-record, 5=stride-read, 6=fwrite/re-fwrite
        7=fread/Re-fread, 8=random_mix, 9=pwrite/Re-pwrite, 10=pread/Re-pread
        11=pwritev/Re-pwritev, 12=preadv/Re-preadv)
    [...]    

$ /opt/iozone/bin/iozone -i 0 -i 1 -i 2 -i 8 -r 4k -s 1g -t 6 -I -F /testmnt1/testfile1 /testmnt2/testfile1 /testmnt3/testfile1 /testmnt4/testfile1 /testmnt5/testfile1 /testmnt6/testfile1 > iozone.out

$ cat iozone.out
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.489 $
		Compiled for 64 bit mode.
		Build: linux-AMD64

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
	             Vangel Bojaxhi, Ben England, Vikentsi Lapa,
	             Alexey Skidanov, Sudhir Kumar.

	Run began: Mon Mar  1 12:25:11 2021

	Record Size 4 kB
	File size set to 1048576 kB
	O_DIRECT feature enabled
	Command line used: /opt/iozone/bin/iozone -i 0 -i 1 -i 2 -i 8 -r 4k -s 1g -t 6 -I -F /testmnt1/testfile1 /testmnt2/testfile1 /testmnt3/testfile1 /testmnt4/testfile1 /testmnt5/testfile1 /testmnt6/testfile1
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
	Throughput test with 6 processes
	Each process writes a 1048576 kByte file in 4 kByte records

	Children see throughput for  6 initial writers 	=  151721.80 kB/sec
	Parent sees throughput for  6 initial writers 	=  146366.56 kB/sec
	Min throughput per process 			=   24712.40 kB/sec
	Max throughput per process 			=   25707.35 kB/sec
	Avg throughput per process 			=   25286.97 kB/sec
	Min xfer 					= 1007996.00 kB

	Children see throughput for  6 rewriters 	=  152089.88 kB/sec
	Parent sees throughput for  6 rewriters 	=  152084.55 kB/sec
	Min throughput per process 			=   25109.69 kB/sec
	Max throughput per process 			=   25674.81 kB/sec
	Avg throughput per process 			=   25348.31 kB/sec
	Min xfer 					= 1025500.00 kB

 	Children see throughput for  6 readers 		=    7618.06 kB/sec
	Parent sees throughput for  6 readers 		=    7618.04 kB/sec
	Min throughput per process 			=    1268.31 kB/sec
	Max throughput per process 			=    1270.73 kB/sec
	Avg throughput per process 			=    1269.68 kB/sec
	Min xfer 					= 1046580.00 kB

	Children see throughput for 6 re-readers 	=    7629.77 kB/sec
	Parent sees throughput for 6 re-readers 	=    7629.74 kB/sec
	Min throughput per process 			=    1270.79 kB/sec
	Max throughput per process 			=    1273.63 kB/sec
	Avg throughput per process 			=    1271.63 kB/sec
	Min xfer 					= 1046240.00 kB

	Children see throughput for 6 random readers 	=    7605.91 kB/sec
	Parent sees throughput for 6 random readers 	=    7605.89 kB/sec
	Min throughput per process 			=    1266.91 kB/sec
	Max throughput per process 			=    1268.54 kB/sec
	Avg throughput per process 			=    1267.65 kB/sec
	Min xfer 					= 1047228.00 kB

	Children see throughput for 6 mixed workload 	=   79687.92 kB/sec
	Parent sees throughput for 6 mixed workload 	=   78974.22 kB/sec
	Min throughput per process 			=    1275.41 kB/sec
	Max throughput per process 			=   25449.38 kB/sec
	Avg throughput per process 			=   13281.32 kB/sec
	Min xfer 					=   52552.00 kB

	Children see throughput for 6 random writers 	=  146210.38 kB/sec
	Parent sees throughput for 6 random writers 	=  143822.17 kB/sec
	Min throughput per process 			=   24206.99 kB/sec
	Max throughput per process 			=   24653.06 kB/sec
	Avg throughput per process 			=   24368.40 kB/sec
	Min xfer 					= 1029604.00 kB

iozone test complete.   

iostat

In the above test, we use a flash array with single 500TB LUN created. There are 4 active paths from host to the single LUN. Six logical volumes and filesystems are created on the single LUN.

The following is a piece of iostat output for one of the four disks(paths) when the mix random read/write workload is running. The read throughput is ~940KB/s and the write throughput is 19MB/s.

To measure the maximum throughput, we need keep increasing the number of read/write threads until the throughput(KB/s) is capped. Also, the iosize has big impact on the throughput. We may test with different I/O sizes.

1
2
3
4
5
6
7
8
9
10
Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdd               0.00     0.00  189.80 4255.20   759.20 17020.80     8.00     1.13    0.25    3.07    0.13   0.18  81.80
sdd               0.00     0.00  237.20 4708.20   948.80 18832.80     8.00     1.35    0.27    3.08    0.13   0.20  96.94
sdd               0.00     0.00  235.40 4898.20   941.60 19592.80     8.00     1.37    0.27    3.06    0.13   0.19  99.54
sdd               0.00     0.00  229.60 4575.60   918.40 18302.40     8.00     1.32    0.27    3.09    0.13   0.20  94.28
sdd               0.00     0.00  229.94 4822.55   919.76 19290.22     8.00     1.32    0.26    3.06    0.13   0.19  96.89
sdd               0.00     0.00  228.80 4512.40   915.20 18049.60     8.00     1.30    0.27    3.07    0.13   0.20  94.40
sdd               0.00     0.00  234.20 4810.20   936.80 19240.80     8.00     1.33    0.26    3.08    0.13   0.19  97.10
sdd               0.00     0.00  246.60 4497.40   986.40 17989.60     8.00     1.35    0.28    3.06    0.13   0.20  97.04
sdd               0.00     0.00  106.60 1665.60   426.40  6661.70     8.00     0.55    0.31    3.05    0.14   0.22  38.26

References

This post is licensed under CC BY 4.0 by the author.

Kubernetes and Gluster performance

Max open files limit

Comments powered by Disqus.