fio directory and filename options
To run fio benchmark on multiple files or deives, we should understand the following fio options.
directory=str
Prefix filenames with this directory. Used to place files in a different location than ./. You can specify a number of directories by separating the names with a ‘:’ character. These directories will be assigned equally distributed to job clones created by numjobs as long as they are using generated filenames. If specific filename(s) are set fio will use the first listed directory, and thereby matching the filename semantic (which generates a file for each clone if not specified, but lets all clones use the same file if set).
filename=str
Fio normally makes up a filename based on the job name, thread number, and file number (see filename_format). If you want to share files between threads in a job or several jobs with fixed file paths, specify a filename for each of them to override the default. If the ioengine is file based, you can specify a number of files by separating the names with a ‘:’ colon. So if you wanted a job to open /dev/sda and /dev/sdb as the two working files, you would use filename=/dev/sda:/dev/sdb. This also means that whenever this option is specified, nrfiles is ignored. The size of regular files specified by this option will be size divided by number of files unless an explicit size is specified by filesize.
Run fio on single directory
The following example runs four fio jobs on single directory dir1. Four different files are laid out automatically before the benchmark.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
$ fio --name=4kwrite --ioengine=libaio --directory=dir1 --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
4kwrite: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=128
...
fio-3.7
Starting 4 processes
4kwrite: Laying out IO file (1 file / 10240MiB)
4kwrite: Laying out IO file (1 file / 10240MiB)
4kwrite: Laying out IO file (1 file / 10240MiB)
4kwrite: Laying out IO file (1 file / 10240MiB)
bs: 4 (f=4): [W(4)][4.5%][r=0KiB/s,w=394MiB/s][r=0,w=101k IOPS][eta 01m:46s]
<...>
$ ps -ef |grep fio | grep -v grep
root 25940 27212 23 21:10 pts/1 00:00:00 fio --name=4kwrite --ioengine=libaio --directory=dir1 --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 25976 25940 27 21:10 ? 00:00:01 fio --name=4kwrite --ioengine=libaio --directory=dir1 --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 25977 25940 28 21:10 ? 00:00:01 fio --name=4kwrite --ioengine=libaio --directory=dir1 --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 25978 25940 28 21:10 ? 00:00:01 fio --name=4kwrite --ioengine=libaio --directory=dir1 --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 25979 25940 27 21:10 ? 00:00:01 fio --name=4kwrite --ioengine=libaio --directory=dir1 --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
$ lsof | egrep "dir1"
fio 25976 root 3u REG 253,2 10737418240 23782491 /dir1/4kwrite.0.0
fio 25977 root 3u REG 253,2 10737418240 23782492 /dir1/4kwrite.3.0
fio 25978 root 3u REG 253,2 10737418240 23782495 /dir1/4kwrite.2.0
fio 25979 root 3u REG 253,2 10737418240 5234528 /dir1/4kwrite.1.0
$ ls -la dir1 | grep write
-rw-r--r-- 1 root root 10737418240 Mar 1 21:11 4kwrite.0.0
-rw-r--r-- 1 root root 10737418240 Mar 1 21:11 4kwrite.1.0
-rw-r--r-- 1 root root 10737418240 Mar 1 21:11 4kwrite.2.0
-rw-r--r-- 1 root root 10737418240 Mar 1 21:11 4kwrite.3.0
Run fio on multiple directories
The following example runs four fio jobs on two directories dir1 and dir2. Two files are laid out automatically under each directory.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
$ fio --name=4kwrite --ioengine=libaio --directory=dir1:dir2 --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
$ ps -ef |grep fio | grep write
root 27362 27212 3 21:13 pts/1 00:00:01 fio --name=4kwrite --ioengine=libaio --directory=dir1:dir2 --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 27396 27362 29 21:13 ? 00:00:08 fio --name=4kwrite --ioengine=libaio --directory=dir1:dir2 --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 27397 27362 30 21:13 ? 00:00:08 fio --name=4kwrite --ioengine=libaio --directory=dir1:dir2 --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 27398 27362 31 21:13 ? 00:00:09 fio --name=4kwrite --ioengine=libaio --directory=dir1:dir2 --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 27399 27362 30 21:13 ? 00:00:08 fio --name=4kwrite --ioengine=libaio --directory=dir1:dir2 --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
$ lsof | egrep "dir1|dir2"
fio 27396 root 3u REG 253,2 10737418240 23782491 /dir1/4kwrite.0.0
fio 27397 root 3u REG 253,2 10737418240 538334779 /dir2/4kwrite.3.0
fio 27398 root 3u REG 253,2 10737418240 23782492 /dir1/4kwrite.2.0
fio 27399 root 3u REG 253,2 10737418240 538334780 /dir2/4kwrite.1.0
$ ls -ltr dir*/
dir2/:
total 20971520
-rw-r--r-- 1 root root 10737418240 Mar 1 21:13 4kwrite.3.0
-rw-r--r-- 1 root root 10737418240 Mar 1 21:13 4kwrite.1.0
dir1/:
total 20971520
-rw-r--r-- 1 root root 10737418240 Mar 1 21:13 4kwrite.2.0
-rw-r--r-- 1 root root 10737418240 Mar 1 21:13 4kwrite.0.0
If the option filename is specified, only the first listed directory will be used to create files.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
$ fio --name=4kwrite --ioengine=libaio --directory=dir1:dir2 --filename=testfile --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
$ ps -ef |grep fio | grep write
root 29764 27212 8 21:17 pts/1 00:00:00 fio --name=4kwrite --ioengine=libaio --directory=dir1:dir2 --filename=testfile --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 29798 29764 33 21:17 ? 00:00:04 fio --name=4kwrite --ioengine=libaio --directory=dir1:dir2 --filename=testfile --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 29799 29764 35 21:17 ? 00:00:04 fio --name=4kwrite --ioengine=libaio --directory=dir1:dir2 --filename=testfile --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 29800 29764 35 21:17 ? 00:00:04 fio --name=4kwrite --ioengine=libaio --directory=dir1:dir2 --filename=testfile --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 29801 29764 33 21:17 ? 00:00:03 fio --name=4kwrite --ioengine=libaio --directory=dir1:dir2 --filename=testfile --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
$ lsof | egrep "dir1|dir2"
fio 29798 root 3u REG 253,2 10737418240 23782491 /dir1/testfile
fio 29799 root 3u REG 253,2 10737418240 23782491 /dir1/testfile
fio 29800 root 3u REG 253,2 10737418240 23782491 /dir1/testfile
fio 29801 root 3u REG 253,2 10737418240 23782491 /dir1/testfile
$ ls -la dir*/
dir1/:
total 10485760
drwxr-xr-x 2 root root 22 Mar 1 21:17 .
drwxr-xr-x 7 root root 225 Mar 1 20:28 ..
-rw-r--r-- 1 root root 10737418240 Mar 1 21:18 testfile
dir2/:
total 0
drwxr-xr-x 2 root root 6 Mar 1 21:16 .
drwxr-xr-x 7 root root 225 Mar 1 20:28 ..
Run multiple fio jobs on single file
The following example runs four jobs on single file.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
$ fio --name=4kwrite --ioengine=libaio --directory=dir1 --filename=testfile --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
$ ps -ef |grep fio | grep write
root 28819 27212 9 21:16 pts/1 00:00:00 fio --name=4kwrite --ioengine=libaio --directory=dir1 --filename=testfile --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 28884 28819 34 21:16 ? 00:00:03 fio --name=4kwrite --ioengine=libaio --directory=dir1 --filename=testfile --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 28885 28819 33 21:16 ? 00:00:02 fio --name=4kwrite --ioengine=libaio --directory=dir1 --filename=testfile --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 28886 28819 36 21:16 ? 00:00:03 fio --name=4kwrite --ioengine=libaio --directory=dir1 --filename=testfile --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
root 28887 28819 35 21:16 ? 00:00:03 fio --name=4kwrite --ioengine=libaio --directory=dir1 --filename=testfile --blocksize=4k --readwrite=write --filesize=10G --end_fsync=1 --numjobs=4 --iodepth=128 --direct=1 --group_reporting
$ lsof | egrep "dir1"
fio 28884 root 3u REG 253,2 10737418240 23782491 /dir1/testfile
fio 28885 root 3u REG 253,2 10737418240 23782491 /dir1/testfile
fio 28886 root 3u REG 253,2 10737418240 23782491 /dir1/testfile
fio 28887 root 3u REG 253,2 10737418240 23782491 /dir1/testfile
Run fio on multiple files under different directories
In this example, there are 2 jobs to write 8 files under different directories.
1
$ fio --name=fiojob --ioengine=libaio --filename=/fiodata1/testfile:/fiodata2/testfile:/fiodata3/testfile:/fiodata4/testfile:/fiodata5/testfile:/fiodata6/testfile:/fiodata7/testfile:/fiodata8/testfile --blocksize=4k --readwrite=write --filesize=5G --end_fsync=1 --numjobs=2 --iodepth=512 --direct=1 --group_reporting --output fio.out
Comments powered by Disqus.