How to run fio on multiple files

Posted by Iceberg on October 13, 2021

fio directory and filename option

  • directory=str

    Prefix filenames with this directory. Used to place files in a different location than ./. You can specify a number of directories by separating the names with a ‘:’ character. These directories will be assigned equally distributed to job clones created by numjobs as long as they are using generated filenames. If specific filename(s) are set fio will use the first listed directory, and thereby matching the filename semantic (which generates a file for each clone if not specified, but lets all clones use the same file if set).

  • filename=str

    Fio normally makes up a filename based on the job name, thread number, and file number (see filename_format). If you want to share files between threads in a job or several jobs with fixed file paths, specify a filename for each of them to override the default. If the ioengine is file based, you can specify a number of files by separating the names with a ‘:’ colon. So if you wanted a job to open /dev/sda and /dev/sdb as the two working files, you would use filename=/dev/sda:/dev/sdb. This also means that whenever this option is specified, nrfiles is ignored. The size of regular files specified by this option will be size divided by number of files unless an explicit size is specified by filesize.

Run fio on single directory

In this experiment, we run 64 fio jobs against single directory. 64 files will be laid out under the target directory during fio test. Each job writes one file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ fio --blocksize=4k --directory=/mnt/nfsmnt1 --ioengine=libaio --readwrite=write --size=50M --name=test --numjobs=64 --group_reporting --direct=1 --iodepth=128 --end_fsync=1 &

$ ps -ef | grep fio | grep nfsmnt1 | wc -l
65

$ ls -ltr /mnt/nfsmnt1/ | grep test | wc -l
64

$ ls -ltr /mnt/nfsmnt1/ | sort -n -k9
-rw-r--r-- 1 root root 52428800 Oct 15 00:30 test.0.0
<omitted...>
-rw-r--r-- 1 root root 52428800 Oct 15 00:30 test.63.0

$ lsof | grep nfs | grep test
fio        9468          root    4u      REG               0,57   8138752         11 /mnt/nfsmnt1/test.0.0 (host1:/nfsshare)
<omitted...>
fio        9468          root    4u      REG               0,57   8138752         11 /mnt/nfsmnt1/test.63.0 (host1:/nfsshare)

Run fio on multiple directories

In this experiment, we run 64 fio jobs against 64 directories. One file will be laid out under each of the 64 directories during fio test. Each job writes one file.

1
2
3
4
5
6
7
8
9
10
11
12
13
$ fio --blocksize=4k --directory=/mnt/nfsmnt1:/mnt/nfsmnt2:/mnt/nfsmnt3:/mnt/nfsmnt4:/mnt/nfsmnt5:/mnt/nfsmnt6:/mnt/nfsmnt7:/mnt/nfsmnt8:/mnt/nfsmnt9:/mnt/nfsmnt10:/mnt/nfsmnt11:/mnt/nfsmnt12:/mnt/nfsmnt13:/mnt/nfsmnt14:/mnt/nfsmnt15:/mnt/nfsmnt16:/mnt/nfsmnt17:/mnt/nfsmnt18:/mnt/nfsmnt19:/mnt/nfsmnt20:/mnt/nfsmnt21:/mnt/nfsmnt22:/mnt/nfsmnt23:/mnt/nfsmnt24:/mnt/nfsmnt25:/mnt/nfsmnt26:/mnt/nfsmnt27:/mnt/nfsmnt28:/mnt/nfsmnt29:/mnt/nfsmnt30:/mnt/nfsmnt31:/mnt/nfsmnt32:/mnt/nfsmnt33:/mnt/nfsmnt34:/mnt/nfsmnt35:/mnt/nfsmnt36:/mnt/nfsmnt37:/mnt/nfsmnt38:/mnt/nfsmnt39:/mnt/nfsmnt40:/mnt/nfsmnt41:/mnt/nfsmnt42:/mnt/nfsmnt43:/mnt/nfsmnt44:/mnt/nfsmnt45:/mnt/nfsmnt46:/mnt/nfsmnt47:/mnt/nfsmnt48:/mnt/nfsmnt49:/mnt/nfsmnt50:/mnt/nfsmnt51:/mnt/nfsmnt52:/mnt/nfsmnt53:/mnt/nfsmnt54:/mnt/nfsmnt55:/mnt/nfsmnt56:/mnt/nfsmnt57:/mnt/nfsmnt58:/mnt/nfsmnt59:/mnt/nfsmnt60:/mnt/nfsmnt61:/mnt/nfsmnt62:/mnt/nfsmnt63:/mnt/nfsmnt64 --ioengine=libaio --readwrite=write --size=50M --name=test --numjobs=64 --group_reporting --direct=1 --iodepth=128 --end_fsync=1 &

$ ps -ef | grep fio | grep nfsmnt | wc -l
65

$ mount | grep nfsmnt | wc -l
64

$ ls -la /mnt/nfsmnt63
total 51208
drwxrwxrwx   2 root root     4096 Oct 14 22:14 .
drwxr-xr-x. 66 root root     4096 Oct 14 22:14 ..
-rw-r--r--   1 root root 52428800 Oct 14 22:17 test.63.0

Run multiple fio jobs on single file

In this experiment, we run 64 jobs against single file. The filename option is used to provided a fixed file name which will be shared by all the jobs.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ fio --blocksize=4k --filename=/mnt/nfsmnt1/testfile --ioengine=libaio --readwrite=write --size=50M --name=test --numjobs=64 --group_reporting --direct=1 --iodepth=128 --end_fsync=1

$ ps -ef | grep fio | grep testfile | wc -l
65

$ ls -ltr /mnt/nfsmnt1/
total 4
-rw-r--r-- 1 root root 2793472 Oct 15 01:05 testfile

$ lsof | grep fio | grep testfile
fio       18268          root    4u      REG               0,57    602112         11 /mnt/nfsmnt1/testfile (host1:/nfsshare)
fio       18268          root    5u      REG               0,57    606208         11 /mnt/nfsmnt1/testfile (host1:/nfsshare)
fio       18269          root    4u      REG               0,57    610304         11 /mnt/nfsmnt1/testfile (host1:/nfsshare)
fio       18270          root    4u      REG               0,57    614400         11 /mnt/nfsmnt1/testfile (host1:/nfsshare)
fio       18271          root    4u      REG               0,57    618496         11 /mnt/nfsmnt1/testfile (host1:/nfsshare)
<omitted...>

Reference