Vdbench file system operations
Specify file system operations in FWD and RD
In the File system Workload Definition(FWD), we can specify a single file sysetm operation that must be executed for the workload, with the parameter ‘operation=’. The valid operation can be mkdir,rmdir,create,delete,open,close,read,write,getattr and setattr. We can specify ‘fwd=xxx,rdpct=nn’ to allow for mixed read and write operations against the same file.
If one or more operations need to be specified, the parameter ‘fwd=xxx,operations=’ in Run Definition(RD) can be leveraged to override the single operation in FWD. For example, the parameter ‘operations=mkdir’ or ‘operations=(read,getattr)’ can be specified in RD. Vdbench will fail the operation if the file structure is not created before it is accessed.
In the following example, we will run vdbench to make directories, create files, getattr, delete files and remove directories.
Make directories
Prepare the job file to make the directories:
$ cat jobfile/vdb_mkdir.job
hd=default,vdbench=/home/tester/vdbench_test,shell=ssh,user=root
hd=host1,jvms=1,system=192.168.1.2
fsd=fsd1,anchor=/mnt/bench1,depth=5,width=5,files=2,size=4k
fwd=fwd1,fsd=fsd1,host=host1,operation=mkdir
rd=rd1,fwd=fwd1,format=no,fwdrate=max
Run vdbench job to make the directories:
$ ./vdbench -f jobfile/vdb_mkdir.job
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Vdbench distribution: vdbench50407 Tue June 05 9:49:29 MDT 2018
For documentation, see 'vdbench.pdf'.
02:06:05.755 input argument scanned: '-fjobfile/vdb_mkdir.job'
02:06:05.833 Anchor size: anchor=/mnt/bench1: dirs: 3,905; files: 6,250; bytes: 24.414m (25,600,000)
02:06:06.097 Starting slave: ssh 192.168.1.2 -l root /home/tester/vdbench_test/vdbench SlaveJvm -m 192.168.1.3 -n 192.168.1.2-10-220316-02.06.05.716 -l host1-0 -p 5570
02:06:06.527 All slaves are now connected
02:06:08.002 Starting RD=rd1; elapsed=30; fwdrate=max. For loops: None
Mar 16, 2022 ..Interval.. .ReqstdOps... ...cpu%... read ....read..... ....write.... ..mb/sec... mb/sec .xfer.. ...mkdir.... ....open.... ...close....
rate resp total sys pct rate resp rate resp read write total size rate resp rate resp rate resp
02:06:09.065 1 3905.0 0.018 3.7 0.41 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 3905 0.018 0.0 0.000 0.0 0.000
02:06:09.102 avg_2-1 NaN 0.000 NaN NaN 0.0 NaN 0.000 NaN 0.000 NaN NaN NaN 0 NaN 0.000 NaN 0.000 NaN 0.000
02:06:09.102 std_2-1
02:06:09.103 max_2-1
02:06:09.216
02:06:09.216 Miscellaneous statistics:
02:06:09.216 (These statistics do not include activity between the last reported interval and shutdown.)
02:06:09.216 DIRECTORY_CREATES Directories created: 3,905 3,905/sec
02:06:09.216
02:06:09.932 Vdbench execution completed successfully. Output directory: /home/tester/vdbench_test/output
$ ls -la /mnt/bench1/vdb.1_1.dir/vdb.2_1.dir/vdb.3_1.dir/vdb.4_1.dir/vdb.5_1.dir/
total 8
drwxr-xr-x 2 root root 4096 Mar 16 02:06 .
drwxr-xr-x 7 root root 4096 Mar 16 02:06 ..
Create files in the existing directories
Prepare the job file to create files in the existing directories:
$ cat jobfile/vdb_create.job
hd=default,vdbench=/home/tester/vdbench_test,shell=ssh,user=root
hd=host1,jvms=1,system=192.168.1.2
fsd=fsd1,anchor=/mnt/bench1,depth=5,width=5,files=2,size=4k
fwd=fwd1,fsd=fsd1,host=host1,operation=create
rd=rd1,fwd=fwd1,format=restart,fwdrate=max
Note that we specify the parameter ‘format=restart’. It will create files that have not been created and will also expand files that have not reached the proper file size.
Run vdbench job to create files:
$ ./vdbench -f jobfile/vdb_create.job
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Vdbench distribution: vdbench50407 Tue June 05 9:49:29 MDT 2018
For documentation, see 'vdbench.pdf'.
02:06:11.387 input argument scanned: '-fjobfile/vdb_create.job'
02:06:11.468 Anchor size: anchor=/mnt/bench1: dirs: 3,905; files: 6,250; bytes: 24.414m (25,600,000)
02:06:11.719 Starting slave: ssh 192.168.1.2 -l root /home/tester/vdbench_test/vdbench SlaveJvm -m 192.168.1.3 -n 192.168.1.2-10-220316-02.06.11.352 -l host1-0 -p 5570
02:06:12.148 All slaves are now connected
02:06:13.002 Starting RD=format_for_rd1
02:06:13.238 host1-0: anchor=/mnt/bench1 mkdir complete.
02:06:13.418 host1-0: anchor=/mnt/bench1 create complete.
Mar 16, 2022 ..Interval.. .ReqstdOps... ...cpu%... read ....read..... ....write.... ..mb/sec... mb/sec .xfer.. ...mkdir.... ...rmdir.... ...create... ....open.... ...close.... ...delete...
rate resp total sys pct rate resp rate resp read write total size rate resp rate resp rate resp rate resp rate resp rate resp
02:06:14.063 1 6250.0 0.019 12.7 0.99 0.0 0.0 0.000 6250.0 0.019 0.00 24.41 24.41 4096 0.0 0.000 0.0 0.000 6250 0.219 6250 0.071 6250 0.006 0.0 0.000
02:06:14.103 avg_2-1 NaN 0.000 NaN NaN 0.0 NaN 0.000 NaN 0.000 NaN NaN NaN 0 NaN 0.000 NaN 0.000 NaN 0.000 NaN 0.000 NaN 0.000 NaN 0.000
02:06:14.103 std_2-1
02:06:14.103 max_2-1
02:06:14.217
02:06:14.217 Miscellaneous statistics:
02:06:14.217 (These statistics do not include activity between the last reported interval and shutdown.)
02:06:14.218 FILE_CREATES Files created: 6,250 6,250/sec
02:06:14.218 WRITE_OPENS Files opened for write activity: 6,250 6,250/sec
02:06:14.218 DIR_EXISTS Directory may not exist (yet): 7 7/sec
02:06:14.218 FILE_CLOSES Close requests: 6,250 6,250/sec
02:06:14.218
02:06:15.001 Starting RD=rd1; elapsed=30; fwdrate=max. For loops: None
02:06:15.344
02:06:15.344 Message from slave host1-0:
02:06:15.345 Anchor: /mnt/bench1
02:06:15.345 Vdbench is trying to create a new file, but all files already exist,
02:06:15.345 and no threads are currently active deleting files
02:06:15.345 FwgThread.canWeGetMoreFiles(): Shutting down threads for operation=create
02:06:15.345
Mar 16, 2022 ..Interval.. .ReqstdOps... ...cpu%... read ....read..... ....write.... ..mb/sec... mb/sec .xfer.. ...mkdir.... ...rmdir.... ...create... ....open.... ...close.... ...delete...
rate resp total sys pct rate resp rate resp read write total size rate resp rate resp rate resp rate resp rate resp rate resp
02:06:16.021 1 0.0 0.000 1.8 0.09 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 0.0 0.000 0.0 0.000 0.0 0.000 0.0 0.000
02:06:16.030 avg_2-1 NaN 0.000 NaN NaN 0.0 NaN 0.000 NaN 0.000 NaN NaN NaN 0 NaN 0.000 NaN 0.000 NaN 0.000 NaN 0.000 NaN 0.000 NaN 0.000
02:06:16.030 std_2-1
02:06:16.030 max_2-1
02:06:16.135 Miscellaneous statistics: All counters are zero
02:06:16.602 Vdbench execution completed successfully. Output directory: /home/tester/vdbench_test/output
$ ls -la /mnt/bench1/vdb.1_1.dir/vdb.2_1.dir/vdb.3_1.dir/vdb.4_1.dir/vdb.5_1.dir/
total 16
drwxr-xr-x 2 root root 4096 Mar 16 02:06 .
drwxr-xr-x 7 root root 4096 Mar 16 02:06 ..
-rw-r--r-- 1 root root 4096 Mar 16 02:06 vdb_f0000.file
-rw-r--r-- 1 root root 4096 Mar 16 02:06 vdb_f0001.file
Getattr
Prepare the job file to getattr:
$ cat jobfile/vdb_getattr.job
hd=default,vdbench=/home/tester/vdbench_test,shell=ssh,user=root
hd=host1,jvms=1,system=192.168.1.2
fsd=fsd1,anchor=/mnt/bench1,depth=5,width=5,files=2,size=4k
fwd=fwd1,fsd=fsd1,host=host1,operation=getattr
rd=rd1,fwd=fwd1,format=no,fwdrate=max
Run vdbench job to getattr:
$ ./vdbench -f jobfile/vdb_getattr.job
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Vdbench distribution: vdbench50407 Tue June 05 9:49:29 MDT 2018
For documentation, see 'vdbench.pdf'.
02:06:18.059 input argument scanned: '-fjobfile/vdb_getattr.job'
02:06:18.145 Anchor size: anchor=/mnt/bench1: dirs: 3,905; files: 6,250; bytes: 24.414m (25,600,000)
02:06:18.408 Starting slave: ssh 192.168.1.2 -l root /home/tester/vdbench_test/vdbench SlaveJvm -m 192.168.1.3 -n 192.168.1.2-10-220316-02.06.18.020 -l host1-0 -p 5570
02:06:18.778 All slaves are now connected
02:06:20.002 Starting RD=rd1; elapsed=30; fwdrate=max. For loops: None
Mar 16, 2022 ..Interval.. .ReqstdOps... ...cpu%... read ....read..... ....write.... ..mb/sec... mb/sec .xfer.. ....open.... ...close.... ..getattr...
rate resp total sys pct rate resp rate resp read write total size rate resp rate resp rate resp
02:06:21.062 1 39541 0.015 8.1 0.84 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 39541 0.015
02:06:22.032 2 77608 0.010 5.3 1.47 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 77608 0.010
02:06:23.016 3 83661 0.009 4.5 1.26 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 83661 0.009
02:06:24.014 4 83054 0.009 4.4 1.35 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 83054 0.009
02:06:25.014 5 83753 0.009 4.3 1.26 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 83753 0.009
02:06:26.012 6 83907 0.009 4.3 1.23 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 83907 0.009
02:06:27.012 7 83520 0.009 4.3 1.26 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 83520 0.009
02:06:28.011 8 82677 0.009 4.4 1.19 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82677 0.009
02:06:29.016 9 83093 0.009 4.6 1.19 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 83093 0.009
02:06:30.012 10 84090 0.009 4.5 1.23 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 84090 0.009
02:06:31.011 11 83013 0.009 4.2 1.16 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 83013 0.009
02:06:32.013 12 83158 0.010 4.3 1.38 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 83158 0.010
02:06:33.010 13 74121 0.011 4.3 1.19 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 74121 0.011
02:06:34.009 14 77405 0.010 4.4 1.35 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 77405 0.010
02:06:35.011 15 82874 0.010 4.6 1.28 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82874 0.010
02:06:36.011 16 82855 0.010 4.6 1.34 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82855 0.010
02:06:37.010 17 82508 0.010 4.6 1.38 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82508 0.010
02:06:38.007 18 82766 0.010 4.6 1.38 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82766 0.010
02:06:39.009 19 82048 0.010 4.3 1.22 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82048 0.010
02:06:40.010 20 81357 0.010 4.4 1.25 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 81357 0.010
02:06:41.008 21 82723 0.010 4.4 1.28 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82723 0.010
02:06:42.007 22 82702 0.010 4.5 1.50 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82702 0.010
02:06:43.008 23 82793 0.010 4.3 1.41 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82793 0.010
02:06:44.009 24 82465 0.010 4.4 1.28 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82465 0.010
02:06:45.009 25 82695 0.010 4.4 1.32 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82695 0.010
02:06:46.008 26 82119 0.010 4.6 1.47 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82119 0.010
02:06:47.008 27 82297 0.010 4.8 1.72 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82297 0.010
02:06:48.007 28 82508 0.010 4.4 1.35 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82508 0.010
02:06:49.008 29 82545 0.010 4.4 1.41 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82545 0.010
02:06:50.007 30 82641 0.010 4.7 1.44 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82641 0.010
02:06:50.015 avg_2-30 82170 0.010 4.5 1.33 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 82170 0.010
02:06:50.015 std_2-30 2147.1 0.018 2147 0.018
02:06:50.016 max_2-30 84090 5.854 84090 5.854
02:06:50.234
02:06:50.234 Miscellaneous statistics:
02:06:50.234 (These statistics do not include activity between the last reported interval and shutdown.)
02:06:50.234 GET_ATTR Getattr requests: 2,422,766 80,758/sec
02:06:50.234
02:06:51.201 Vdbench execution completed successfully. Output directory: /home/tester/vdbench_test/output
$ ls -la /mnt/bench1/vdb.1_1.dir/vdb.2_1.dir/vdb.3_1.dir/vdb.4_1.dir/vdb.5_1.dir/
total 16
drwxr-xr-x 2 root root 4096 Mar 16 02:06 .
drwxr-xr-x 7 root root 4096 Mar 16 02:06 ..
-rw-r--r-- 1 root root 4096 Mar 16 02:06 vdb_f0000.file
-rw-r--r-- 1 root root 4096 Mar 16 02:06 vdb_f0001.file
Delete files
Prepare the job file to delete files:
$ cat jobfile/vdb_delete.job
hd=default,vdbench=/home/tester/vdbench_test,shell=ssh,user=root
hd=host1,jvms=1,system=192.168.1.2
fsd=fsd1,anchor=/mnt/bench1,depth=5,width=5,files=2,size=4k
fwd=fwd1,fsd=fsd1,host=host1,operation=delete
rd=rd1,fwd=fwd1,format=no,fwdrate=max
Run vdbench job to delete files:
$ ./vdbench -f jobfile/vdb_delete.job
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Vdbench distribution: vdbench50407 Tue June 05 9:49:29 MDT 2018
For documentation, see 'vdbench.pdf'.
02:06:52.656 input argument scanned: '-fjobfile/vdb_delete.job'
02:06:52.757 Anchor size: anchor=/mnt/bench1: dirs: 3,905; files: 6,250; bytes: 24.414m (25,600,000)
02:06:53.060 Starting slave: ssh 192.168.1.2 -l root /home/tester/vdbench_test/vdbench SlaveJvm -m 192.168.1.3 -n 192.168.1.2-10-220316-02.06.52.608 -l host1-0 -p 5570
02:06:53.542 All slaves are now connected
02:06:55.002 Starting RD=rd1; elapsed=30; fwdrate=max. For loops: None
02:06:55.403
02:06:55.403 Message from slave host1-0:
02:06:55.403 Anchor: /mnt/bench1
02:06:55.403 Vdbench is trying to delete a file, but no files are available, and no
02:06:55.404 threads are currently active creating new files.
02:06:55.404 FwgThread.canWeGetMoreFiles(): Shutting down threads for operation=delete
02:06:55.404
Mar 16, 2022 ..Interval.. .ReqstdOps... ...cpu%... read ....read..... ....write.... ..mb/sec... mb/sec .xfer.. ....open.... ...close.... ...delete...
rate resp total sys pct rate resp rate resp read write total size rate resp rate resp rate resp
02:06:56.061 1 6250.0 0.044 7.3 0.61 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 0.0 0.000 0.0 0.000 6250 0.044
02:06:56.096 avg_2-1 NaN 0.000 NaN NaN 0.0 NaN 0.000 NaN 0.000 NaN NaN NaN 0 NaN 0.000 NaN 0.000 NaN 0.000
02:06:56.096 std_2-1
02:06:56.097 max_2-1
02:06:56.210
02:06:56.210 Miscellaneous statistics:
02:06:56.210 (These statistics do not include activity between the last reported interval and shutdown.)
02:06:56.210 FILE_DELETES Files deleted: 6,250 6,250/sec
02:06:56.211 FILE_MUST_EXIST File does not exist (yet): 1 1/sec
02:06:56.211
02:06:56.949 Vdbench execution completed successfully. Output directory: /home/tester/vdbench_test/output
$ ls -la /mnt/bench1/vdb.1_1.dir/vdb.2_1.dir/vdb.3_1.dir/vdb.4_1.dir/vdb.5_1.dir/
total 8
drwxr-xr-x 2 root root 4096 Mar 16 02:06 .
drwxr-xr-x 7 root root 4096 Mar 16 02:06 ..
Remove directories
Prepare the job file to remove directories:
$ cat jobfile/vdb_rmdir.job
hd=default,vdbench=/home/tester/vdbench_test,shell=ssh,user=root
hd=host1,jvms=1,system=192.168.1.2
fsd=fsd1,anchor=/mnt/bench1,depth=5,width=5,files=2,size=4k
fwd=fwd1,fsd=fsd1,host=host1,operation=rmdir
rd=rd1,fwd=fwd1,format=no,fwdrate=max
Run vdbench job to remove directories:
$ ./vdbench -f jobfile/vdb_rmdir.job
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Vdbench distribution: vdbench50407 Tue June 05 9:49:29 MDT 2018
For documentation, see 'vdbench.pdf'.
02:06:58.406 input argument scanned: '-fjobfile/vdb_rmdir.job'
02:06:58.484 Anchor size: anchor=/mnt/bench1: dirs: 3,905; files: 6,250; bytes: 24.414m (25,600,000)
02:06:58.736 Starting slave: ssh 192.168.1.2 -l root /home/tester/vdbench_test/vdbench SlaveJvm -m 192.168.1.3 -n 192.168.1.2-10-220316-02.06.58.369 -l host1-0 -p 5570
02:06:59.158 All slaves are now connected
02:07:00.002 Starting RD=rd1; elapsed=30; fwdrate=max. For loops: None
Mar 16, 2022 ..Interval.. .ReqstdOps... ...cpu%... read ....read..... ....write.... ..mb/sec... mb/sec .xfer.. ...rmdir.... ....open.... ...close....
rate resp total sys pct rate resp rate resp read write total size rate resp rate resp rate resp
02:07:01.051 1 3905.0 0.039 5.4 0.65 0.0 0.0 0.000 0.0 0.000 0.00 0.00 0.00 0 3905 0.039 0.0 0.000 0.0 0.000
02:07:01.089 avg_2-1 NaN 0.000 NaN NaN 0.0 NaN 0.000 NaN 0.000 NaN NaN NaN 0 NaN 0.000 NaN 0.000 NaN 0.000
02:07:01.089 std_2-1
02:07:01.090 max_2-1
02:07:01.204
02:07:01.205 Miscellaneous statistics:
02:07:01.205 (These statistics do not include activity between the last reported interval and shutdown.)
02:07:01.205 DIRECTORY_DELETES Directories deleted: 3,905 3,905/sec
02:07:01.205
02:07:01.550 Vdbench execution completed successfully. Output directory: /home/tester/vdbench_test/output
$ ls -la /mnt/bench1/vdb.1_1.dir/vdb.2_1.dir/vdb.3_1.dir/vdb.4_1.dir/vdb.5_1.dir/
ls: cannot access /mnt/bench1/vdb.1_1.dir/vdb.2_1.dir/vdb.3_1.dir/vdb.4_1.dir/vdb.5_1.dir/: No such file or directory