fio is a flexible I/O tester which generates I/O and measure I/O performance on the target storage system. In the case we want to run the fio workload on the cloud deployments, we can containerize fio. Also we can encapsulate necessary packages in the docker image so that it can be easily deployed to avoid package dependency.
There are ready-to-use fio docker image online if you search with google. In this article, we discuss how to create a docker image which consumes a python script to run fio workload.
Docker can build images automatically by reading the instructions from a Dockerfile. A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using docker build users can create an automated build that executes several command-line instructions in succession.
The following is Dockerfile we are going to use to build the docker image.
$ cat Dockerfile FROM python:3-alpine RUN apk add --no-cache \ fio==3.28-r1 \ sudo \ lsblk \ util-linux \ procps COPY perfbench/perfbench.py / COPY perfbench/run.sh / ENTRYPOINT [ "/run.sh" ]
We use Alpine Linux which is a security-oriented, lightweight Linux distribution based on musl libc and busybox. Since we need python support, we leverge the official python:3-alpine image which is based on Alpine Linux.
We install the latest supported fio-3.28 to the docker image. And we install packages like sudo, lsblk, util-linux and procps which are needed by the python script. We copy the python script and wrapper shell script to the root directory. The run.sh script will be run once the container is started in order to run fio benchmark.
The following is the run.sh script.
$ cat perfbench/run.sh #!/bin/sh [ -z "$FIO_DATA_DIR" ] && echo "FIO_DATA_DIR variable is required." && exit 1; [ -z "$FIO_LOG_DIR" ] && echo "FIO_LOG_DIR variable is required." && exit 1; [ ! -d "$FIO_DATA_DIR" ] && echo "The data directory $FIO_DATA_DIR does not exist." && exit 1; [ ! -d "$FIO_LOG_DIR" ] && echo "The result directory $FIO_LOG_DIR does not exit." && exit 1; echo "Running fio benchmark on directory $FIO_DATA_DIR" python perfbench.py --dir $FIO_DATA_DIR --logdir $FIO_LOG_DIR
Now, we can build the docker image with the Dockerfile.
$ docker build -t perfbench . Sending build context to Docker daemon 19.46kB Step 1/5 : FROM python:3-alpine Step 2/5 : RUN apk add --no-cache fio==3.28-r1 sudo lsblk util-linux procps Step 3/5 : COPY perfbench/perfbench.py / Step 4/5 : COPY perfbench/run.sh / Step 5/5 : ENTRYPOINT /run.sh Successfully built 9c0957911607 Successfully tagged perfbench:latest $ docker image list REPOSITORY TAG IMAGE ID CREATED SIZE perfbench latest f63e13d57991 32 minutes ago 60MB python 3-alpine c7100ae3ac4d 2 weeks ago 48.7MB
Use the following command to run fio benchmark. Note that we have defined the fio benchmark logic in the customized python script consumed by the container.
$ docker run --rm --privileged -v /data:/data -e FIO_DATA_DIR=/data -e FIO_LOG_DIR=/data/result perfbench
Note that the option “–privileged” is to allow the python script in the container to drop cache in this case. The same purpose can also be approached with the following method. Then we can do “echo 3 > drop_caches” in the container to drop cache.
$ docker run --rm -v /proc/sys/vm/drop_caches:/drop_caches -v /data:/data -e FIO_DATA_DIR=/data -e FIO_RESULT_DIR=/data/result perfbench