Docker-based fio benchmarking - Part One

Posted on Edited on In , Views:

Why Docker-based fio benchmarking

fio is a flexible I/O tester which generates I/O and measure I/O performance on the target storage system. In the case we want to run the fio workload on the cloud deployments, we can containerize fio. Also we can encapsulate necessary packages in the docker image so that it can be easily deployed to avoid package dependency.

There are ready-to-use fio docker image online if you search with google. In this article, we discuss how to create a docker image which consumes a python script to run fio workload.

Build docker image with Dockerfile

Docker can build images automatically by reading the instructions from a Dockerfile. A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using docker build users can create an automated build that executes several command-line instructions in succession.

The following is Dockerfile we are going to use to build the docker image.

$ cat Dockerfile
FROM python:3-alpine

RUN apk add --no-cache \
    fio==3.28-r1 \
    sudo \
    lsblk \
    util-linux \
    procps

COPY perfbench/perfbench.py /
COPY perfbench/run.sh /

ENTRYPOINT [ "/run.sh" ]

We use Alpine Linux which is a security-oriented, lightweight Linux distribution based on musl libc and busybox. Since we need python support, we leverge the official python:3-alpine image which is based on Alpine Linux.

We install the latest supported fio-3.28 to the docker image. And we install packages like sudo, lsblk, util-linux and procps which are needed by the python script. We copy the python script and wrapper shell script to the root directory. The run.sh script will be run once the container is started in order to run fio benchmark.

The following is the run.sh script.

$ cat perfbench/run.sh
#!/bin/sh

[ -z "$FIO_DATA_DIR" ] && echo "FIO_DATA_DIR variable is required." && exit 1;
[ -z "$FIO_LOG_DIR" ] && echo "FIO_LOG_DIR variable is required." && exit 1;
[ ! -d "$FIO_DATA_DIR" ] && echo "The data directory $FIO_DATA_DIR does not exist." && exit 1;
[ ! -d "$FIO_LOG_DIR" ] && echo "The result directory $FIO_LOG_DIR does not exit." && exit 1;
echo "Running fio benchmark on directory $FIO_DATA_DIR"
python perfbench.py --dir $FIO_DATA_DIR --logdir $FIO_LOG_DIR

Now, we can build the docker image with the Dockerfile.

$ docker build -t perfbench .
Sending build context to Docker daemon  19.46kB
Step 1/5 : FROM python:3-alpine
Step 2/5 : RUN apk add --no-cache     fio==3.28-r1     sudo     lsblk     util-linux     procps
Step 3/5 : COPY perfbench/perfbench.py /
Step 4/5 : COPY perfbench/run.sh /
Step 5/5 : ENTRYPOINT /run.sh
Successfully built 9c0957911607
Successfully tagged perfbench:latest

$ docker image list
REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
perfbench                            latest              f63e13d57991        32 minutes ago      60MB
python                               3-alpine            c7100ae3ac4d        2 weeks ago         48.7MB

Run fio benchmark with docker

Use the following command to run fio benchmark. Note that we have defined the fio benchmark logic in the customized python script consumed by the container.

$ docker run --rm --privileged -v /data:/data -e FIO_DATA_DIR=/data -e FIO_LOG_DIR=/data/result perfbench

Note that the option “–privileged” is to allow the python script in the container to drop cache in this case. The same purpose can also be approached with the following method. Then we can do “echo 3 > drop_caches” in the container to drop cache.

$ docker run --rm -v /proc/sys/vm/drop_caches:/drop_caches -v /data:/data -e FIO_DATA_DIR=/data -e FIO_RESULT_DIR=/data/result perfbench

Reference