Home
FlamingBytes
Cancel

Scale a system to support millions of users

Start from single server Building a large scale system is not one time effort, it should be an iterative process as the user workload increases. The journey could just start with single server and...

Set in Golang

Go does not have set by default but there is way to imitate it in Go. The following is an example code piece to create “set” in Go by setting a map with empty values. $ cat test.go package main ...

Must-read books and articles for Software and Performance Engineering

Books The Art of Computer Programming Understanding the Linux Kernel, 3rd Edition Linux Device Drivers, 3rd Edition Designing Data-Intensive Appli...

Vdbench file system operations

Specify file system operations in FWD and RD In the File system Workload Definition(FWD), we can specify a single file sysetm operation that must be executed for the workload, with the parameter ‘...

Proc filesystem - diskstats

The /proc/diskstats file displays the I/O statistics of block devices. Here we have a system which has one disk sda used by Linux operating system and two disks sdb sdc for other purpose. $ cat /...

dstat - A replacement for vmstat, iostat and ifstat

Introduction dstat is a versatile tool for generating system resource statistics. It can be a replacement for vmstat, iostat and ifstat. It overcomes some of the limitations and adds some extra fe...

Pull an image from a private docker registry in Kubernetes Pod

Log in to Docker Hub In order to pull a image from Docker Hub, you must authenticate with a registry. Use docker tool to log in to the Docker Hub as below. A username and password is needed to log...

Keep a docker container running and not exiting

In the kubernetes environment, we can keep a container(pod) alive and avoid it exits immediately after starting. Method one Use a long-time-run command in Dockerfile CMD ["sh", "-c",...

Deploy application with kubernetes statefulset

StatefulSet is the workload API object used to manage stateful applications. Like a Deployment, a StatefulSet manages Pods that are based on an identical container spec. Unlike a Deployment, a Sta...

Docker-based fio benchmarking - Part Two

In this article, we will build a docker image which is based on python versioned alpine Linux. Alpine Linux is much smaller than most distribution base images (~5MB), and thus leads to much slimmer...

Docker-based fio benchmarking - Part One

Why Docker-based fio benchmarking fio is a flexible I/O tester which generates I/O and measure I/O performance on the target storage system. In the case we want to run the fio workload on the clou...

Concepts you should know

Data structure Bloom filter A Bloom filter is a space-efficient probabilistic data structure that is used to test whether an element is a member of a set. For example, checking availability of us...

Storage concepts you should know

Object Storage Object storage, also known as object-based storage, is a strategy that manages and manipulates data storage as distinct units, called objects. These objects are kept in a single sto...

Network concepts you should know

IP address Networking is about one computer sending a message to another computer. This message is called packet in the IP world. It is just like a postcard in the postal service. The postal servi...

Database concepts you should know

SQL vs. NoSQL Source Benefits of NoSQL databases NoSQL databases offer many benefits over relational databases. NoSQL databases have flexible data models, scale horizontally, have incredibly f...

Distributed system concepts you should know

This post contains basic explanations for concepts you should know related to distributed system. API and REST API Application Programming Interface, abbreviated as API, enables connection betwee...

Estimate the scale of the system

It is always a good idea to estimate the scale of the system as it help reflect if the designed system could fulfill the functional requirements. The requirements might include: Number of users...

What is high availability

What is high availability? High availability refers to a system or component that is operational without interruption for long periods of time. High availability is measured as a percentage, with...

Latency numbers

Latency Comparison Numbers (~2012) Conclusion Disk is much slower than the memory Avoid the disk seek if possible Compressing data is worth to consider before sending over network It t...

Data volume units

Data volumes The volume of data in a single file or file system can be described by a unit called a byte. However, data volumes can become very large when dealing with Earth satellite data. Below ...