FlamingBytes
  • Home
  • Tags
  • Archive
  • About
Surprised! How can the write IOPS be 10x faster than the cloud native disk performance

Surprised! How can the write IOPS be 10x faster than the cloud native disk performance

In a competitive fio write performance benchmark between 3rd party storage solution and cloud native disk, we noticed that the write performance for the 3rd party storage solution is 10x faster than the cloud native. This usually seems impossible since we would argue that no one can beat the raw
relentlesstorm Jan 25, 2023
fio direct I/O error with 1k blocksize

fio direct I/O error with 1k blocksize

When to run fio write with small blocksize(e.g. 1k), the following error is seen. $ fio --blocksize=1k --ioengine=libaio --readwrite=randwrite --filesize=2G --group_reporting --direct=1 --iodepth=128 --randrepeat=1 --end_fsync=1 -- name=job1 --numjobs=1 --filename=/mnt/fiomnt/fio.dat job1: (g=0): rw=
relentlesstorm Jan 23, 2023
Tracking stock financial metrics in tradingview

Tracking stock financial metrics in tradingview

In this post, we learn how to use tradingview.com to track financial metrics of the interested stock. It helps understand how the financial metrics support the stock price under the hood.
relentlesstorm Jan 17, 2023
Using ANOVA in R to analyze US COVID data to understand age impact to death rate

Using ANOVA in R to analyze US COVID data to understand age impact to death rate

A great example to use ANOVA in R to analyze complicated data.
relentlesstorm Jan 12, 2023
Using R to analyze US COVID pandemic waves and peaks

Using R to analyze US COVID pandemic waves and peaks

Goal According to weekly data between 2020 to 2022, we want to get to know the waves and peaks of COVID pandemic in these years. Download the data We will continue to use NCHS(National Center for Health Statistics) as our data source. Visit https://data.cdc.gov/browse?category=
relentlesstorm Jan 11, 2023
Using R to analyze the quarterly US COVID data

Using R to analyze the quarterly US COVID data

An R example to analyze big amount of data
relentlesstorm Jan 10, 2023
Getting started with R

Getting started with R

Installation The simplest way is to use Homebrew: $ brew install r Another way is to download installation package from https://cloud.r-project.org/ "Hello world" of R Run it from R console Command R will start a R console, and you can run R code inside R console. (base) ➜ benchling
relentlesstorm Jan 8, 2023
How to redirect the default "404 page not found" error page in Ghost blog

How to redirect the default "404 page not found" error page in Ghost blog

When users access a non-existent page, they may see a page which shows the message "404 page not found" in Ghost blog. We can redirect the page to another page by creating a error-404.hbs file under the theme folder. In the following example, we redirect the users to the
relentlesstorm Jan 5, 2023
How to install GUI desktop on CentOS 7

How to install GUI desktop on CentOS 7

Install GNOME desktop via the groups option of yum: $ yum update $ yum -y groups install "GNOME Desktop" Inform the startx command which desktop env to run: $ echo "exec gnome-session" >> ~/.xinitrc Manually start the GUI desktop: $ startx Automaticlly start the GUI desktop after reboot: $ systemctl set-default graphical.target $ reboot
relentlesstorm Jan 4, 2023
Using signals with kill command in Linux

Using signals with kill command in Linux

Usage of signals with kill $ kill -l 1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL 5) SIGTRAP 6) SIGABRT 7) SIGBUS 8) SIGFPE 9) SIGKILL 10) SIGUSR1 11) SIGSEGV 12) SIGUSR2 13) SIGPIPE 14) SIGALRM 15) SIGTERM 16) SIGSTKFLT 17) SIGCHLD 18) SIGCONT 19) SIGSTOP 20) SIGTSTP 21) SIGTTIN 22)
relentlesstorm Jan 4, 2023
Using virtctl to access virtual machine in Kubernetes

Using virtctl to access virtual machine in Kubernetes

Install the virtctl client tool Basic VirtualMachineInstance operations can be performed with the stock kubectl utility. However, the virtctl binary utility is required to use advanced features such as: * Serial and graphical console access It also provides convenience commands for: * Starting and stopping VirtualMachineInstances * Live migrating VirtualMachineInstances * Uploading virtual machine
relentlesstorm Jan 2, 2023
Using node selector to assign virtual machines to a node

Using node selector to assign virtual machines to a node

In some circumstances, we want to control which node the virtual machine or pod deploys to. The node selector can be used to assign virtual machine or pod to a node. Add label to a node The label can be added to a node from either command line or openshift
relentlesstorm Dec 30, 2022
Deploy application with kubernetes statefulset

Deploy application with kubernetes statefulset

StatefulSet is the workload API object used to manage stateful applications. Like a Deployment, a StatefulSet manages Pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are
relentlesstorm Dec 29, 2022
Keep a docker container running and not exiting

Keep a docker container running and not exiting

In the kubernetes environment, we can keep a container(pod) alive and avoid it exits immediately after starting. Method one 1. Use a long-time-run command in Dockerfile CMD ["sh", "-c", "tail -f /dev/null"] or CMD ["sh", "-c", "sleep infinity"] 2. Build the docker image and push to docker repository
relentlesstorm Dec 28, 2022
Hosting the Ghost blog on GitHub

Hosting the Ghost blog on GitHub

Install Ghost locally GitHub Pages is a great solution for hosting static website. If you use the Ghost to manage your website, you can install Ghost locally and convert it to a static website in order to host it on Github. Install Ubuntu virtual machine on Windows(optional) Install Ghost-CLI
relentlesstorm Dec 27, 2022
Pull an image from a private docker registry in Kubernetes Pod

Pull an image from a private docker registry in Kubernetes Pod

Log in to Docker Hub In order to pull a image from Docker Hub, you must authenticate with a registry. Use docker tool to log in to the Docker Hub as below. A username and password is needed to log in. $ docker login The login process creates or updates the
relentlesstorm Dec 27, 2022
Deploy kubernetes statefulset with affinity

Deploy kubernetes statefulset with affinity

Here is an example to demonstrate how to assign statefulset and pod to the target worker node. affinity helps define which node will be assigned for the pod by using the node labels. $ kubectl get nodes -o wide --show-labels $ kubectl label nodes portworx.io/fio=true $ cat statefulset-node-select.yaml apiVersion:
relentlesstorm Dec 27, 2022
Using kubeconfig to configure access to remote Kubernetes cluster

Using kubeconfig to configure access to remote Kubernetes cluster

Install kubectl binary The Kubernetes command-line tool, kubectl, allows you to run commands against Kubernetes clusters. You can use kubectl to deploy applications, inspect and manage cluster resources, and view logs. [root@localhost ~]# curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)
relentlesstorm Dec 26, 2022
Lambda architecture in big data system

Lambda architecture in big data system

What is Lambda Architecture When working with very large data sets, it can take a long time to run the sort of queries that clients need. These queries can't be performed in real time, and often require algorithms such as MapReduce that operate in parallel across the entire data set.
relentlesstorm Dec 25, 2022
Use sysbench for CockroachDB performance benchmarking

Use sysbench for CockroachDB performance benchmarking

Intro Cockroach uses TPC-C as the official OLTP workload benchmark since it's a more realistic measurement by modeling the real world applications. However, sysbench is a straight-forward throughput/latency benchmarking tool. It is a scriptable multi-threaded benchmark tool based on LuaJIT. It is most frequently used for database benchmarks, but
relentlesstorm Dec 8, 2022
Using cgroups to limit block device bandwidth

Using cgroups to limit block device bandwidth

The block I/O controller specifies upper IO rate limits on devices. $ mount | egrep "/cgroup |/blkio" tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio) $ lscgroup | grep blkio blkio:/ In this post,
relentlesstorm Dec 5, 2022
Using cgroups to limit Memory usage

Using cgroups to limit Memory usage

The memory controller isolates the memory behaviour of a group of tasks from the rest of the system. $ mount | egrep "/cgroup |/memory" tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755) cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory) $ lssubsys
relentlesstorm Dec 4, 2022
CockroachDB performance benchmarking

CockroachDB performance benchmarking

CockroachDB key concepts * Range - CockroachDB stores all user data (tables, indexes, etc.) and almost all system data in a giant sorted map of key-value pairs. This keyspace is divided into "ranges", contiguous chunks of the keyspace, so that every key can always be found in a single range. From
relentlesstorm Dec 4, 2022
Using cgroups to limit CPU utilization

Using cgroups to limit CPU utilization

Intro to cgroups Cgroups(control groups) make it possible to allocate system resources such as CPU time, memory, disk I/O and network bandwidth, or combinations of them, among a group of tasks(processes) running on a system. The following commands output the available subsystems(resource controllers) for the cgroups.
relentlesstorm Dec 3, 2022
Deploy CockroachDB in kubernetes cluster

Deploy CockroachDB in kubernetes cluster

Deploy Kubernetes cluster $ kubectl get nodes NAME STATUS ROLES AGE VERSION node0 Ready 115d v1.19.2 node1 Ready 115d v1.19.2 node2 Ready 115d v1.19.2 node3 Ready master 115d v1.19.2 Start CockroachDB cluster Start the CockroachDB nodes with a configuration file that has been
relentlesstorm Dec 3, 2022
Setup CockroachDB in docker container

Setup CockroachDB in docker container

Create docker volume Create docker volume on each host: [root@host1 ~]# docker volume create --driver local --opt type=ext4 --opt device=/dev/nvme2n1 vol1 [root@host1 ~]# mkfs.ext4 /dev/nvme2n1 [root@host1 ~]# docker inspect vol1 [ { "CreatedAt": "2022-09-13T01:10:50Z", "Driver": "local", "Labels": {}, "Mountpoint": "/var/lib/docker/volumes/vol1/_data", "Name"
relentlesstorm Dec 2, 2022
Setup CockroachDB cluster with HAProxy load balancing

Setup CockroachDB cluster with HAProxy load balancing

Each CockroachDB node is an equally suitable SQL gateway to your cluster, but to ensure client performance and reliability, it's important to use load balancing. * Performance: Load balancers spread client traffic across nodes. This prevents any one node from being overwhelmed by requests and improves overall cluster performance (queries per
relentlesstorm Dec 1, 2022
How to delete partition in Linux

How to delete partition in Linux

Identify the disk which contains the partitions $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 9.5G 0 rom nvme0n1 259:6 0 1.5T 0 disk ├─nvme0n1p1 259:7 0 200M 0 part /boot/efi ├─nvme0n1p2 259:8 0 1G 0 part /boot └─nvme0n1p3
relentlesstorm Nov 11, 2022
Useful Elastic cluster APIs

Useful Elastic cluster APIs

Useful Elastic cluster APIs
relentlesstorm Oct 28, 2022
Benchmarking Elasticsearch cluster with Rally

Benchmarking Elasticsearch cluster with Rally

Install Rally on each node Prerequisites $ yum update $ yum install openssl-devel bzip2-devel libffi-devel $ yum groupinstall "Development Tools" Install Python 3.8+ $ wget https://www.python.org/ftp/python/3.8.15/Python-3.8.15.tar.xz $ tar xf Python-3.8.15.tar.xz $ vim Python-3.8.15/Modules/Setup SSL=
relentlesstorm Oct 26, 2022
Getting started with Elastic Rally benchmark

Getting started with Elastic Rally benchmark

Rally is the macrobenchmarking framework for Elasticsearch. This post follows the instructions here to install Rally and run the very first benchmark(aka race). Install python Python 3.8+ including pip3 is required for Rally. $ yum update $ yum install openssl-devel bzip2-devel libffi-devel $ yum groupinstall "Development Tools" $ curl -O https://www.
relentlesstorm Oct 25, 2022
Overwrite the Elastic Rally track parameters

Overwrite the Elastic Rally track parameters

Each track allows to overwrite the corresponding parameters using --track-params. For example, the track geonames allows to overwrite the following parameters. * bulk_size (default: 5000) * bulk_indexing_clients (default: 8): Number of clients that issue bulk indexing requests. * ingest_percentage (default: 100): A number between 0 and 100 that defines
relentlesstorm Oct 24, 2022
Key concepts in Elastic Rally

Key concepts in Elastic Rally

Pipelines A pipeline is a series of steps that are performed to get benchmark results. This is not intended to customize the actual benchmark but rather what happens before and after a benchmark. An example will clarify the concept: If you want to benchmark a binary distribution of Elasticsearch, Rally
relentlesstorm Oct 23, 2022
Getting started with Elasticsearch and Kibana

Getting started with Elasticsearch and Kibana

Install elasticsearch Add user for elasticsearch [root@vm1 home]# groupadd es [root@vm1 home]# useradd es -g es [root@vm1 home]# passwd es [root@vm1 home]# cd es Download elasticsearch [root@vm1 es]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.4.3-linux-x86_64.tar.gz [root@vm1 es]# tar
relentlesstorm Oct 17, 2022
How to setup the Grafana and Prometheus dashboard

How to setup the Grafana and Prometheus dashboard

Install node-exporter on each target nodes $ docker run --restart=always --name nd-export -d -p 9100:9100 -v /proc:/host/proc -v /sys:/host/sys -v /:/rootfs:ro,rslave --net=host quay.io/prometheus/node-exporter $ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d8c7fc8a4e40 quay.io/prometheus/node-exporter
relentlesstorm Oct 15, 2022
Install golang from source

Install golang from source

$ wget https://dl.google.com/go/go1.15.5.linux-amd64.tar.gz $ tar -C /usr/local -xvzf go1.15.5.linux-amd64.tar.gz $ cat /usr/local/go/VERSION go1.15.5 $ vim .bash_profile PATH=$PATH:$HOME/bin:/usr/local/go/bin export PATH $ source .bash_profile $ go version go
relentlesstorm Oct 8, 2022
go-ycsb load db terminated with message 'Got signal [hangup] to exit'

go-ycsb load db terminated with message 'Got signal [hangup] to exit'

During database load phase with go-ycsb, the load process is terminated and the message 'Got signal [hangup] to exit' is reported. $ nohup ./bin/go-ycsb load cockroach -P workloads/workloadd --threads 96 -p pg.host= -p pg.port=26257 -p pg.user=root -p pg.db=test -p pg.sslmode=disable
relentlesstorm Oct 6, 2022
Limit usable memory size in Linux grub

Limit usable memory size in Linux grub

In one case, I need to limit the usable memory size to 128GB from a system which has 1TB memory. $ cat /proc/meminfo | grep MemTotal MemTotal: 1056493068 kB Edit /etc/default/grub file Add "mem=128G" to the end of line "GRUB_CMDLINE_LINUX". $ cat /etc/default/grub GRUB_TIMEOUT=
relentlesstorm Oct 5, 2022
YCSB performance benchmark on CockroachDB

YCSB performance benchmark on CockroachDB

Intro to go-ycsb go-ycsb is a Go port of YCSB. It fully supports all YCSB generators and the Core workload so we can do the basic CRUD benchmarks with Go. Install golang package $ wget https://dl.google.com/go/go1.19.2.linux-amd64.tar.gz $ tar -C /usr/local -xvzf
relentlesstorm Oct 2, 2022
Understanding Azure disk bursting

Understanding Azure disk bursting

Currently, there are two managed disk types that can burst, premium SSDs, and standard SSDs. Other disk types cannot currently burst. There are two models of bursting for disks: An on-demand bursting model, where the disk bursts whenever its needs exceed its current capacity. This model incurs additional charges anytime
relentlesstorm Sep 15, 2022
Using netstat to check network package retransmission

Using netstat to check network package retransmission

$ while [ 1 ]; do netstat -s | grep 'segments retransmited' ; sleep 1; done 2500906 segments retransmited 2500912 segments retransmited 2501072 segments retransmited 2501102 segments retransmited 2501191 segments retransmited
relentlesstorm Sep 2, 2022
Compile Linux kernel on CentOS

Compile Linux kernel on CentOS

Build preparations To perform a successful kernel build, the following packages need to be installed: * yum groupinstall "Development Tools" * yum install ncurses-devel * yum install qt3-devel (This is only necessary if you wish to use "make xconfig" instead of "make gconfig" or "make menuconfig".) In the following example, we use "make
relentlesstorm Jul 19, 2022
Deploy systemtap on multiple systems

Deploy systemtap on multiple systems

Normally, SystemTap scripts can only be run on systems where SystemTap is deployed together with the following required kernel packages. * kernel-devel-$(uname -r) * kernel-debuginfo-$(uname -r) * kernel-debuginfo-common-$(uname -m)-$(uname -r) In the case that it's neither feasible nor desired to install the denpendent packages on all the target
relentlesstorm Jul 17, 2022
Using systemtap to analyze latency of the kernel module function

Using systemtap to analyze latency of the kernel module function

In this post, we continue to explore how to use SystemTap to analyze the latency of the kernel module function. In the following example, we want to analyze the latency of the function "nfsd_vfs_write" from the kernel module "nfsd". Deploy the SystemTap packages Refer to this post to
relentlesstorm Jul 16, 2022
Getting started with systemtap for Linux system profiling

Getting started with systemtap for Linux system profiling

Intro to SystemTap SystemTap is a tracing and probing tool that allows users to study and monitor the activities of the operating system (particularly, the kernel) in fine detail. It provides information similar to the output of tools like netstat, ps, top, and iostat; however, SystemTap is designed to provide
relentlesstorm Jul 15, 2022
Using ftrace to analyze latency of the kernel module function

Using ftrace to analyze latency of the kernel module function

In this example, we study the latency of the function "nfsd_vfs_write" from kernel module "nfsd". ftrace configuration The following ftrace options are used in this example. There are 8 nfsd processes to be traced. * current_tracer: function_graph * pids: 3591 3592 3593 3594 3595 3596 3597 3598 * filters:
relentlesstorm Jul 14, 2022
Dynamically tracing with user-defined tracepoint in perf

Dynamically tracing with user-defined tracepoint in perf

In this post, we are going to explore how to use perf for dynamic tracing with user-defined tracepoint. When we say dynamic tracing, the kernel event(function) to be traced is not predefined in perf. Instead, we add the tracepoint event manually. We will study it based on a real
relentlesstorm Jul 13, 2022
tuned - dynamic adaptive system tuning daemon

tuned - dynamic adaptive system tuning daemon

Tuned Tuned is a daemon that uses udev to monitor connected devices and statically and dynamically tunes system settings according to a selected profile. Tuned is distributed with a number of predefined profiles for common use cases like high throughput, low latency, or powersave. It is possible to modify the
relentlesstorm Jul 12, 2022
Install and uninstall debuginfo package on CentOS

Install and uninstall debuginfo package on CentOS

Install debuginfo package On CentOS, we can install debuginfo package as below. * Modify /etc/yum.repos.d/CentOS-Debuginfo.repo by setting "enabled=1" * Run "yum install kernel-debuginfo" [root@host1 ~]# uname -r 3.10.0-1160.el7.x86_64 [root@host1 ~]# vim /etc/yum.repos.d/CentOS-Debuginfo.repo [base-debuginfo] name=CentOS-7 -
relentlesstorm Jul 11, 2022
Using sar to understand system activity

Using sar to understand system activity

About sadc and sar sadc is known as system activity data collector. It samples system data a specified number of times (count) at a specified interval measured in seconds (interval). It writes in binary format to the specified outfile or to standard output. The sadc command is intended to be
relentlesstorm Jul 6, 2022
Using nconnect to improve NFS performance

Using nconnect to improve NFS performance

The nfs mount option "nconnect=n" exists in all Linux distributions with kernel 5.3 or higher. nconnect enables multiple TCP connections for a single NFS mount. From the nfs manual page: nconnect=n When using a connection oriented protocol such as TCP, it may sometimes be advantageous to set
relentlesstorm Jun 30, 2022
How to setup NFS (Network File System) on RHEL/CentOS

How to setup NFS (Network File System) on RHEL/CentOS

We need to install NFS packages on the NFS Server as well as on NFS Client machine. We can install it via “yum” for RHEL/CentOS. On the NFS server side, run the following commands to setup NFS server. $ systemctl status nfs $ mkfs.ext4 /dev/nvme9n1 $ mkdir /mnt/nfsshare $ vim
relentlesstorm Jun 29, 2022
SysRq - Linux Magic System Request Key Hacks

SysRq - Linux Magic System Request Key Hacks

What is the magic SysRq key? It is a ‘magical’ key combo you can hit which the kernel will respond to regardless of whatever else it is doing, unless it is completely locked up. How do I enable the magic SysRq key? You need to say “yes” to ‘Magic SysRq
relentlesstorm May 21, 2022
journalctl - Query the systemd journal

journalctl - Query the systemd journal

journalctl may be used to query the contents of the systemd journal as written by systemd-journald.service. If called without parameters, it will show the full contents of the journal, starting with the oldest entry collected. The following are some of the common used options: * --no-full, --full, -l Ellipsize fields
relentlesstorm May 20, 2022
Flame graph - A visualization for profiling and debugging

Flame graph - A visualization for profiling and debugging

Flame graphs can be generated from the output of many different software profilers, including profiles for different resources and event types. The article written by Brendan Gregg describes how the flame graph works. We can start to work with the flame graph by following commands. $ yum install git $ git clone
relentlesstorm May 19, 2022
A six-step framework to tackle any system design question

A six-step framework to tackle any system design question

Introduction In the book of System Design Interview by Lewis C. Lin, a six-step framework called PEDALS is introduced to help answer any system design question. PEDALS stand for: * Process requirements * Estimate * Design the service * Articulate the data model * List the architectural components * Scale The systematic method will power you
relentlesstorm May 8, 2022
The complete guide to System Design

The complete guide to System Design

The complete guide to System Design in 2022 covers: * What is System Design? * System Design fundamentals * Horizontal and vertical scaling * Microservices * Proxy servers * CAP theorem * Redundancy and replication * Storage * Block storage * File storage * Object storage * Redundant Disk Arrays (RAID) * Message queues * Kafka * File systems * Google File System (GFS) * Hadoop Distributed
relentlesstorm May 4, 2022
Understanding RPO and RTO

Understanding RPO and RTO

The disasters could occur unexpectedly and it could take down the whole data center in an application region. In the real world, a well designed disaster recovery solution would help restore the applications and related data at the time of disaster. There are two critical metrics to measure how timely
relentlesstorm May 1, 2022
Understanding the Linux inodes

Understanding the Linux inodes

Linux makes a clear distinction between the contents of a file and the information about a file. All information needed by the filesystem to handle a file is included in a data structure called an inode. Each file has its own inode, which the filesystem uses to identify the file.
relentlesstorm Apr 30, 2022
Understanding MapReduce with an example

Understanding MapReduce with an example

What is MapReduce MapReduce is a programming framework that allows us to perform distributed and parallel processing on large data sets in a distributed environment. * MapReduce consists of two distinct tasks — Map and Reduce. * As the name MapReduce suggests, reducer phase takes place after the mapper phase has been completed.
relentlesstorm Apr 29, 2022
AngularJS table creation for dynamic number of columns

AngularJS table creation for dynamic number of columns

{{column}}
relentlesstorm Apr 22, 2022
Hide and display html elements

Hide and display html elements

relentlesstorm Apr 21, 2022
Access a sharedv4 volume outside of a Portwrox cluster

Access a sharedv4 volume outside of a Portwrox cluster

To access a sharedv4 volume outside of the Portworx cluster, add the allow_ips label to the volume you wish to export, specifying a semi-colon separated list of IP addresses of non-Portworx nodes you wish to mount your sharedv4 volume to: $ /opt/pwx/bin/pxctl -j volume create datavol --sharedv4
relentlesstorm Apr 20, 2022
Using iperf to measure the network bandwidth

Using iperf to measure the network bandwidth

iPerf is a tool for measurements of the maximum achievable bandwidth on IP network. The following iperf option can be used on the client side to saturate the network bandwidth if single client thread is not sufficient. * -P, --parallel # number of parallel client threads to run Benchmark a 100GbE network
relentlesstorm Apr 19, 2022
How to deploy MySQL and phpMyAdmin with Docker

How to deploy MySQL and phpMyAdmin with Docker

Create the MySQL docker container $ docker pull mysql/mysql-server:5.7 $ docker run -d --name=mysqldb -e MYSQL_ROOT_HOST=% -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=perfdb -v /var/lib/osd/mounts/mysql_data:/var/lib/mysql -p 3306:3306 mysql/mysql-server:5.7 $ docker logs mysqldb $ docker
relentlesstorm Apr 12, 2022
What is Performance Engineering

What is Performance Engineering

relentlesstorm Apr 8, 2022
Useful tools in Linux operating system

Useful tools in Linux operating system

Performance tools * vmstat * iostat * ifstat * netstat * nfsstat * mpstat * nstat * dstat * sar * iftop * pidstat * xosview Benchmark tools * fio * iozone * iperf * netperf * vdbench * sysbench * pgbench * YCSB * SPEC SFS 2014/SPECstorage Solution 2020 * VMmark Debugging tools * htop * lslk * lsof * top Process tracing * ltrace * strace * pstack/gstack * ftrace * systemtap * ps * pmap * blktrace * ebpf Binary
relentlesstorm Mar 25, 2022
OpenZFS - A pooled storage file system

OpenZFS - A pooled storage file system

OpenZFS The OpenZFS project is an open source derivative of the Oracle ZFS project. OpenZFS is an outstanding storage platform that encompasses the functionality of traditional filesystems, volume managers, and more, with consistent reliability, functionality and performance across all distributions. Source1 Source2 ZFS and ZFS Pooled Storage The ZFS file
relentlesstorm Mar 16, 2022
Fio initial write, overwrite and append write

Fio initial write, overwrite and append write

In this post, we try to study how the overwrite and file_append options work in fio benchmark. overwrite and file_append options * overwrite=bool If true, writes to a file will always overwrite existing data. If the file doesn’t already exist, it will be created before the write
relentlesstorm Mar 15, 2022
Fio fsync, end_fsync, fdatasync and sync

Fio fsync, end_fsync, fdatasync and sync

Intro to fsync, end_fsync, fdatasync and sync * fsync=int If writing to a file, issue an fsync(2) (or its equivalent) of the dirty data for every number of blocks given. For example, if you give 32 as a parameter, fio will sync the file after every 32 writes
relentlesstorm Mar 14, 2022
Scale a system to support millions of users

Scale a system to support millions of users

Start from single server Building a large scale system is not one time effort, it should be an iterative process as the user workload increases. The journey could just start with single server and single user request. 1. User access a website through domain name, such as example.com 2.
relentlesstorm Mar 13, 2022
Set in golang

Set in golang

Go does not have set by default but there is way to imitate it in Go. The following is an example code piece to create "set" in Go by setting a map with empty values. $ cat test.go package main import ( "fmt" ) // Main function func main() { set := make(map[string]
relentlesstorm Mar 12, 2022
Must-read books and articles for Software and Performance Engineering

Must-read books and articles for Software and Performance Engineering

Books * The Art of Computer Programming * Understanding the Linux Kernel, 3rd Edition * Linux Device Drivers, 3rd Edition * Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems, 1st Edition * Building Microservices: Designing Fine-Grained Systems 1st Edition * Site Reliability Engineering: How Google Runs Production Systems 1st E
relentlesstorm Mar 11, 2022
Vdbench file system operations

Vdbench file system operations

Specify file system operations in FWD and RD In the File system Workload Definition(FWD), we can specify a single file sysetm operation that must be executed for the workload, with the parameter 'operation='. The valid operation can be mkdir,rmdir,create,delete,open,close,read,write,getattr and
relentlesstorm Mar 10, 2022
Proc filesystem - diskstats

Proc filesystem - diskstats

The /proc/diskstats file displays the I/O statistics of block devices. Here we have a system which has one disk sda used by Linux operating system and two disks sdb sdc for other purpose. $ cat /etc/centos-release CentOS Linux release 7.9.2009 (Core) $ uname -r 3.10.0-1160.
relentlesstorm Mar 5, 2022
dstat - A replacement for vmstat, iostat and ifstat

dstat - A replacement for vmstat, iostat and ifstat

Introduction dstat is a versatile tool for generating system resource statistics. It can be a replacement for vmstat, iostat and ifstat. It overcomes some of the limitations and adds some extra features. Dstat allows you to view all of your system resources instantly, you can eg. compare disk usage in
relentlesstorm Mar 4, 2022
Docker-based fio benchmarking - Part Two

Docker-based fio benchmarking - Part Two

In this article, we will build a docker image which is based on python versioned alpine Linux. Alpine Linux is much smaller than most distribution base images (~5MB), and thus leads to much slimmer images in general. In the python versioned alpine Linux image, we can add additional packages to
relentlesstorm Feb 23, 2022
Docker-based fio benchmarking - Part One

Docker-based fio benchmarking - Part One

Why Docker-based fio benchmarking fio is a flexible I/O tester which generates I/O and measure I/O performance on the target storage system. In the case we want to run the fio workload on the cloud deployments, we can containerize fio. Also we can encapsulate necessary packages in
relentlesstorm Feb 22, 2022
Concepts you should know

Concepts you should know

Data structure Bloom filter A Bloom filter is a space-efficient probabilistic data structure that is used to test whether an element is a member of a set. For example, checking availability of username is set membership problem, where the set is the list of all registered username. The price we
relentlesstorm Feb 15, 2022
Storage concepts you should know

Storage concepts you should know

Object Storage Object storage, also known as object-based storage, is a strategy that manages and manipulates data storage as distinct units, called objects. These objects are kept in a single storehouse and are not ingrained in files inside other folders. Instead, object storage combines the pieces of data that make
relentlesstorm Feb 14, 2022
Network concepts you should know

Network concepts you should know

IP address Networking is about one computer sending a message to another computer. This message is called packet in the IP world. It is just like a postcard in the postal service. The postal service can take a postcard addressed to someone and deliver it. Similarly, it needs to provide
relentlesstorm Feb 13, 2022
Database concepts you should know

Database concepts you should know

SQL vs. NoSQL Source Benefits of NoSQL databases NoSQL databases offer many benefits over relational databases. NoSQL databases have flexible data models, scale horizontally, have incredibly fast queries, and are easy for developers to work with. * Flexible data models NoSQL databases typically have very flexible schemas. A flexible schema allows
relentlesstorm Feb 12, 2022
Distributed system concepts you should know

Distributed system concepts you should know

This post contains basic explanations for concepts you should know related to distributed system. API and REST API Application Programming Interface, abbreviated as API, enables connection between computers or computer programs. It is a Software Interface that offers services to other software to enhance the required functionalities. REST API is
relentlesstorm Feb 11, 2022
Estimate the scale of the system

Estimate the scale of the system

It is always a good idea to estimate the scale of the system as it help reflect if the designed system could fulfill the functional requirements. The requirements might include: * Number of users * Number of active users(NAU) * Requrests per second(RPS) * Logins per seconds * Transactions per second(TPS) for
relentlesstorm Feb 5, 2022
What is high availability

What is high availability

What is high availability? High availability refers to a system or component that is operational without interruption for long periods of time. High availability is measured as a percentage, with a 100% percent system indicating a service that experiences zero downtime. This would be a system that never fails. It’
relentlesstorm Feb 3, 2022
Latency numbers

Latency numbers

Latency Comparison Numbers (~2012) Conclusion * Disk is much slower than the memory * Avoid the disk seek if possible * Compressing data is worth to consider before sending over network * It takes time to send data between data centers if they are in different regions Reference * https://computers-are-fast.github.io/ * http://research.
relentlesstorm Feb 2, 2022
Data volume unit

Data volume unit

Data volumes The volume of data in a single file or file system can be described by a unit called a byte. However, data volumes can become very large when dealing with Earth satellite data. Below is a table to explain data volume units (Credit: Roy Williams, Center for Advanced
relentlesstorm Feb 1, 2022
Install git from source

Install git from source

$ wget https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.38.1.tar.xz $ tar xvf git-2.38.1.tar.xz $ cd git-2.38.1/ $ make configure $ ./configure --prefix=/usr/local $ make install $ git --version git version 2.38.1 Reference * https://mirrors.edge.kernel.org/pub/software/scm/
relentlesstorm Jan 7, 2022
Understanding thin provisioning volume and snapshot

Understanding thin provisioning volume and snapshot

Thin provisioning volume Logical volume can be thinly provisioned. It allows storage administrator to overcommit the physical storage. In other words, it's possible to create a logical volume which is larger than the available extents. Create thin provisioning volume In the following example, we create a 500GiB thin pool and
relentlesstorm Jan 6, 2022
How to configure SAR data collection on RHEL8

How to configure SAR data collection on RHEL8

On RHEL8, it uses systemd instead of cron jobs to manage SAR data collection service. Run the following command to check if the SAR data collection is started. [root@h04-11 ~]# cat /etc/redhat-release Red Hat Enterprise Linux release 8.5 (Ootpa) [root@h04-11 ~]# systemctl status sysstat-collect.timer ● sysstat-collect.timer -
relentlesstorm Jan 4, 2022
Solve LeetCode problems in Visual Studio Code

Solve LeetCode problems in Visual Studio Code

Install the leetcode extension in VS Code Login to LeetCode endpoint with cookie 1. Login to leetcode from Google Chrome 2. In Chrome, Inspect -> Network -> Fetch/XHR 3. Click on any button in leetcode page, and in Inspector to the right under the "Name" tab find and select
relentlesstorm Dec 5, 2021
libaio init failed due to resource temporarily unavailable

libaio init failed due to resource temporarily unavailable

Issue Description $ sudo fio --blocksize=64k --directory=/mnt/bench1 --filename=testfile --ioengine=libaio --readwrite=randread --size=10G --name=test --numjobs=512 --group_reporting --direct=1 --iodepth=128 --randrepeat=1 --disable_lat=0 --gtod_reduce=0 test: (g=0): rw=randread, bs=(R) 64.0KiB-64.0KiB, (W) 64.0KiB-64.0KiB, (T)
relentlesstorm Dec 4, 2021
How to uninstall ceph storage cluster

How to uninstall ceph storage cluster

Check the pools, images and OSDs [ceph: root@host1 /]$ ceph osd tree ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF -1 83.83411 root default -3 27.94470 host host1 0 ssd 3.49309 osd.0 up 1.00000 1.00000 1 ssd 3.49309 osd.1 up 1.00000
relentlesstorm Oct 14, 2021
fio benchmark on multiple files

fio benchmark on multiple files

fio directory and filename options To run fio benchmark on multiple files or deives, we should understand the following fio options. * directory=str Prefix filenames with this directory. Used to place files in a different location than ./. You can specify a number of directories by separating the names with a
relentlesstorm Oct 13, 2021
fio benchmark on multiple devices

fio benchmark on multiple devices

In this post, we study how to run fio benchmark on multiple devices. We also try to understand how the iodepth reflects on each device. We start with single device and the following global parameters are used. * blocksize=16k * filesize=50G (write/read 50G data on each device) * iodepth=64
relentlesstorm Oct 12, 2021
Backup and restore MySQL database

Backup and restore MySQL database

Backup a database mysqldump is a command-line utility which can be used to generate backups of MySQL database. $ mysqldump -u root --password= mydb > mydb_dump_`date +"%Y%m%d_%H%M%S"`.sql $ ls -ltr | grep mydb -rw-r--r--. 1 root root 4834575 Sep 28 21:11 mydb_dump_20210928_144610.
relentlesstorm Sep 28, 2021
Create Image Gallery in Jekyll without Plugin

Create Image Gallery in Jekyll without Plugin

An image gallery can be easily built by using LightBox and Image Gallery scripts in Jekyll. 1. LightBox Lightbox is a solution that loads your image links, your Youtube links and your Vimeo links automatically in a minimalistic and responsive pseudo window/overlay. No adjustment to your links is required,
relentlesstorm Sep 19, 2021
Using sysbench for OLTP workload performance benchmark

Using sysbench for OLTP workload performance benchmark

Intro to Sysbench sysbench is a scriptable multi-threaded benchmark tool based on LuaJIT. It is most frequently used for database benchmarks, but can also be used to create arbitrarily complex workloads that do not involve a database server. sysbench comes with the following bundled benchmarks: * oltp_*.lua: a collection of
relentlesstorm Sep 7, 2021
An easy guide to install kubernetes cluster with kubeadm

An easy guide to install kubernetes cluster with kubeadm

Kubernetes can be installed with the following deployments tools. * Bootstrapping clusters with kubeadm * Installing Kubernetes with kops * Installing Kubernetes with Kubespray In this article, we learn to install Kubernetes cluster with kubeadm. Installing kubeadm Prepare the cluster nodes We have three Centos nodes to install Kubernetes cluster. $ cat /etc/centos-release
relentlesstorm Aug 29, 2021
Vdbench performance test on filesystem

Vdbench performance test on filesystem

File system testing - Method 1 Vdbench filesystem testing terminologies: * Anchor - A directory or a filesystme mount point. A file system structure will be created by specifying the structure information including directory depth, width, number of files and file size. Multiple anchor can be defined and used by filesystem
relentlesstorm Aug 24, 2021
Vdbench performance test on raw device

Vdbench performance test on raw device

Terminology * Master and Slave: Vdbench runs as two or more Java Virtual Machines (JVMs). The JVM that you start is the master. The master takes care of the parsing of all the parameters, it determines which workloads should run, and then will also do all the reporting. The actual workload
relentlesstorm Aug 23, 2021
Deploy ceph cluster on Ubuntu 18.04 and CentOS 7.8

Deploy ceph cluster on Ubuntu 18.04 and CentOS 7.8

In this article, we learn to deploy ceph cluster on ubuntu 18.04. Three nodes are used for this study. We target to deploy the most recent ceph release which is called Pacific. With this release, we can use cephadm to create a ceph cluster by bootstrapping on a single
relentlesstorm Aug 22, 2021
Capture and analyze network packets with tcpdump

Capture and analyze network packets with tcpdump

Capture packets with tcpdump In this example, we only capture 1000 packets(-c1000) and use IP addresses and ports(-nn) for easier analysis. The raw packets are written to file "tcpdump.1000" for further analsis. $ tcpdump -i eth5 -c1000 -nn -w tcpdump.1000 View the packets data We can use
relentlesstorm Aug 20, 2021
Useful tools to analyze network latency

Useful tools to analyze network latency

ping Ping is one of the most basic commands in network management, verifying network connectivity through the roundtrip times taken by the ICMP protocol packets sent to a target host. ping - send ICMP ECHO_REQUEST to network hosts * -c count Stop after sending count ECHO_REQUEST packets. With deadline
relentlesstorm Aug 13, 2021
Iouring - A modern asynchronous I/O interface for Linux

Iouring - A modern asynchronous I/O interface for Linux

The Linux kernel has had asynchronous I/O since version 2.5, but it was seen as difficult to use and inefficient. io_uring (previously known as aioring) is a Linux kernel system call interface for storage device asynchronous I/O operations addressing performance issues with similar interfaces provided by
relentlesstorm Aug 4, 2021
btrfs - A modern copy on write(CoW) filesystem for Linux

btrfs - A modern copy on write(CoW) filesystem for Linux

Introduction btrfs is a modern copy on write (CoW) filesystem for Linux aimed at implementing advanced features while also focusing on fault tolerance, repair and easy administration. Its main features and benefits are: * Snapshots which do not make the full copy of files * RAID - support for software-based RAID 0,
relentlesstorm Aug 3, 2021
Setting up LVM volumes on a mdraid array

Setting up LVM volumes on a mdraid array

Setting up mdraid array $ mdadm --create /dev/md0 --name=mdvol --level=raid0 --raid-devices=2 /dev/nvme1n1 /dev/nvme2n1 /dev/nvme3n1 /dev/nvme4n1 mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md0 started. $ cat /proc/mdstat Personalities : [raid0] [linear] md0 : active raid0 nvme4n1[3] nvme3n1[2] nvme2n1[1] nvme1n1[
relentlesstorm Aug 2, 2021
Linux Software RAID

Linux Software RAID

What is RAID? RAID stands for either Redundant Array of Independent Disks, or Redundant Array of Inexpensive Disks. The intention of RAID is to spread your data across several disks, such that a single disk failure will not lose that data. The current RAID drivers in Linux support the following
relentlesstorm Aug 1, 2021
RAID - Redundant Array of Inexpensive Disks

RAID - Redundant Array of Inexpensive Disks

RAID stands for Redundant Array of Inexpensive (Independent) Disks. On most situations you will be using one of the following four levels of RAIDs. * RAID 0 * RAID 1 * RAID 5(6) * RAID 10 (also known as RAID 1+0) RAID 0 Following are the key points to remember for RAID
relentlesstorm Jul 31, 2021
Upgrade kernel version in centos 7

Upgrade kernel version in centos 7

Before upgrade $ cat /etc/centos-release CentOS Linux release 7.4.1708 (Core) $ uname -r 3.10.0-693.el7.x86_64 $ cat /boot/grub2/grubenv saved_entry=CentOS Linux (3.10.0-693.el7.x86_64) 7 (Core) Install the newer kernel package $ rpm -ivh kernel-ml-5.7.12-1.el7.elrepo.x86_64.
relentlesstorm Jul 18, 2021
Migrate docker data to different directory

Migrate docker data to different directory

This post guides us how to migrate the docker data from the existing directory to a target directory in the case that the old directory runs out of space. Stop the docker daemon $ cat /etc/centos-release CentOS Linux release 7.9.2009 (Core) $ systemctl stop docker.service $ ps aux | grep
relentlesstorm Jul 10, 2021
Using gstack for docker container process

Using gstack for docker container process

In a Docker container environment, we won't get a valid stack trace directly on the container host as below. $ ps -ef |grep smbd root 171118 167977 0 Apr22 ? 00:00:02 /usr/sbin/smbd --foreground --no-process-group root 171166 171118 0 Apr22 ? 00:00:00 /usr/sbin/smbd --foreground --no-process-group root
relentlesstorm Apr 27, 2021
Network bonding

Network bonding

Network bonding enables the combination of two or more network interfaces into a single-bonded (logical) interface, which increases the bandwidth and provides redundancy. If a specific network interface card (NIC) experiences a problem, communications are not affected significantly as long as the other slave NICs remain active. Bonding modes supported
relentlesstorm Apr 19, 2021
Enable ssh passwordless login for large amount of servers

Enable ssh passwordless login for large amount of servers

Install sshpass package To install sshpass package on the source server from which to enable ssh passwordless login to remote servers: -bash-4.2# rpm -ivh sshpass-1.06-1.el7.x86_64.rpm Enable ssh passwordless login to remote servers To enable ssh passwordless login to a large amount of remote servers,
relentlesstorm Apr 18, 2021
Tuning kernel parameters in Docker

Tuning kernel parameters in Docker

Configure namespaced kernel parameters(sysctl) at runtime The --sysctl sets namespaced kernel parameters (sysctls) in the container. For example, to turn on IP forwarding in the containers network namespace, run this command: $ docker run --sysctl net.ipv4.ip_forward=1 someimage Note Not all sysctls are namespaced. Docker does not
relentlesstorm Apr 17, 2021
Sorting algorithm

Sorting algorithm

In this post, it includes the following sorting algorithms. The code is self explained. * selection sort * insert sort * bubble sort * quick sort * merge sort import java.util.Arrays; public class Sort { public static void selection_sort(int[] arr) { for (int i = 0; i < arr.length - 1; i++) { int min_
relentlesstorm Apr 16, 2021
Thread synchronization

Thread synchronization

Mutexes A mutex is basically a lock that we set (lock) before accessing a shared resource and release (unlock) when we’re done. While it is set, any other thread that tries to set it will block until we release it. If more than one thread is blocked when we
relentlesstorm Apr 11, 2021
Interprocess communication(IPC)

Interprocess communication(IPC)

Pipes Pipes are the oldest form of UNIX System IPC and are provided by all UNIX systems. Pipes have two limitations. 1. Historically, they have been half duplex (i.e., data flows in only one direction). Some systems now provide full-duplex pipes, but for maximum portability, we should never assume
relentlesstorm Apr 10, 2021
Using bcc-tools for dynamic kernel tracing

Using bcc-tools for dynamic kernel tracing

In Red Hat Enterprise Linux 8.1, Red Hat ships a set of fully supported on x86_64 dynamic kernel tracing tools, called bcc-tools, that make use of a kernel technology called extended Berkeley Packet Filter (eBPF). With these tools, you can quickly gain insight into certain aspects of system
relentlesstorm Mar 25, 2021
blktrace - A block layer IO tracing utility

blktrace - A block layer IO tracing utility

What can blktrace do? Let's dive into the block device layer and see how the I/O are handled in disk queues. The following stack shows the I/O paths including the block device layer. The application can issue I/O requests directly to block device or through file systems.
relentlesstorm Mar 11, 2021
Understanding await in iostat

Understanding await in iostat

What's meaning of await in iostat? The following is the description provided for await field in iostat man page. $ man iostat await The average time (in milliseconds) for I/O requests issued to the device to be served. This includes the time spent by the requests in queue and the
relentlesstorm Mar 10, 2021
Buffered and Direct IO

Buffered and Direct IO

Buffered and Direct I/O The VxFS responds with read-ahead for sequential read I/O. This results in buffered I/O. The data is prefetched and retained in buffers, in anticipation of application asking for it. The data buffers are commonly referred to as the VxFS buffer cache. This is
relentlesstorm Mar 9, 2021
Network ring buffer

Network ring buffer

Introduction Receive ring buffers are shared between the device driver and NIC. The card assigns a transmit (TX) and receive (RX) ring buffer. As the name implies, the ring buffer is a circular buffer where an overflow simply overwrites existing data. It should be noted that there are two ways
relentlesstorm Mar 4, 2021
Semaphore

Semaphore

What is semaphore A semaphore is a very relaxed type of lockable object. A given semaphore has a predefined maximum count, and a current count. You take ownership of a semaphore with a wait operation, also referred to as decrementing the semaphore, or even just abstractly called P. You release
relentlesstorm Mar 3, 2021
Max open files limit

Max open files limit

System wide open files limit To check system wide files limit: $ cat /proc/sys/fs/file-max 4875932 $ sysctl -a | grep file-max fs.file-max = 4875932 To change system wide files limit: $ echo "fs.file-max = 4875932" >> /etc/sysctl.conf $ sysctl -p /etc/sysctl.conf User level open files limit To check hard/
relentlesstorm Mar 2, 2021
Iozone - A filesystem benchmark tool

Iozone - A filesystem benchmark tool

Introduction IOzone is a filesystem benchmark tool. The benchmark generates and measures a variety of file operations. Iozone is useful for performing a broad filesystem analysis of a vendor’s computer platform. The benchmark tests file I/O performance for the following operations: * Read, write, re-read, re-write, read backwards, read
relentlesstorm Mar 1, 2021
Kubernetes and Gluster performance

Kubernetes and Gluster performance

Kubernetes and Gluster Intro Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications. Gluster is a scalable, distributed file system that aggregates disk storage resources from multiple servers into a single global namespace. Gluster performance study In this article, we will
relentlesstorm Feb 28, 2021
GlusterFS - A distributed file syste

GlusterFS - A distributed file syste

What is Gluster? Gluster is a scalable, distributed file system that aggregates disk storage resources from multiple servers into a single global namespace. Advantages * Scales to several petabytes * Handles thousands of clients * POSIX compatible * Uses commodity hardware * Can use any ondisk filesystem that supports extended attributes * Accessible using industry standard
relentlesstorm Feb 27, 2021
CBT - Changed Block Tracking

CBT - Changed Block Tracking

Introduction Changed Block Tracking is an incremental backup technology for virtual machines. It helps create faster and smaller backups. It has the following advantages. * Reduce backup time * Save disk space by only storing changed data to the previous backup Block changes are tracked in the virtualization layer, outside the virtual
relentlesstorm Feb 26, 2021
Perf - The official Linux profiler

Perf - The official Linux profiler

Introduction perf, aka perf_events, is the official Linux profiler and included in the Linux kernel source under tools/perf. It can instrument CPU performance counters, tracepoints, kprobes, and uprobes. It is capable of system profiling. Performance Monitoring Counter(PMC) Show system wide PMC statistics The following example shows PMC
relentlesstorm Feb 20, 2021
Using fio to generate millions of files

Using fio to generate millions of files

fio is the acronym for Flexible IO Tester and is a tool for I/O performance measurement. From time to time, I use fio to run performance benchmark test, either to investigate I/O performance for certain storage or reproduce performance issues. In this article, we are not target to
relentlesstorm Feb 19, 2021
Analyze the library and system calls

Analyze the library and system calls

When the application source code is not available, we can trace the function calls during its runtime in order to understand how the application code works. Library calls take place in user space. System calls is the fundamental interface between application and the kernel. System calls are not invoked directly
relentlesstorm Feb 18, 2021
Memory-Mapped I/O

Memory-Mapped I/O

Introduction Memory-mapped I/O lets us map a file on disk into a buffer in memory so that, when we fetch bytes from the buffer, the corresponding bytes of the file are read. Similarly, when we store data in the buffer, the corresponding bytes are automatically written to the file.
relentlesstorm Feb 4, 2021
Memory fragmentation

Memory fragmentation

Memory fragmentation Memory page availability can be checked from /proc/buddyinfo as below. $ cat /proc/buddyinfo Node 0, zone DMA 1 0 1 0 1 1 1 0 1 1 3 Node 0, zone DMA32 3342 2441 2138 5025 1871 236 3 0 0 0 0 Node 0, zone Normal
relentlesstorm Feb 3, 2021
Memory overcommit

Memory overcommit

The Linux kernel supports the following overcommit handling modes 0 - Heuristic overcommit handling. Obvious overcommits of address space are refused. Used for a typical system. It ensures a seriously wild allocation fails while allowing overcommit to reduce swap usage. root is allowed to allocate slightly more memory in this
relentlesstorm Feb 2, 2021
kmalloc and vmalloc

kmalloc and vmalloc

kmalloc The function allocates contiguous region in physical memory. It's fast and doesn't clear the allocated memory content. kmalloc function is defined in include/linux/slab.h /** * kmalloc - allocate memory * @size: how many bytes of memory are required. * @flags: the type of memory to allocate. * * kmalloc is the normal
relentlesstorm Feb 1, 2021
Useful HBA commands

Useful HBA commands

Check HBA card type $ lspci | grep -i fibre 18:00.0 Fibre Channel: QLogic Corp. ISP2722-based 16/32Gb Fibre Channel to PCIe Adapter (rev 01) 18:00.1 Fibre Channel: QLogic Corp. ISP2722-based 16/32Gb Fibre Channel to PCIe Adapter (rev 01) Physical slot, driver, module information $ lspci -v -s
relentlesstorm Nov 10, 2020
Monitor RMAN backup and restore progress

Monitor RMAN backup and restore progress

The following script will allow you to monitor progress of an RMAN backup or Restore from 4 different perspectives (channel, session wait events, datafiles, backuppieces). The script is run from SQLPlus and takes a date input value in the format ‘dd-mon-rr hh24:mi:ss’. The date supplied does not have
relentlesstorm Nov 9, 2020
RMAN backup set, pieces, and datafiles

RMAN backup set, pieces, and datafiles

RMAN backs up the datafiles, control file, archived log files, and server parameter files in a RMAN specific format called a backup piece. In a nutshell, a backup set is a bundle of dbf, ctl and redo file that can restore a database. A set of one or more such
relentlesstorm Nov 8, 2020
Level 0 and Level 1 Incremental Backups

Level 0 and Level 1 Incremental Backups

Incremental backups can be either level 0 or level 1. A level 0 incremental backup, which is the base for subsequent incremental backups, copies all blocks containing data, backing the datafile up into a backup set just as a full backup would. The only difference between a level 0 incremental
relentlesstorm Nov 7, 2020
What is patrol read

What is patrol read

What is patrolread The Patrol Read feature is designed as a preventative measure to ensure hard drive health and data integrity. Patrol Read scans for and resolves potential problems on configured hard drives. Patrol Read Behavior The following is an overview of Patrol Read behavior: * Patrol Read runs on all
relentlesstorm Nov 5, 2020
Shell, sed and awk scripting

Shell, sed and awk scripting

Shell Special variables in Shell: * $0 - The filename of the current script. * $# - The number of arguments supplied to a script. * $* - All the arguments are double quoted. If a script receives two arguments, $* is equivalent to $1 $2. * $@ - All the arguments are individually double quoted. If a
relentlesstorm Nov 1, 2020
Linux System Administration

Linux System Administration

Package management Install the package: $ rpm -ivh blktrace Upgrade the package: $ rpm -Uvh blktrace Remove the installed package: $ rpm -ev blktrace $ rpm -ev --nodeps blktrace Display the installed package info: $ rpm -qi blktrace Name : blktrace Relocations: (not relocatable) Version : 1.0.1 Vendor: Red Hat, Inc. Release : 6.el6 Build
relentlesstorm Sep 2, 2020
Sublime Jekyll Package

Sublime Jekyll Package

sublime-jekyll is a Sublime Text package for Jekyll static sites. This package is aimed at streamlining the process of managing and creating new Jekyll posts. It does this by providing quick and easy access to new post and draft commands, snippets and completions for Liquid template tags and filters, as
relentlesstorm Aug 20, 2020
Start jekyll with systemd service

Start jekyll with systemd service

Create systemd file $ cat /etc/systemd/system/jekyll.service # Description: to make `jekyll serve` a system service and start on boot # # Usage: place this file at `/etc/systemd/system/jekyll.service` # then run # sudo systemctl start jekyll.service # sudo systemctl enable jekyll.service [Unit] Description=Jekyll service After=syslog.target
relentlesstorm Aug 19, 2020
Install Jekyll on MacOS

Install Jekyll on MacOS

Install Command Line Tools First, you need to install the command-line tools to be able to compile native extensions, open a terminal and run: $ xcode-select --install Install Ruby Jekyll requires Ruby > 2.5.0. macOS Catalina 10.15 comes with ruby 2.6.3, so you’re fine. If you’
relentlesstorm Jul 20, 2020
Transform Plain Text Into Static Websites And Blogs

Transform Plain Text Into Static Websites And Blogs

Quick start instructions $ gem install bundler jekyll $ jekyll new my-awesome-site $ cd my-awesome-site $ bundle exec jekyll serve # => Now browse to http://localhost:4000 Check out the [Jekyll][jekyll] for more info on how to translate your plain text into static websites and blogs. Reference * https://jekyllrb.com/
relentlesstorm Jul 11, 2020
FlamingBytes © 2023. Powered by Ghost