Docker Deep Dive: Understanding the Fundamentals of containers
As a developer, you know how important it is to have a reliable and efficient workflow. That's why many developers love Docker - it's a powerful containerization platform that makes it easy to package, deploy, and manage applications and their dependencies. by using containers. Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package.
Docker uses the resource isolation features of the Linux kernel such as cgroups and kernel namespaces, and a union-capable file system such as OverlayFS and others to allow independent "containers" to run within a single Linux instance, avoiding the overhead of starting and maintaining virtual machines.
cgroups
![](https://abdelouahabmbarki.com/content/images/2022/12/Screenshot-from-2022-12-21-20-58-35.png)
Cgroups are a Linux kernel feature that allows you to allocate resources (such as CPU, memory, and I/O bandwidth) to processes or groups of processes. Docker uses cgroups to limit the resources that each container can use, ensuring that containers do not interfere with each other or with the host system.
To demonstrate how Docker uses cgroups (Control Groups) in Linux, we can create a simple Docker container that limits the CPU and memory usage of a process. Here is an example of how to do this:
Run the Docker container with the docker run
command, using the --cpu-shares
and --memory
options to limit the CPU and memory usage of the container and the --device-read-bps
and --device-write-bps
options to specify the maximum read and write bandwidth in bytes per second that the container is allowed to use for all block devices.
To get the block device name Use the df
command
df
docker run -d --name demo-cgroups --device-read-bps /dev/your-device:500m --device-write-bps /dev/your-device:500m --cpu-shares 512 --memory 512m ubuntu:latest sleep infinity
This will run the Docker container with a CPU share of 512 and a memory limit of 512 MB. limit the read and write bandwidth of a Docker container to 500 MB/s for the block device specified /dev/your-device
You can use the docker stats
command to view the I/O bandwidth usage of the container
docker stats demo-cgroups
![](https://abdelouahabmbarki.com/content/images/2022/12/image-6.png)
Playing with cgroups in Linux:
You can use the cgroup
command line utility to manage cgroups in Linux you can install it on debian based distro using apt-get install cgroup-tools
. Here is how we can use cgroup
to manipulate cgroups:
To create a new cgroup, use the cgcreate
command:
sudo cgcreate -g memory:/mycgroup
This will create a new cgroup named "mycgroup" in the "memory" hierarchy.
Create a demo process to test the cgroup
sleep infinity &
To move a process into a cgroup, use the cgclassify
command:
sudo cgclassify -g memory:/mycgroup YOUR_PROCESS_ID
This will move process with PID YOUR_PROCESS_ID into the "mycgroup" cgroup.
To set a resource limit for a cgroup, use the cgset
command:
sudo cgset -r memory.limit_in_bytes=256M mycgroup
This will set a memory limit of 256MB for the "mycgroup" cgroup.
To view the status of a cgroup, use the cgget
command:
sudo cgget -r memory.usage_in_bytes mycgroup
This will display the current memory usage for the "mycgroup" cgroup.
More information about cgroups can be found here
Kernel namespaces
![](https://abdelouahabmbarki.com/content/images/2022/12/Screenshot-from-2022-12-21-21-46-37.png)
Kernel namespaces are a Linux kernel feature that allow you to create isolated environments within a single Linux instance. Docker uses kernel namespaces to create separate namespaces for each container, allowing each container to have its own view of the system. This isolation ensures that containers do not interfere with each other or with the host system.
To demonstrate how Docker uses namespaces in Linux, we can create a simple Docker container that runs a process in a separate namespace. Here is an example of how to do this:
Run a container for the test with :
docker run -it --name demo-ns ubuntu:latest /bin/bash
You can verify that the process is running in a separate namespace by using the ps
command inside and outside the container:
# Inside the container
ps aux
![](https://abdelouahabmbarki.com/content/images/2022/12/image-7.png)
# Outside the container
docker exec demo ps aux
![](https://abdelouahabmbarki.com/content/images/2022/12/image-8.png)
The output of the ps
command will show the PID of the process running in the container. You should see that the PID is different inside and outside the container, indicating that the process is running in a separate namespace.
More information about how to manipulate docker container pid namespace can be here
Useful resources to dig deep into Linux namespaces : https://blog.quarkslab.com/digging-into-linux-namespaces-part-1.html
Union file systems
Union file systems, such as OverlayFS, allow you to overlay one file system on top of another, creating a single logical file system that combines the contents of both. Docker uses union file systems to create lightweight containers that share a common base image, reducing the disk space and bandwidth required to store and transfer containers.
![](https://abdelouahabmbarki.com/content/images/2022/12/Screenshot-from-2022-12-21-22-00-45.png)
OverlayFS allows you to overlay one file system on top of another, creating a single logical file system that combines the contents of both. This can be useful in a variety of situations, such as creating lightweight containers that share a common base image or allowing multiple users to share the same file system without modifying the underlying files
To demonstrate how Docker uses union file systems in Linux, we can create a simple Docker container that uses an OverlayFS union file system to combine two different filesystems
Create a temporary directory and add a file to it:
mkdir temp
echo "overlay" > temp/hostname
This will create a temporary directory and add a file named hostname
to it, containing the text "overlay".
Run the Docker container with the docker run
command, using the --mount
option to mount the temporary directory as an OverlayFS layer on top of the base image:
docker run -it --name demo-ufs -v $(pwd)/temp/hostname:/etc/hostname ubuntu:latest cat /etc/hostname
This will run the Docker container with the temporary directory mounted as an OverlayFS layer on top of the base image. The /etc/hostname
file in the base image will be overlaid with the hostname
file in the temporary directory, and the cat
command will display the contents of the overlaid file.
You can verify that the overlay is being used
![](https://abdelouahabmbarki.com/content/images/2022/12/image-9.png)
Useful resources to dig deep into OverlayFS : https://adil.medium.com/container-filesystem-under-the-hood-overlayfs-5a8053fe3a0a
conclusion
In conclusion, Docker is a powerful containerization platform that has revolutionized the way developers build, deploy, and manage applications These Linux kernel features motioned on that post are essential for the functionality and performance of Docker, and they play a crucial role in enabling developers to build, deploy, and manage applications in containers. Docker has become an industry standard for containerization, and it is used by millions of developers and organizations around the world to build, deploy, and manage applications in a fast, efficient, and scalable manner.