In this article, we will discover Docker Engine’s architecture and core components.

Docker initially became available to the public in 2013. The Docker Engine comprises three main elements: the Docker daemon, the REST API, and the Docker CLI.

The Docker daemon acts as the server or process responsible for creating and managing objects, such as images, containers, volumes, and networks on a host. The REST API offers an interface to manipulate objects in Docker. The Docker CLI serves as the command line interface used to execute commands and manage objects within Docker.

How does Docker handle container management on a host?

Previously, Docker utilized a technology called Linux containers or LXC to manage containers on Linux. LXC used capabilities of the Linux kernel, such as namespaces and cgroups, to create isolated environments on Linux, commonly referred to as containers. Working directly with LXC required considerable effort for an average user. However, Docker simplified container management by providing a set of tools. In version 0.9, Docker introduced its execution environment, called libcontainer. Written in Go, the same programming language as Docker, libcontainer reduced Docker’s reliance on the kernel’s LXC technology. With libcontainer, Docker could directly interact with the Linux kernel features, such as namespaces and cgroups. Consequently, libcontainer replaced LXC as Docker’s default execution environment.

Until early 2015, there were no standardized guidelines for container software. The Open Container Initiative (OCI) was formed by Docker, CoreOS, and other industry leaders to address this. The OCI aimed to create open industry standards for container formats and runtime. For this purpose, the OCI developed two specifications: the runtime and image specifications.

The runtime specification defines the life cycle of a container technology. It outlines various commands, such as creating a container, starting it, and deleting it. Many more standards are detailed in this specification, accessible in the respective GitHub repository.

The new Architecture of Docker Engine:

Before the OCI standards were in place, the Docker daemon functioned as a single large monolithic code base, performing multiple functions like running containers, managing networks, volumes, and images on the Docker host, as well as handling image pushing and pulling from Docker repositories. With the OCI standards in version 1.11, the architecture of the Docker Engine was refactored. It was broken down into smaller, reusable components.

The part responsible for running containers became an independent component called runC. runC marked the first OCI-based technology and was contributed by Docker to the Open Compute Project Foundation. Now, containers could be run solely by installing runC without requiring Docker, though some core Docker features would be unavailable. The daemon that managed containers became a separate component known as containerd. It now oversees runC, which utilizes libcontainer to create containers on a host.

When the Docker daemon shuts down or is restarted, a new component called containerd-shim comes into play to ensure the containers are adequately handled during this period. Containerd-shim was introduced to enable containers to function independently of the daemon, thus making them “daemonless.” It takes care of managing the containers and monitoring their state, ensuring they continue to run even when the Docker daemon is temporarily unavailable, and are reattached when the daemon becomes operational again.

Docker Components (Objects):

Docker Engine manages four primary objects: Docker images, containers, networks, and volumes.

Docker images serve as read-only templates for creating Docker containers. These images can be base images of operating systems or applications, like web servers or databases. Users can also create their own images for their applications.

A container is a running instance of an image. Alternatively, a container can be seen as a read-write template. Users can create, start, stop, move, or delete a container using the Docker CLI or the API.

Docker networks facilitate various solutions to enable containers to communicate with each other and the external world.

Containers are designed to be temporary, which implies that the data inside a container is not permanent and is bound to the container’s lifecycle. Volumes are utilized to ensure data remains intact and preserved, even when containers are restarted or reinitialized. Volumes provide a means to store data separately from the container itself, allowing data to survive container restarts and ensuring its availability beyond the container’s lifetime.

A registry stores Docker images and allows users to publish and share images publicly or within a private organization. Docker Hub is a public registry accessible to anyone, and Docker is configured to search for images on Docker Hub by default. Users can also set up and run their private registry. Docker Enterprise Edition includes a private, trusted registry called Docker Trusted Registry (DTR).

Conclusion

Let’s put it all together. When executing a command using the Docker CLI to create a container, the Docker client converts the command into a RESTful API and sends it to the Docker daemon. Upon receiving the instruction, the Docker daemon first checks if the required image is available locally. If not, the image is downloaded from the Docker registry (by default, Docker Hub). Once the image is downloaded, the Docker daemon calls containerd to start the container. Containerd then converts the downloaded image into an OCI-compliant bundle and passes it to containerd-shim, which utilizes runC to initiate the container. runC interacts with the kernel’s namespaces and cgroups to create the container, thus completing the container creation process.