Docker Distributed Application Platform

Docker is an open platform aimed at developers and system administrators. Purpose of the product: creation, delivery and launch of distributed applications.

The platform is created on the principle of container virtualization or virtualization at the operating system level, which forces the operating system kernel to support several user workspaces at the same time, so that several copies of the operating system can be run on the same physical server at the same time and they will run on the same kernel.

The Docker Engine system includes a compact, easy-to-use packaging tool, and Docker Hub, a cloud service for exchanging applications and automating workflows.

Docker helps to quickly assemble the application from components, preventing the appearance of possible roughness between development, quality assurance and operation.

The product is distributed as free software under the Apache 2.0 license. Written in Go.

History

2024

Hackers around the world turn company servers into mining farms because of a hole in Docker's container system

In early June 2024, researchers analyzed a crypto-jacking campaign and concluded that hackers around the world were turning remote API servers of companies into mining farms for cryptocurrency due to a hole in the Docker container system from the open source Commando project.

Attackers use the image image "cmd.cat/chattr" to gain access to the server and manage Docker on a compromised host. It all starts with assessing the availability of the Docker Remote API server, after which hackers initiate the process of creating a container, which makes it possible to access the host operating system. In the absence of the required image, attackers download it from the "cmd.cat" repository and create a container. The script checks the system for a specific file and, if not, downloads and runs the malicious ZiggyStarTux file - an IRC bot based on open source derived from the Kaiten (Tsunami) virus. Although the hacker C2 server was unavailable for analysis, the researchers were able to determine an attack indicator that would monitor malware activity on the network.

Hackers around the world turn remote API servers of companies into mining farms for cryptocurrency due to hole in Docker container system

It is important to note that hackers use Docker images to deploy crypto-jacking scripts on infected systems, Trend Micro experts emphasized. - This method allows attackers to exploit weaknesses in the Docker settings and evade detection by antivirus programs.

It is known that hackers have been using this method since the beginning of 2024. To protect the development environment from such attacks, you should carefully configure containers and APIs, use only official or certified images, and conduct regular security audits.^[1]

Availability for GitVerse users

GitVerse the Russian A "mirror" has become available to users. Docker Hub The developer announced this on June 3, 2024. More. here

2019

Mirantis Acquisition from Docker of Docker Enterprise Platform Business

In November 2019, Mirantis, a developer of OpenStack with Russian roots, bought from Docker Inc a business related to the Docker Enterprise platform - software for fast assembly, debugging and deployment of containerized applications in any environment.

The use of Docker is gaining popularity worldwide as interest in container virtualization grows. The platform is distributed on two models: an open source Docker Community Edition license and a subscription-based Docker Enterprise Edition license. A third of companies on the Forune 100 list use Docker Enterprise, according to Mirantis.

Mirantis co-founder and board member Alex Friedland explained to TAdviser that their company bought all the assets of the Docker Enterprise business. Intellectual property, contracts with more than 700 companies, trademark use, employees, existing and future financial flows passed to it. Mirantis will increase its staff due to the acquisition by 300 people.

After this deal, Docker left part of the business related to developer support and the products Docker Hub (container management and container image distribution platform) and Docker Desktop (microservice development environment).

As a result of this transaction, the combined company becomes the largest independent cloud-native open source company in the field of infrastructure - physical and for applications, which has the scale and resources to serve the largest companies in the world and gives the latter a solid foundation for their digital transformation. We are becoming a real alternative to VMware and IBM Red Hat, and we are also allowing our customers to decide for themselves how to allocate their resources between private and public clouds, "said Alex Friedland in a conversation with TAdviser.

Mirantis says that after the deal, the Docker Enterprise team will continue to develop and support the platform, as well as, together with the Mirantis team, will implement new features on the platform that corporate customers expect, such as the implementation of the "as a service" approach, integration with Mirantis Kubernetes and other cloud technologies.

The co-founder of Mirantis also told TAdviser that their company will provide Docker Enterprise in its existing form - as a subscription to use the software, and will also add the Managed Service option, when the product is provided as a service in which Mirantis is responsible for operation and provides the SLA client.

At the same time, Mirantis makes a special bet on Kubernetes, an open source platform for automating the deployment, scaling and management of containerized applications. Using it, customers can manage Docker containers that they take from open access. Mirantis offers Kubernetes to its customers on a "as a service" model - Kubernetes-as-a-Service (KaaS).

Kubernetes is the main vector of the company's development. Docker Enterprise has been supporting Kubernetes along with Docker Swarm for over two years. Mirantis plans to maintain and develop Docker Swarm for at least the next two years and will offer its customers a smooth transition from Swarm to Kubernetes, "Friedland explained to TAdviser.

As demand for container virtualization grows, many large IT companies and cloud providers are paying great attention to Kubernetes. For example, Amazon, Microsoft, Google and IBM are already offering support for Kubernetes for their cloud platforms. And in November 2019, VMware offered in beta a suite of tools to transform vSphere - its flagship product - into a native platform for Kubernetes clusters. Having formed an extensive portfolio of solutions and services for Kubernetes in a few years, VMware eventually reached absolute integration with the container solution.

Detecting a crypto-jacking worm spread by Docker containers

On October 17, 2019, it became known that a team of researchers from Unit 42 of the company Palo Alto Networks discovered, according to them, the first worm for spreading with cryptojacking. containers Docker

As reported, the malware, called Graboid, is downloaded from C&C servers and is designed to mine the Monero cryptocurrency. To spread, the worm periodically asks the C&C server for information about vulnerable hosts and randomly selects the next target. According to the researchers, on average, each cryptominer is active for 63% of the time, and mining periods are 250 s.

An analysis of the malicious campaign found 2,000 Docker installations connected to the Network with a missing authorization mechanism, which allows an attacker to gain full control over the Docker (Community Edition) engine and host.

During the attack, the cybercriminal can compromise the unprotected Docker daemon, then launch the malicious container from the Docker Hub, obtain scripts and a list of vulnerable hosts from the C&C server, and then repeat the operation to attack the next target.

Graboid includes both worm and cryptocurrency miner features. Each time, the malware randomly selects three targets, sets the worm on the first, stops the miner on the second and starts it on the third, creating unpredictable behavior. The malicious container does not start immediately after the host is hacked, but waits for another compromised host to start the mining process.

Essentially, the miner on each infected host is randomly controlled by all other infected hosts. The motivation for creating such a random mechanism is unclear. This can result from poor design, detection evasion technique (not very effective), a self-sufficient system, or pursue other goals. Palo Alto Networks researchers explain

According to the researchers, the malicious Docker image (pocosow/centos) for October 2019 was downloaded more than 10 thousand times from the Docker Hub. The cryptocurrency mining container that hosts the worm (gakeaws/nginx) has been downloaded more than 6,500 times. The researchers also found that user gakeaws posted a second image of cryptojacking (gakeaws/mysql), which has identical content with gakeaws/nginx.^[2]

2017

Support for Kubernetes

On October 17, 2017, Docker announced official support for Kubernetes in its platform^[3]

According to the developers' vision, the Docker software platform consists of four layers:

1. of the executable environment for launching containers (complies with the standard from OCI, Open Container Initiative),
2. of the Swarm orchestration tool ("turns a group of nodes into a distributed system"),
3. Docker Community Edition (implementation of a simple workflow for assembling and delivering applications in containers),
4. Docker Enterprise Edition.

All these layers are assembled from Open Source components through the Moby project.

Moby has been working on support for Kubernetes for Docker for more than a year and includes the necessary changes in Open Source projects containerd and cri-containerd, LinuxKit, InfraKit, libnetwork, Notary, libentilement.

The main community assembly of the project, officially released by the non-profit organization CNCF, will be used as the Kubernetes distribution integrated with Docker.

Docker Enterprise Edition

On March 3, 2017, Docker announced the release of Docker Enterprise Edition (EE), Docker's commercial platform focused on creating and managing containers, scaling hybrid clouds. The product includes a performance environment, container orchestration tools, control and security tools.

Docker Enterprise Edition Presentation, (2017)

At the same time, the company initiated a certification program that will allow third-party developers to place their applications in the Docker Store. Docker has certified EE to work with distributions^[4]

• CentOS,
• Red Hat Enterprise Linux (RHEL),
• Ubuntu,
• SUSE Linux Enterprise Server (SLES),
• Oracle Linux
• Windows Server 2016.

EE operates in AWS and Azure cloud environments.

The platform is represented by three editions: basic, standard and extended. The basic version offers infrastructure certification and support. Certified containers and plugins are available in the Docker Store. The standard version is complemented by multi-user support with the ability to serve different users in isolation, such as SaaS subscribers, within one service. This revision offers additional tools for managing containers and their images; security features are adapted to work in data centers. The extended version is equipped with antivirus and a tool for monitoring vulnerabilities.

Docker 1.13

On January 19, 2017, Docker announced the release of Docker 1.13. As part of the release, functions for creating containers, managing them and ensuring their security. Added commands with which you can organize the efficient use of disk space by containers.

The docker system prune command removes unused data, the docker system df command shows the user the amount of occupied space on the specified disk.

Squash compression helps Docker containers work more efficiently with disk space. It is made as an experimental option to the "docker build" command. The use of squash ensures the collapse (compression) of multiple layers of the file system formed during the creation of the container into a single layer.

In the Docker 1.12 release, in June 2016, it was possible to integrate the Swarm container orchestration tool directly into the Docker engine. In the Docker 1.13 update, the mode of working with Swarm is further improved.

Using the standard docker-compose command in version 1.13, the user can deploy and manage the Swarm service, set the required number of instances (nodes) for each service. Swarm mode has an improvement: it is integrated with the Secret Management API (Secret Data Management API), with which you can safely store and extract confidential data used in Docker services.

In the terminology of Docker Swarm services, the concept of "secret" (secret) is a data object, for example, a password, SSH private key, SSL certificate or other set of data that cannot be transmitted over the network or stored unencrypted in Dockerfile or in the source code of your application. Starting with Docker 1.13, with the help of Docker secrets, this data can be managed centrally and securely transferred only to containers in which this data is needed.

The release included pairs of incremental updates to mandatory access control technologies in, in Linux particular SELinux (Security Enhanced Linux) and AppArmor. In addition, security fixes made in release 1.12.6, released on January 10, 2017, will be included in version 1.13. It has a closed security hole with a CVE-2106-9962 code, the essence of which is described as "unsafe opening of the file descriptor allows you to elevate privileges" - it allowed data breach from the container.

2016: Docker 1.11

On April 14, 2016, Docker, Inc introduced the release of Docker 1.11, a tool for managing isolated Linux containers. The version provides a high-level API for container manipulation at the isolation level of individual^[5] applications^[5].

Docker 1.11 switched to lightweight runtime runC and container management tools that are compatible with OCI (Open Container Initiative) specifications, which define a single container format and a universal environment for running them.

runC provides a separate set of components for running containers on a wide range of systems, allowing you to do without external dependencies, since support for various isolation technologies is embedded in the runtime of the container. To organize work with containers, runC supports Linux namespaces (namespaces), various Linux security enhancements (SELinux, Apparmor, seccomp, cgroups, capability, pivot_root, uid/gid reset), live migration (used CRIU), Windows 10 container creation capabilities, systemd integration support and portable performance profiles (provided by Google).

Container includes a background process and a command-line client that uses runC to run OCI-compliant containers. From the advanced capabilities of containerd, support for seccomp, unprivileged containers (user namespace), the use of criu for cloning and live migration are noted. Running 1000 containers at the same time provides a launch performance of 126-140 containers per second.

The Docker engine works as an add-on to containerd, which allows you to maintain the interface familiar to Docker users. Container integration has greatly simplified Docker's codebase and eliminated a number of problems. Dividing Docker into independent layers made it easier to maintain the product and significantly improved its quality. Particular attention was paid to performance - the use of additional mechanisms for interprocess interaction not only did not lead to a slowdown in work, but also accelerated work by parallelizing container creation operations.

Changes to Docker 1.11

• Support for load balancing through distribution of requests to containers in round robin mode using DNS;
• Experimental VLAN support in the network infrastructure of containers;
• The ability to use Yubikey hardware devices to generate digital signatures for container images;
• Support for binding arbitrary labels in key/value format to networks and disk partitions by analogy with binding labels to containers and images;
• Improved handling of low disk space in device mapper-based vaults;
• The release of the Docker Compose 1.7 tool, which allows you to organize the work of an application distributed to several hosts, which involves several containers running in a cluster based on Docker Swarm. In the new version, the "docker-compose exec" command (equivalent to "docker exec") and the "--build" option for the "docker-compose up" command are added, initiating the preliminary launch of "docker-compose build";
• The release of the Machine 0.7 tool, designed to quickly deploy hosts in guest environments of systems, virtualizations VirtualBox,, VMware AWS Digital Ocean and. Microsoft Azure Creates a server filling, installs Docker on it and configures the client to work with this server. In the new version, the driver Microsoft Azure has been transferred to the new Azure API;
• Release of the Swarm 1.2 tool, which provides clustering tools for packaged applications. Swarm allows you to manage a cluster of several Docker hosts (for example, created using the Docker Machine) in the form of working with a single virtual host. Since Swarm uses the standard Docker API, it can be used to control other tools that support this API, such as dokku, fig, krane, flynn, deis, docker-ui, shipyard, drone.io, Jenkins. The new version has stabilized support for redevelopment, which provides automatic movement of the container to the working node in the event of a failure of the current node.

2015: Docker 1.8 Release Released

On August 12, 2015, Docker released Docker 1.8. The product contains a high-level API for managing containers at the isolation level of individual applications^[6]

Docker developers noted the ability of the product to run arbitrary processes in isolation mode and then transfer and clone containers formed for these processes to other servers, not caring about the formation of the container filling, taking on all the work of creating, maintaining and maintaining containers. Docker code is written in Go and distributed under the Apache 2.0 license.

Interaction Scheme, 2015

The toolkit is based on the use of standard namespaces-based isolation mechanisms and management groups built into the Linux kernel. To create containers, it is proposed to use libcontainer (wrapper over namespaces and cgroups), it is also possible to use lxc, libvirt, systemd-nspawn, OpenVZ containers using the LibCT library and other isolation systems. To form a container, it is enough to load the basic environment image (docker pull base), after which you can run arbitrary applications in isolated environments (for example, to run bash, you can run "docker run -i -t base/bin/bash").

The most notable innovations in Docker 1.8:

• The Docker Content Trust functionality to verify the authenticity of the container image by digital signature allows you to make sure that the image is located in the repository by the declared publisher. For verification, a public key system is used, in which the image is signed with the publisher's private key, and then can be verified using a publicly available public key. For publishing, verifying and securely updating images, Docker integrates the Notary toolkit, which in turn is based on The Update Framework (TUF). Verification is carried out automatically when executing typical commands such as docker pull, docker push, docker build, docker create and docker run;

• Docker Toolbox is presented, a specialized installer for Windows and, OS X which simplifies the deployment and launch of the Docker developer environment. Docker Toolbox is positioned as a replacement for Boot2Docker and includes a client ON for Docker, Machine and Compose components, as well as a system; virtualizations VirtualBox

• The experimental system added in the previous release for connecting plugins executed in the form of separate processing processes has been transferred to the category of stable ones. Storage plugins have also been transferred to the stable category, for example, allowing you to work with network storage such as Flocker, Blockbridge, Ceph, ClusterHQ, EMC and Portworx;

• The system of drivers for logging, which allows implementing various schemes for storing the system log, including transferring the container logs to an external syslog server, is expanded with the ability to transfer logs to Graylog and Fluentd systems. Added a driver for organizing the rotation of logs on disk;

• The docker cp command can now be used not only to copy files from the container to the host system, but also vice versa. For example, "docker cp foo.txt mycontainer :/foo.txt";

• To run the Docker daemon, a new "docker daemon" command is presented, which should be used instead of the "-d" option. The new command allows you to explicitly separate client options (docker --help) and daemon options (docker daemon --help);

• The ability to configure the output format of the "docker ps" command by specifying the "--format" option;

• Support for configuring the directory with client configuration files by specifying the path in the --config option or the DOCKER_CONFIG environment variable, which makes it possible to run different instances of docker with different configuration sets;

• Release of the Machine 0.4 tool, designed to quickly deploy hosts in guest environments of virtualization systems,,, VirtualBox VMware AWS Digital Ocean and. Microsoft Azure Creates a server filling, installs Docker on it and configures the client to work with this server. The new version has added engine configuration tools for using http proxies;

• Release the Swarm 0.4 tool, which provides clustering tools for packaged applications. Swarm allows you to manage a cluster of several Docker hosts (for example, created using the Docker Machine) in the form of working with a single virtual host. Since Swarm uses the standard Docker API, it can be used to control other tools that support this API, such as dokku, fig, krane, flynn, deis, docker-ui, shipyard, drone.io, Jenkins. The new version has improved the implementation of the built-in scheduler and driver for integration with Mesos (now you can use docker tools to manage the Mesos cluster);

• The release of the Docker Compose 1.4 tool, which allows you to organize the work of an application distributed to several hosts, which involves several containers running in a cluster based on Docker Swarm. In this version, the speed of launching and stopping applications has been significantly increased, the container is re-created only if necessary, parallel work is ensured. Added the ability to assign arbitrary names to containers and support reading the configuration from standard input (you can generate a configuration file on the fly);

2014

Microsoft ports Docker to Windows Server and Windows Azure

On October 22, 2014, Microsoft announced the upcoming migration of Docker platform code to Windows Server and Windows Azure.

The Docker platform is designed for the Linux operating system and uses container virtualization capabilities deeply integrated into the Linux kernel. Porting such software to Windows is not an easy task, but Microsoft is going to do it. The company announced its intention to support and fund the efforts of Open Source developers and implement support for the Docker API in its Windows Azure cloud service.

In June 2014, it became possible to launch Docker hosts on Windows Azure. The user can create a virtual machine in the Microsoft cloud, install Linux on it, run the Docker environment and own applications in it. However, the company seeks to ensure that tasks run on Windows hosts, both in the Azure cloud and in any other public and private environments.

Analysts noted the company's desire to call this technology "Windows containers," thereby noting the likelihood of a significant difference between the internal device of the Windows version of Docker and the original. However, the company promises that all applications running in the Docker environment on October 22, 2014 will work in a Windows environment.

As of October 22, 2014, 45 thousand ready-made images of various applications can be launched in the Docker virtual environment.

The Open Source community noted a feature that became, in a sense, a victory for the community: to provide Docker support, key especially Linux kernels will be implemented inside Windows.

Docker 1.3.2: Fixed Critical Bugs

In the fall of 2014, an extraordinary update of the open container virtualization management system Docker 1.3.2 took place, caused by the need to correct two errors that received critical status. The discovered vulnerabilities allowed an attacker to gain access to the host's file system outside the container, which is extremely dangerous^[7].

The first vulnerability was assigned a CVE-2014-6407 identifier. It allows an attacker to move files from the container to the file system of the host machine using the docker pull or docker load commands. This is due to an error in the processing of hard and symbolic links in the program for extracting data from the container image. As a result, an attacker will be able to execute arbitrary code on the host system and change his own access rights.

The error is contained in all previous versions of Docker. Therefore, the update is mandatory.

The second vulnerability has a CVE-2014-6408 identifier. The error allows you to ignore the restrictions set on an isolated container, which also allows an attacker to go beyond it. This is because you have the ability to assign your own security settings to the image and change the overall security profile for all containers that rely on the same image.

The error was found in releases 1.3.0 and 1.3.1. So, in practice, updates are mandatory here too - hardly anyone uses earlier releases.

Works to enable Docker support on Apache Hadoop

Since 2014, work has been underway to include Docker support in the distributed application framework management environment; Hadoop according to the results of testing of platform options virtualizations for Hadoop, conducted in May 2014, Docker showed on the main operations (for mass creation, restart and destruction of virtual nodes) significantly higher performance than, in KVM particular, on the test of mass creation of virtual computing nodes, the increase in the consumption of processor resources in Docker was recorded 26 times lower than in KVM, and the increase in the consumption of RAM resources - three times lower.

2013

Docker Container Support in the Google Compute Engine

In December 2013, it announced support for deploying Docker containers in the Google Compute Engine environment.

Partial Docker support in Red Hat Enterprise Linux 6.5, full in Fedora version 20

Since November 2013, partial support for Docker has been included in the Red Hat Enterprise Linux version 6.5 distribution, the full one in the 20 version of the Fedora distribution, and an agreement was previously reached with Red Hat to include Docker in the replicated PaaS platform Open Shift since 2014.

Docker support in OpenStack (Havana release)

In October 2013, the Havana release of the replicated IaaS platform was released, OpenStack which implements Docker support (as a driver for OpenStack Nova).

Publishing Docker under the Apache 2.0 license

In March 2013, Docker code was published under the Apache 2.0 license. In October 2013, emphasizing the shift of focus to the new key technology, dotCloud was renamed Docker (the PaaS platform is saved under the name - dotCloud).

2008: Project Launch

The project began in 2008, as an internal development of dotCloud, with the aim of creating a public PaaS platform with support for various programming languages.

Notes

Links

1. Docker

2. Microsoft promised an open application virtualization platform