Docker the Platform of the distributed applications

Docker is the open platform, it is focused on developers and system administrators. Purpose of a product: creation, delivery and start of the distributed applications.

The platform is created on the principle of container virtualization or virtualization at the level of the operating system which forces operating system kernel to support several working spaces of users at the same time, so, that on one physical server it is possible to start at the same time several copies of the operating system and they will work at one core.

In composition of Docker Engine - the compact, easy instrument of packaging, and Docker Hub, a cloud service in work for exchange of applications and automation of workflows.

Docker helps to collect quickly the application from components, preventing emergence of possible roughnesses between development, quality assurance and operation.

The product extends as the open source under the license Apache 2.0. It is written in the Go language.

History

2019

Acquisition of Mirantis at Docker of the business connected with the Docker Enterprise platform

In November, 2019 Mirantis, OpenStack developer with the Russian roots, redeemed from Docker Inc company the business connected with the Docker Enterprise platform – software for fast assembly, debugging and deployment of the containerized applications in any environment.

Use of Docker gains popularity around the world as interest in container virtualization grows. The platform extends on two models: according to the license with the open code Docker Community Edition and the licenses Docker Enterprise Edition on the basis of a subscription. According to Mirantis, a third of the companies from the Forune 100 list uses Docker Enterprise.

The co-founder and the board member of Mirantis Alex Friedland explained TAdviser that their company purchased all assets of business of Docker Enterprise. The intellectual property, contracts with more than 700 companies, use of a trademark, the employees existing and future financial flows passed to it. The staff of Mirantis in connection with acquisition will be increased on 300 people.

Docker after this transaction had part of business connected with support of developers and the products Docker Hub (the platform on management of containers and distribution of images of containers) and Docker Desktop (a development environment of microservices).

As a result of this transaction the united company becomes the largest independent cloud-native the company with the open code in the field of infrastructure - physical and for applications which has scales and resources for service of the largest companies of the world and gives the last a strong basis for their digital transformation. We become a real alternative of VMware and IBM Red Hat and also we allow our clients to decide how to distribute their resources between private and public clouds, - Alex Friedland in a conversation with TAdviser said.

Mirantis says that after the transaction the Docker Enterprise command will continue development and support of the platform and also together with the Mirantis command will be engaged in implementation on the platform of new opportunities which are expected by corporate clients as, for example, approach implementation "as service", integration with Mirantis Kubernetes and other cloud computing.

The co-founder of Mirantis also reported to TAdviser that their company will provide Docker Enterprise as it stands – in the form of a subscription to use of software and also will add the option Managed Service when the product is provided as service at which Mirantis is responsible for operation and provides to the client of SLA.

At the same time on Kubernetes - the platform with the open code for automation of deployment, scaling and management of the containerized applications - in Mirantis do a special rate. Using it, clients can manage Docker-containers which they take from open access. Mirantis offers Kubernetes to the clients on model "as service" - Kubernetes-as-a-Service (KaaS).

Kubernetes is the main vector of development of the company. Docker Enterprise supports more than two years Kubernetes along with Docker Swarm. Mirantis is going to support and develop Docker Swarm within at least next two years and will offer the clients soft junction from Swarm on Kubernetes, - Friedland explained TAdviser.

In process of increase in demand for container virtualization much attention of Kubernetes is paid by many large IT companies and cloud providers. So, for example, Amazon, Microsoft, Google and IBM already offer support of Kubernetes for the cloud platforms. And in November, 2019 VMware offered in the beta a set of means for the vSphere conversion – the flagship product – to the native platform for Kubernetes clusters. Having built for several years an extensive solution portfolio and services for Kubernetes, VMware as a result reached absolute integration into the container solution.

Detection of a worm for the kriptodzheking extending using the containers Docker

On October 17, 2019 it became known of what a team of researchers from Unit 42 companies Palo Alto Networks detected, according to them, the first worm for kriptodzheking extending with the help containers Docker.

As it was reported, the malware which received the name Graboid is loaded with C&C-серверов and is intended for Monero cryptocurrency mining. For distribution the worm periodically requests at C&C-сервера information on vulnerable hosts and in a random way selects the following purpose. According to researchers, on average each cryptominer is active throughout 63% of time, and the periods of mining are 250 pages.

During the analysis of a harmful campaign 2 thousand Docker installations connected to Network with the absent authorization mechanism were revealed what allows the malefactor to receive full control over the Docker engine (Community Edition) and a host.

In the course to the attack the cybercriminal can compromise the unprotected demon of Docker then to start a harmful container from Docker Hub, to receive scripts and the list of vulnerable hosts from C&C-сервера, and then to repeat transaction for an attack on the following purpose.

Graboid includes functions of both a worm, and the miner of cryptocurrency. Every time a malware in a random way selects three purposes, the worm on the first sets, stops operation of the miner on the second and starts it on the third, creating unpredictable behavior. The harmful container is not started right after cracking of a host, and expects when other compromised host begins mining process.

In fact, the miner on each infected host in a random way is controlled by all other infected hosts. The motivation of creation of such accidental mechanism is not clear. It can be result of bad design, technology of deviation from detection (not by a really effective), self-sufficient system or to pursue other aims. researchers of Palo Alto Networks explain

According to researchers, the harmful image of Docker (pocosow/centos) for October, 2019 was loaded more than 10 thousand times from Docker Hub. The container for cryptocurrency mining which is placed by a worm (gakeaws/nginx) was loaded more than 6500 times. Researchers also found out that the user of gakeaws published the second image of a kriptodzheking (gakeaws/mysql) which has the identical contents with gakeaws/nginx.^[1]

2017

Kubernetes support

On October 17, 2017 the Docker company announced official support of Kubernetes in the platform^[2].

According to vision of developers, the software platform of Docker consists of four layers:

1. the performed environment for start of the containers containerd (Open Container Initiative conforms to the standard from OCI),
2. the tool of the orchestration Swarm ("turns group of nodes into distributed system"),
3. Docker Community Edition (implementation of simple workflow on assembly and delivery of applications in containers),
4. Docker Enterprise Edition (safe management of containers in production).

All these layers gather from Open Source-components by means of the Moby project.

Work in Moby on support of Kubernetes for Docker is conducted over a year and includes necessary changes in Open Source-проектах containerd and cri-containerd, LinuxKit, InfraKit, libnetwork, Notary, libentitlement.

As the Kubernetes distribution kit integrated with Docker the main community-assembly of the project which is officially released by CNCF non-profit organization will be used.

Docker Enterprise Edition

On March 3, 2017 the Docker company announced release of Docker Enterprise Edition (EE) - the commercial Docker platform focused on creation and management of containers, scalings of hybrid cloud environments. As a part of a product the execution environment, tools of the orchestration of containers, management tools and safety.

Representation of Docker Enterprise Edition, (2017)

At the same time the company initiated the program of certification which will allow third-party developers to place the applications in Docker Store. Docker certified EE for work with^[3]:

• CentOS,
• Red Hat Enterprise Linux (RHEL),
• Ubuntu,
• SUSE Linux Enterprise Server (SLES),
• Oracle Linux
• Windows Server 2016.

EE works in cloud environments AWS and Azure.

The platform is submitted by three editions: basic, standard and expanded. In the basic version certification of infrastructure and support is offered. The certified containers and plug-ins are available in Docker Store. The standard version is added with multi-user support with an opportunity to separately service different users, for example subscribers of SaaS, within one service. Within this edition additional tools for management of containers and their images are offered; security protections are adapted for work in DPCs. The expanded version is completed with an antivirus and the tool for monitoring of vulnerabilities.

Docker 1.13

On January 19, 2017 the Docker company announced release of release of Docker 1.13. As a part of function release for creation of containers, management of them and ensuring their security. Commands using which it is possible to organize effective use of disk space containers are added.

The "docker system prune" command deletes not used data, the "docker system df" command shows to the user the volume of the occupied space on the set disk.

Function of compression squash helps to work more effectively with a disk space to the containers Docker. It is executed in the form of an experimental option to the "docker build" command. Use of squash provides collapse (compression) of the multiple layers of the file system which are formed in the course of creation of a container in a uniform layer.

In release of Docker 1.12, in June of the 2016th, there was an ability to integrate means of the orchestration of the containers Swarm directly in the Docker engine. In updating of Docker 1.13 the operation mode from Swarm is in addition improved.

By means of the standard docker-compose command in version 1.13 the user can deploy the Swarm service and manage it, set the required number of copies (nodes) for each service. In the Swarm mode improvement appeared: it is integrated with Secret Management API (API for management of confidential data) through which it is possible to store and take safely the confidential data used in Docker services.

In terminology of Docker Swarm services the concept "secret" (secret) represents a data object, for example, the password, the personal SSH key, the SSL certificate or other data set which cannot be transferred on network or to store in not encrypted form in Dockerfile or in the source code of your application. Starting with the version of Docker 1.13 by means of Docker secrets these data can be managed on a centralized basis and to transfer by a safe image them only to those containers in which there is a need for these data.

SELinux (Security Enhanced Linux) and AppArmor was a part of release couples of incremental updates of technologies of mandatory access control in Linux, in particular. In addition, will pass security corrections made in release 1.12.6 which is released on January 10, 2017 into version 1.13. In it the hole in security with CVE-2106-9962 code which essence is described as "unsafe opening of a file descriptor is closed allows to increase the level of privileges" — it allowed date leak from a container.

2016: Docker 1.11

On April 14, 2016 Docker, Inc provided tools release for management isolated by Linux containers Docker 1.11. Within the version high-level API is provided for manipulation with containers at the isolation level of the separate^[4].

Docker 1.11 passed to the lightweight runtime runC and the managing containerd tools compatible to the OCI specifications (Open Container Initiative) defining a uniform format of containers and the universal environment of an environment for their start.

runC provides the isolated set of components for start of containers on a broad spectrum of systems, allowing to do without external dependences as support of different technologies of isolation is built in container runtime. For the organization of work with the containers runC supports namespaces of Linux (namespaces), different means of increase in security of Linux (SELinux, Apparmor, seccomp, cgroups, capability, pivot_root, reset of uid/gid), live-migration (CRIU is used), possibilities of creation of containers in Windows 10, support of integration about systemd and transferable profiles of performance (are provided to Google).

Containerd the background process and the client of the command line using runC for start of the containers meeting the OCI specification includes. From enhanced capabilities of containerd support of seccomp, unprivileged containers (user namespace), application of criu for cloning and live-migration is noted. At simultaneous start of 1000 containers containerd provides start performance in 126-140 containers per second.

The Docker engine works as a superstructure on containerd that allows to save the interface, usual for users of Docker. Integration of containerd significantly simplified code base of Docker and saved from a number of problems. Separation of Docker into independent layers simplified maintenance of a product and significantly increased its quality. Special attention was paid to performance - involvement of additional mechanisms of inter-process communication not only did not lead to work deceleration, but also accelerated work thanks to parallelization of transactions of creation of containers.

Changes in Docker 1.11

• Support of balancing of loading through distribution of requests to containers in the "round robin" mode using DNS;
• Experimental support of VLAN in network infrastructure of containers;
• Possibility of use of Yubikey hardware devices for formation of digital signatures for images of containers;
• Support of a binding of any tags in a format key/value to networks and disk sections by analogy with a binding of tags to containers and images;
• Processing of a status of shortage of disk space in storages based on device mapper is improved;
• Release of the Docker Compose 1.7 tool allowing to organize work of the application distributed per several hosts in which work it is involved several containers started in a cluster based on Docker Swarm. In the new version it is added the "docker-compose exec" command (an analog of "docker exec") and an option "-build" for the "docker-compose up" command, "docker-compose build" initiating preliminary start;
• Release of the Machine 0.7 tool intended for fast deployment of hosts in guest environments of the systems of virtualization of VirtualBox, VMware, AWS, Digital Ocean and Microsoft Azure. Performs creation of a fittings of the server, installation on it Docker and setup of the client for work with this server. In the new version the Microsoft Azure driver is translated to new Azure API;
• Release of the Swarm 1.2 tool providing means of a clustering for the applications packed into containers. Swarm gives the chance to manage a cluster from several hosts of Docker (for example, created using Docker Machine) in the form of work with one virtual host. As Swarm uses regular Docker API, it can be applied to management and other supporting given API tools, such as dokku, fig, krane, flynn, deis, docker-ui, shipyard, drone.io, Jenkins. In the new version support of the re-planning providing automatic movement of a container of a working node in case of failure of the current node is stabilized.

2015: The release of Docker 1.8 is released

On August 12, 2015 the Docker company provided release of the Docker 1.8 tools. The product supports high-level API^[5] to management of containers at the isolation level of separate applications^[5].

Docker developers noted a product capability to start any processes in isolation mode and then to transfer and clone the containers created for these processes on other servers, without caring for formation of a stuffing of a container, undertaking all work on creation, service and maintenance of containers. The Docker code is written in the Go language and is distributed under the license Apache 2.0.

Scheme of interaction, 2015

Tools are based on use of the regular mechanisms of isolation which are built in a kernel of Linux on the basis of namespaces (namespaces) and management teams (cgroups). For creation of containers it is offered to use libcontainer (a wrapper over namespaces and cgroups), application of lxc, libvirt, systemd-nspawn, OpenVZ of containers using LibCT library and other systems of isolation is also possible. For formation of a container it is enough to load a basic image of an environment (docker pull base) then it is possible to start any applications in the isolated environments (for example, for start of bash it is possible to execute "docker run-i-t base/bin/bash").

The most noticeable innovations in Docker 1.8:

• The functionality of Docker Content Trust for check of reliability of an image of a container according to the digital signature, allows to make sure that the image is placed in a repository by the stated publisher. For verification the system of public keys at which the image is signed by private key of the publisher is used, and then it can be checked by means of publicly available public key. For the publication, verification and safe updating of images the Notary tools based, in turn, on a framework of The Update Framework (TUF) are integrated into Docker. Check is performed automatically at accomplishment of standard commands, such as docker pull, docker push, docker build, docker create and docker run;

• Docker Toolbox, the specialized installer for Windows and OS X simplifying deployment and start of an environment of Docker developer is provided. Docker Toolbox is positioned as replacement of Boot2Docker and the client software for Docker, the Machine and Compose components and also the system of virtualization VirtualBox includes;

• The experimental system added in last release for connection of the plug-ins executed in the form of separate processes processors is transferred to discharge of stable. Plug-ins for the organization of storages, for example, allowing to work with network storages, such as Flocker, Blockbridge, Ceph, ClusterHQ, EMC and Portworx are also transferred to discharge stable;

• The system of drivers for maintaining the logs allowing to implement different schemes of preserving of the system magazine including transfer of logs of a container on the external syslog-server, is expanded a possibility of transfer of logs to the Graylog and Fluentd systems. The driver for the organization of rotation of logs on a disk is added;

• The "docker cp" command can be applied not only to copying of files from a container per host system now, but also on the contrary. For example, "docker cp foo.txt mycontainer:/foo.txt";

• For start of the demon of Docker the new command "docker daemon" which should be used instead of an option "-d is provided". The new command allows to separate obviously client options (docker - help) and options of the demon (docker daemon - help);

• Possibility of setup of an output format of the "docker ps" command through the indication of an option "-format";

• Support of setup of a directory with files of a configuration of the client through the indication of a way to options is config or a variable of an environment DOCKER_CONFIG that gives the chance to start different copies of docker with different sets of a configuration;

• Release of the Machine 0.4 tool intended for fast deployment of hosts in guest environments of the systems of virtualization of VirtualBox, VMware, AWS, Digital Ocean and Microsoft Azure. Performs creation of a fittings of the server, installation on it Docker and setup of the client for work with this server. In the new version engine customizers for use of a http-proxy are added;

• Release of the Swarm 0.4 tool providing means of a clustering for the applications packed into containers. Swarm gives the chance to manage a cluster from several hosts of Docker (for example, created using Docker Machine) in the form of work with one virtual host. As Swarm uses regular Docker API, it can be applied to management and other supporting given API tools, such as dokku, fig, krane, flynn, deis, docker-ui, shipyard, drone.io, Jenkins. In the new version implementation of the built-in scheduler and the driver for integration with Mesos is improved (now it is possible to use the docker tools for management of Mesos cluster);

• Release of the Docker Compose 1.4 tool allowing to organize work of the application distributed per several hosts in which work it is involved several containers started in a cluster based on Docker Swarm. In this version the speed of start and a stop of applications is considerably increased, recreation of a container is made only if necessary, side-by-side execution of works is provided. The feature for purpose of any names to containers and support of reading a configuration from standard input is added (it is possible to generate the file of a configuration on the fly);

2014

Microsoft ports Docker on Windows Server and Windows Azure

On October 22, 2014 Microsoft announced the forthcoming transfer of the code of the Docker platform on Windows Server and Windows Azure.

The Docker platform is developed for the Linux operating system and uses the possibilities of container virtualization which are deeply integrated into Linux core. Port it software on Windows - a difficult task, however Microsoft is going to make it. The company expressed the intention to support and finance efforts of Open Source-developer and to implement support of API Docker in the cloud service of Windows Azure.

In June, 2014 the possibility of start of Docker-hosts on Windows Azure appeared. The user can create the virtual machine in a cloud of Microsoft, set on it Linux, launch the Docker environment and in it - the applications. However, the company aims to provide start of tasks on Windows hosts, both in Azure cloud, and in any other public and private environments.

Analysts noted the aspiration of the company to call this technology "containers Windows", thereby noting the probability of essential difference of the internal Windows version Docker device from the original. However, the company promises - all applications started in the environment of Docker for October 22, 2014 will work in Windows environment.

For October 22, 2014 in virtual environment of Docker it is possible to start 45 thousand ready images of different applications.

Open Source community noted feature which became, somewhat, a victory of community: for ensuring support of Docker in Windows, in fact, Linux kernels key especially will be implemented.

Docker 1.3.2: critical errors are corrected

In the fall of 2014 the exit of extraordinary updating of open system of management of container virtualization of Docker 1.3.2 caused by need of correction of two errors which received the status of critical took place. The detected vulnerabilities allowed the malefactor to get access to the file system of the host machine outside a container that^[6].

Identifier CVE-2014-6407 was appropriated to the first vulnerability. It allows the malefactor to move files from a container to the file system of the host machine using the docker pull or docker load commands. It is possible because of an error when processing tough and symbolical links in the program of extraction of data from an image of a container. As a result the malefactor will be able to execute any code in a host system and to change own access rights.

The error contains in all previous versions of Docker. Thus, updating is obligatory.

The second vulnerability has identifier CVE-2014-6408. The error allows to ignore the restrictions set for the isolated container that also allows the malefactor to go beyond his limits. It is connected with the fact that the user has an opportunity to appropriate to an image own parameters of security and to change the general profile of security for all containers relying on a similar image.

The error is found in releases with numbers 1.3.0 and 1.3.1. So, in practice and here updates is obligatory — hardly someone uses earlier releases.

Works on inclusion of support of Docker in the Apache Hadoop environment

Since 2014 works on inclusion of support of Docker on Wednesday of management of a framework of the distributed applications of Hadoop are conducted; based on testing of versions of the virtualization platform for Hadoop, carried out in May, 2014, Docker showed on the main transactions (on mass creation, restart and destruction of virtual nodes) significantly higher performance, than KVM, in particular, on the test of mass creation of virtual computing nodes the gain of consumption of processor resources in Docker is recorded 26 times below, than in KVM, and the gain of resource consumption of RAM — is three times lower.

2013

Support of Docker-containers in the environment of Google Compute Engine

In December, 2013 it is announced support of deployment of Docker-containers in the environment of Google Compute Engine.

The partial support of Docker in Red Hat Enterprise Linux 6.5 complete - in Fedora of version 20

Since November, 2013 partial support of Docker is included in the Red Hat Enterprise Linux distribution kit of version 6.5, complete — in the 20th version of the Fedora distribution kit, the agreement with Red Hat on inclusion since 2014 of Docker in the replicated Open Shift PaaS-platform is earlier reached.

Support of Docker in OpenStack (Havana release)

In October, 2013 the release of Havana replicated IaaS- the platform OpenStack in which Docker support is implemented is released (as the driver for OpenStack Nova).

The publication Docker under the license Apache 2.0

In March, 2013 the Docker code is published under the license Apache 2.0. In October, 2013 emphasizing focus shift to a new key technology, dotCloud is renamed into Docker (the PaaS-platform is saved under the name — dotCloud).

2008: Project startup

The project is begun in 2008 as internal development of dotCloud company, for the purpose of creation of the public PaaS-platform with support of different programming languages.

Notes

Links

1. Docker

2. Microsoft promised the open virtualization platform of applications