Red Hat Ceph Storage

Ceph Storage is a packet of the program configured object and file cloud storage.

Ceph is free program object network of storage. It provides file and block interfaces of access. It can be used in systems operating in structure of a cluster from several Linux machines or thousands of nodes. The built-in mechanisms of the duplicated replication of data provide high survivability of a system, during the adding or removal of nodes the data array is automatically balanced taking into account the made changes. Such separation allows to optimize costs due to use of inexpensive servers and hard drives and to implement object storages of a class "web scale" for modern scenarios of use. Besides, in the new version the flexibility and scalability of DWH for support of multipetabyte data arrays considerably increases.

Data processing and metadata in Ceph is separated into different groups of nodes in a cluster, approximately as it is made in Lustre, with that distinction that processing becomes at the level of the user, without demanding special support from a kernel of operating systems of nodes. Ceph can work over block devices, in one file or using the existing file system of a node (for example, XFS).

Program systems of data storage of Red Hat Ceph Storage and Red Hat Gluster Storage are the powerful and flexible solutions focused on support of workloads of certain types. So, Red Hat Ceph Storage represents the reliable unified platform of storage for object storages and cloud infrastructures with data arrays of the petabyte sizes. In turn, Red Hat Gluster Storage is a powerful distributed file system for the organization of flexible services of data storage for data centers on the basis of physical servers, virtual machines, private and public cloud environments and also infrastructures of container applications. Both of these solutions do not require the specialized equipment for deployment, are actively supported by community of software development with the open code and the partner companies and also widely applied in the corporate sector worldwide.

2020: Red Hat Ceph Storage 4

On March 12, 2020 the Red Hat company announced release of Red Hat Ceph Storage 4, upgraded version of the object storage which provides scaling to level petabyte and is intended for the cloud focused applications and the systems of analytical data processing. The solution is based on release of Nautilus of the open source project of community Ceph.

According to the company, thanks to the increased scalability and functions updated, simplifying operation, Red Hat Ceph Storage 4 helps representatives of the most different industries, including the financial sector, state institutions, the autoindustry and telecommunication companies, more effectively to support such tasks as application development, data analysis, artificial intelligence (AI), machine learning (ML) and other types of workloads of the next generation.

Giving to the organizations an opportunity to unroll the object storages of the petabyte size compatible to Amazon S3 (Simplified Storage Service), Red Hat Ceph Storage allows to scale performance when preserving all advantages of cloud economy. During recent internal testing the latest version of Red Hat Ceph Storage showed more than double gain of performance for workloads with intensive data writing, having confirmed the importance for meeting requirements of the modern applications which are actively operating with large volumes of data.

The next generation of the workloads connected with data processing increases load of infrastructures of storage and does them more difficult that sharply raises an automation role for operation of such infrastructures, especially at their scaling. At the expense of higher level of automation at management and placement of data, Red Hat Ceph Storage 4 helps IT specialists to pass smoothly from the infrastructure models focused on data warehouses to service-oriented schemes to expand a range of supported applications and target audiences of storage systems. The developed possibilities of self-maintenance and selfrecovery of Red Hat Ceph Storage 4 in the field of the automated replication, recovery and initialization of systems (provisioning) improve not only scalability, but also business continuity indicators.

Red Hat Ceph Storage 4 is optimized under requests of the clients for effective storage and management of quickly growing amounts of data generated by modern workloads without attraction of an additional operations staff. Red Hat Ceph Storage 4 offers IT commands the simplified and intuitive operation, locating for this purpose a number of the updated functions:

• The simplified installation process – standard installation takes less than 10 minutes.
• The management console (management dashboard) updated a web – the unified and high-level control of transactions helps IT specialists to reveal and solve quickly arising problems at any moment.
• The updated service quality monitoring function – confirmation of QoS-indicators of the data warehouse from a position of applications in a multitenantny cloud environment.
• The integrated bucket-notifications – support of the processing pipelines of data optimized for Kubernetes serverless-architecture for automation.

For March, 2020 Red Hat Ceph Storage 4 is already available to downloading.

2018: Transition of the project under the aegis Linux Foundation

In November, 2018 Red Hat announced transfer of the project of the distributed Ceph storage under protection of the Linux Foundation organization. This solution it was announced about two weeks later after sale of Red Hat of IBM corporation.

Within Linux Foundation the new organization Ceph Foundation which integrated more than 30 participants will be engaged in financing, development of infrastructure of Ceph and support of the community connected with the project. Among them — Red Hat, SUSE, Intel, Canonical, China Mobile, DigitalOcean, OVH, Western Digital and ZTE.  Also this non-profit organization will replace the Ceph Advisory Board advisory board.

Red Hat gave to Linux Foundation the project of the distributed Ceph storage

It is supposed what after transformation of Ceph into the independent Ceph Foundation project will create the neutral platform within which the technology will develop and be controlled. At the same time this platform will not be tied to preferences of any producers and will promote closer interaction with community.

The control structure of Ceph Leadership Team will still make the technical solutions connected with development of the project. Her representative will be included into management board of Ceph Foundation and will become some kind of link in development process.

Ceph has a long-term experience in implementation of successful projects which help the companies to manage effectively fast-growing and extending needs for data storage — the chief executive of Linux Foundation Jim Zemlin says. — Under Linux Foundation wing the Ceph Foundation organization will be able to adapt investments from wider group to help with support of infrastructure which is necessary for preserving of success and stability of an ecosystem of Ceph.

By November, 2018 Ceph uses many large companies, including BMW, SAP and Salesforce.com.^[1]

2017: Red Hat Ceph Storage 2.3

In June, 2017 Red Hat, the solution provider with open code, announced release of Red Hat Ceph Storage 2.3, the new version software-defined storage systems (DWH). The system constructed on the basis of the open Ceph 10.2 (Jewel) project is compatible to the client of the Hadoop S3A file system now, supports the Network File System (NFS) protocol and can be unrolled in containers. These innovations increase universality of object storage Ceph Storage, allowing to use it as for the systems of analytical processing Big Data (Big Data), and for the loadings working with files. The new version of Ceph Storage represents the next step of Red Hat to creation of flexible and elastic DWH which can be optimized for a wide range of corporate tasks and computing environments.

This version of Red Hat Ceph Storage received the effective NFS interface for Ceph Object Gateway. It will allow users and applications, depending on solvable tasks and the available tools, to address the same data sets both on object S3, and on file to NFS interfaces. Using multi-saytovye possibilities of Red Hat Ceph Storage, it is possible to provide data access in global clusters under the NFS protocol. As a result, the clients using Ceph Storage as file storage receive flexibility, scalability and cost efficiency of object storage during gradual upgrade of a storage system.

The compatibility with the client of the Hadoop S3A file system allows to use object storage Red Hat Ceph Storage in the systems of analytical processing of Big Data, such as Apache Hadoop MapReduce, Hive and Spark. Thanks to it users will be able to estimate all its advantages: management of lifecycle of objects and metadata, cost reduction due to effective reservation given using erasure coding and also an opportunity to scale computing powers and tanks of storage independently of each other.

Besides, Red Hat Ceph Storage supports deployment in containers now. The container image of a product which functionally does not differ from a traditional packet can be loaded on the website Red Hat Container Registry. The container version of Ceph Storage together with the Red Hat Ansible automation system allows to execute installation and product update as atomic transactions that simplifies management and accelerates deployment of software-defined DWH.

2016: Red Hat Ceph Storage 2

On June 22, 2016 Red Hat announced an exit of Ceph Storage 2 - the software-defined DWH created on the platform of the version of Ceph Jewel Open Source project (v10.2.0). In release performance improvement of processing of object loadings and convenience of work of users^[2] are promised^[2].

Ceph (2016) components

Support of POSIX - the compatible Ceph Filesystem file system (CephFS) which is capable to store data in clusters of distributed system of Ceph Storage Cluster is implemented. At first CephFS will have the status tech preview and will be able to be used in IaaS-infrastructures based on OpenStack and a cloud service of the separated Manila file system. Appeared the opportunities useful to corporate configurations of object storage and the providing increased scalability, security and strict compatibility with the commonly accepted API interfaces.

Features of release

• work with the global object clusters supporting single name space and synchronization of data between the clusters developed in different regions;
• the increased security due to integration with such popular authentication systems as the Active Directory, LDAP and OpenStack Identity (Keystone) v3;
• close compatibility with the Amazon S3 and OpenStack Object Storage environments (Swift), including support of AWS v4 Client Signatures, version control of objects and their mass removal.

The product was equipped with the management console of Red Hat Storage Console 2. Affirms that it simplifies as deployment of DWH of Ceph, management of it: time of deployment is reduced from day to hours.

The release of the commercial version of Red Hat Ceph Storage 2 is declared for summer of 2016.

History

The project of a system was created by Sage Weil in 2007 within the doctoral dissertation. Later other developers were connected to creation of a system. Developing and commercializing a system in 2012 Weil founded^[3].

Notes