The supercomputer in 10000 cores is assembled in "cloud" of Amazon

Jason Stowe, the expert in high-performance computing systems, asked two engineers a simple question recently: whether it is possible to construct a 10000-core cluster in "cloud"?

This really good round number, Stou, the CEO and the founder of Cycle Computing company which helps clients with receiving quick and effective access to computing powers like supercomputers which are usually reserved for the universities and the large research organizations told.

Cycle Computing already created several clusters in a cloud of Elastic Compute Cloud of Amazon company which is scaled to several thousand cores. But Stou wanted to bring him to the following level. Creation of a reserve in 10000 cores on Amazon probably became repeatedly, but Stou does not know of such achievements, meaning that the packet of technology planning and applications launch, optimized for work on high-performance computers (HPC) is applied.

Stou says that to find references to something bigger they did not manage. If a system was tested on speed, then its Linux cluster started on Amazon, perhaps would be rather large to enter the list of five hundred most high-performance supercomputers of the world. But first of all, it was required to find clients who will benefit from use of such big cluster. There is no sense to untwist such big computing environment if it is not intended for real work.

The company of biotechnologies of Genentech (San Francisco) where computing powers for studying of proteinaceous communications were required for the scientist Jacob Corn became the client who selected the 10th 000-core "cloud" cluster.

Relying on the recommendations of Amazon in optimality of time for a request of 10,000 cores, Cycle Computing and Genentech untwisted a cluster on March 1 right after midnight. Though offered Amazon the virtual machines optimized for high-performance computing, Cycle and Genentech selected Linux a cluster of "standard vanilla CentOS" to save money, on the advice of Stou. CentOS is the version of Linux on the basis of Linux Red Hat.

10,000 cores were made of 1250 machines with eight cores each and also 8.75 Terabyte of OZU and 2 Petabyte of disk space. Increasing to several thousand cores for time, process took 45 minutes before the cluster was built entirely. Everything went without problems. "When we requested the 10000th core, we received it", Stou told.

The cluster fulfilled eight hours at the cost of $8500, including all payment types.

For Genentech it became cheaper and simpler in comparison with an alternative of purchase of 10,000 cores for the data center, and payment of their idle time throughout the most part of the life, Korn told. According to him, use of the existing resources for accomplishment of modeling could take several weeks or months instead of eight hours which were required on Amazon. Genentech gained from use of a large number of cores because these calculations "are embarrassingly parallel", without any communications between nodes therefore performance becomes "scaled linearly with increase in number of cores", the researcher told.

Stou noticed that the company already works on big expansion. Everything what Cycle - the client who will be able to seize these opportunities needs.