Project of upgrade of engineering infrastructure of DPC of the Moscow Institute of Physics and Technology (MIPT)

The iCore company, the Russian system integrator of a full stroke focused on the innovative solutions in the field of IT and multimedia in cooperation with Softline company which acted as the general contractor in this project completed upgrade of engineering infrastructure of the data processing center (DPC) of the Moscow Institute of Physics and Technology (MIPT) — the main server platform of the leading technical college of the country.

At upgrade of data center of MIPT the latest developments in the field of power supply, conditioning and communication technologies were used.

Customer's tasks:

1. Upgrade of engineering infrastructure of the existing data processing center for the purpose of compliance to higher requirements for reliability of work of a computing cluster and also for a possibility of installation of more power-intensive IT equipment.

2. Increase in the capacity (power) in the platform for ensuring centralization of computing resources of the university taking into account plans of future development of IT infrastructure.

Wide experience of consortium of iCore and Softline companies on creation of data processing centers and existence of the high partner statuses of world leaders of the industry (APC by Schneider Electric, Huber+Suhner, etc.) allowed the companies to propose to the customer the most optimal solution on the price and the used technologies and to win the state competition.

The iCore company performed the following complex of works:

1. Development and approval of the project documentation on upgrade of DPC.

2. Upgrade of the main engineering subsystems:

A number of the existing systems of data center (in particular, the system of automatic gas fire fighting) were successfully integrated into new infrastructure that allowed to save the investments into a construction object made earlier by the customer.

The platform is completely ready to installation of a computing cluster and transfer of the IT equipment from other platforms of the university.

The new architecture of data center of MIPT is characterized by unique reliability, high energy efficiency and light scalability.

Reliability

1. Electric power supply systems and coolings of DPC of MIPT are organized based on infrastructure of InfraStruXure (ISX) Type B of the Schneider Electric company known in all technology world for the highest level of reliability of the equipment.

The solutions Schneider Electric not only are reliable, but also are extremely ergonomic and graceful.

Based on infrastructure of ISX on an object are constructed:

2. Key engineering subsystems are reserved according to the scheme of 100% duplication 2N (chillers, the UPS, a cable system) and according to scheme N+1 (interline conditioners, fankoyla). The DPC of MIPT corresponds to the third level of reliability (TIER 3) according to international standard TIA-942 on infrastructure of data processing centers, and the indicator of smooth operation is 99.982% that completely corresponds to new problems of a computing cluster of the university.

Energy efficiency

In data center of MIPT the best energy-saving technologies are used:

1. Localization of a "hot" corridor — the most energy efficient cooling system of the server hardware from all known on a ratio price/quality.

2. The water cooling system on chillers of production Uniflair by Schneider Electric with freecooling function — a possibility of economy up to 30% of annual energy consumption.

According to iCore company, the indicator of Power Usage Effectiveness (PUE) of data center of MIPT according to the results of upgrade is 1.5 that testifies to very high level of energy efficiency of the platform. Future statistics of the created system of automatic equipment and scheduling will prove justification of project solutions and the stated level of energy efficiency of this object.

Scalability

Engineering infrastructure of DPC is constructed based on modular components (interline conditioners, chillers, the UPS, a cable system, automatic equipment controllers).

The modular architecture closes all current demands of the customer for 100% and does not require use of the excessive expensive equipment.

The central data center of MIPT is ready to further accumulation of computing powers — fast and flexible.

Cooling system and climate control

In the machine hall and the premises of the UPS of data center of MIPT the permanent microclimate is supported: air temperature is in limits +20 … +25 °C, a relative humidity fluctuates in the range from 40% to 65%. Thus, the IT equipment of MIPT is reliably protected from overheating and failure: the possibility of loss of dew (when it is too cold and wet) and emergence of a static discharge (when too hot and chilly) is excluded.

1. Cooling of the machine hall

"Hot" corridor

Separation of flows of a hot and cold air — a key factor of energy efficiency of DPC. Engineers of iCore selected for implementation from the machine hall the system of isolation of a "hot" corridor allowing to achieve the highest economy of electricity in comparison with any other conditioning systems.

The choice of vendor — APC by Schneider Electric — is not accidental. The first system of isolation of "hot" corridors (HACS, Hot-Aisle Containment System) is patented just by this company.

The constructed "hot" corridor represents two rows from 18 racks of APC 42U located "a back to a back" with 8 interline APC InRow RC conditioners between them.

The "hot" corridor is isolated from the external environment by special doors on end faces and panels on top. Intra line conditioners take away heat from this corridor, give upward for cooling and blow the cooled air to the external premises which then under static pressure moves to the face of racks. The active IT equipment takes away a cold air, stretches it via the fan and issues already heated air in a "hot" corridor.

Interline conditioners work at a cold water. Heat extraction is performed under a raised floor through a water cooling system: one pipe pumps a cold water, another takes away heated water.

Chillers

Basic Element of the water cooling system is the refrigerator — the chiller. In data center of MIPT two chillers of production of Uniflair by Schneider Electric company (one worker, other reserve) with a refrigerating capacity of 185 kW everyone are installed. Chillers are placed on a roof of the building and work with rotation 24 hours.

Chiller task — heat transport from the cooled water to fresh air. For this purpose in the chiller freon which changes the aggregate state circulates, i.e. passes from liquid into gaseous (evaporates), on the contrary from gaseous in liquid (is condensed). At evaporation of freon there is energy absorption, at its condensation — selection.

The water cooling system of data center of MIPT is designed according to the double-circuit scheme. As the heat carrier of an external circuit 40% solution of ethylene glycol is used. In an internal cooling circuit water circulates. Such scheme, on the one hand, allows to exclude freezing of the heat carrier going on pipes outside of the building at negative temperatures, on the other hand, reduces a zone of use of ethylene glycol hazardous to health, protecting it from the machine hall with normal water.

The chillers installed on a roof banish through themselves 40% solution of ethylene glycol, cooling it up to the certain temperature, and return back. The heat exchanger reports ethylene glycol temperature to water which, in turn, using the pump station comes to interline conditioners, takes away heat from a "hot" corridor and rises upward.

Freecooling

In cold season at air temperature +5 s below chillers of data center of MIPT pass into the mode of free cooling (free cooling) at which the heat carrier is cooled at the expense of fresh air.

The mode of freecooling allows to exclude operation of compressors which are the main consumers of the electric power in the chiller and by that to receive cooled water almost free of charge. Fans and the frikulingovy pump remain the only consumers of the electric power.

2. Cooling of the premises of the UPS

Implementation in the conditioning system UPS the chiller – fankoil became the most optimal solution, according to engineers of iCore. As conditioners two channel fankoil Carrier (Italy) which make giving of a cold air and a fence hot for the subsequent cooling in the refrigerator are used. The third fankoil is in a reserve.

System of regular power supply

The new DPC of MIPT is expected consumption of 180 kW. From them 141 kW are the share of power supply of 16 server cabinets and 2 wiring closets.

The power supply system is executed on the modular APC by Schneider Electric components.

At failure of the main source of power supply the data center automatically passes to power supply using the UPS. Capacity of the UPS accumulators is expected a power supply of all data center within 15 minutes.

For power supply of IT loading two modular Symmetra PX UPSes of 160 kW are used.

Power supply of the engineering equipment (circulation pumps of chillers and pumps of a water circuit) is performed using the modular MGE Galaxy 3500 20 UPS of kW.

Engineers of iCore provided a possibility of service and upgrade of a system of regular power supply without output of all complex from the mode of the main work.

Structured cabling system

The transport environment of data center of MIPT is executed based on innovation of Huber+Suhner company — a building cable system of the LiSA Solutions series which sales began only at the end of 2013. "Heart" of a system is made by shielded copper cables of category 6a Huber+Suhner with the improved data transmission parameters. It is the first installation of a system in Russia. Specialists of iCore transmitted results of installation to the headquarters of Huber+Suhner to Switzerland and received the certificate on a 25-year guarantee.

Along with a copper subsystem, SCS of data center of MIPT includes fiber-optical cable assemblies and the preterminirovanny optical fiber cables Huber+Suhner. Data transfer rate makes 10 Gbps.

Connection of cabinets is among themselves organized using a yellow plastic cable tray of Huber+Suhner set on a roof of a "hot" corridor.

The organization of a copper and optical subsystem is performed using four cross zones located in two wiring closets.

Automated system of supervisory control

Specialists of iCore implemented in data center of MIPT the automated system of supervisory control (ASSC) on the basis of new modular controllers of production of Delta Controls company.

The constructed system of automatic equipment allows to conduct the centralized observation of a condition of equipment of all engineering subsystems of an object:

In view of all possible risks connected with the water system of cold supply, specialists of iCore paid special attention to installation of the sensors of detection of a leakage intended for protection of the equipment of DPC against possible leakages of water at accidents in the systems of a water supply system and heating. At hit of moisture on the tape sensor of a leakage dispatching system immediately receives a signal of flooding.

The system of visualization is organized using the software of ORCAview giving an opportunity of remote control a status of DPC. The Automated Workplace (AW) of ORCAview operator combines in itself ease of use of symbolic circuits and power of the system tool of real time.

Warranty and service support

Provide to Softline together with iCore service and warranty service of a number of MIPT which is earlier available in data center and all complex of the new equipment within 5 years. Existence of own support services allows integrators to guarantee to the customer a response time on a request no more than 1 hour in mode 7/365.

New engineering infrastructure of the center the data processing of MIPT constructed according to the latest world trends is the key to fast and smooth operation of information systems of the university.

The iCore company offers the Russian companies implementation of all stages of creation of data center: from the choice of the platform before "turnkey" commissioning of the object.