Download document () of 20

Deploying Open Compute rack and power architectures (ORV3)

Enterprise customers seeking to deploy Open Compute hardware may find it difficult to transition from a traditional rack and power architecture. Compare the architectures and learn the advantages of using a system-based approach to deploying Open Compute architecture.

A system-based approach to OCP adoption

The best place for an enterprise to begin the process of deploying OCP standards for testing or production is by evaluating compute and networking options. Deploying an OCP system can be daunting at first, but avoiding proprietary and component-based designs will make it easier. A true system-based approach allows the enterprise to order one part number with all the rack and power hardware preconfigured. The rack comes on a shock pallet for quick deployment. This alleviates the need to procure parts from multiple vendors and assemble on site. The only integration required is server and networking hardware.

Why change what works?

  • OCP is a third-generation standard, with ORV3 refining the power and rack topology into something flexible and universal.
  • Hyperscale adoption over the past 10 years has developed a vibrant hardware and software ecosystem.
  • Large-scale deployments have refined the designs to increase reliability and reduce deployment time.
  • The 48V DC power shelf architecture is both reliable and efficient.
  • The 30 kW DC power shelf standard increases power density 150 to 300 percent compared to traditional AC power distribution.
  • The ORV3 rack enclosure accommodates mounting for liquid cooling manifolds to support cold plate technology, part of the reason higher power density is possible.
  • The busbar system removes server power cord connections, eliminating potential points of failure.
  • The power shelf reduces stranded power by using more incoming power and having less power on standby.
  • Open-source hardware and software can reduce operating costs significantly.

Preconfigured reference designs

A reference design provides a convenient foundation for a system-based approach. A reference design defines a preconfigured solution and offers you a blueprint for deployment. Someone with expertise and experience working with OCP standards has validated the engineering design to ensure the components work as a system, removing the guesswork.

There are important power and thermal considerations when selecting a busbar and power shelf combination. All the pitfalls of component incompatibility can be avoided by following a reference design architecture.

Open rack reference design

A. OCP “OU” mounting rails for 21‑inch rack equipment (Convertible to EIA rails)
B. EIA “RU” mounting rails for 19‑inch rack equipment (Convertible to OCP rails)
C. ORV3 busbar (Blind-mate power connection)
D. ORV3 power shelf (15 kW N+1; includes six 3 kW modular power supplies)

Note: Shown without doors and side panels.

eaton-ocp-open-rack-reference-design.PNG

Origins of the Open Compute Project

The Open Compute Project, also known as OCP, was founded by Facebook (now Meta) in 2011. At the time, Facebook was scaling their data centers quickly and dealing with all the challenges that come with that:

  • Speed to deploy new hardware
  • Cost of deployment (both hardware and installation)
  • Efficiency
  • MTTR (mean time to repair)

Facebook has a strong hacker culture that drives them to find new solutions and shortcuts to move faster. They approached their data center growing pains with a fresh mindset and proceeded to transform data center norms, reimagining typical server, rack and power design.

spacer

The original OCP rack design was designated Open Rack Version 1 (ORV1). This system was a triplet rack that challenged the traditional EIA 19-inch rack mount standard, increasing the width to 21 inches and creating a new 1.89-inch height standard called “OU” instead of the traditional EIA 1.75-inch “RU” height standard. The new approach also introduced a new power topology. In a traditional data center environment, the servers and networking equipment have individual AC power supplies. The Open Rack standard introduced the power shelf concept. The power shelf is basically a large power supply fitted with modular rectifiers. In the ORV1 standard, the power shelf accepts AC power and distributes 12V DC power to OCP servers using busbars in the back of the rack. This offers several benefits:

  • The power supplies are centrally located and accessible from the front of the rack to simplify servicing.
  • The busbars are fixed in the rear of the rack. Servers slide in and dock with the busbars, reducing installation time and removing server power cables from the design.
  • The power shelf design is universal, allowing multiple manufacturers to make the power shelf and rectifier modules.
spacer
As Facebook looked to include more users and manufacturers in the Open Compute Project, they quickly learned that other approaches had value and developed working groups devoted to advancing new standards. The Open Rack Version 2 (ORV2) standard introduced 48V DC power as an alternative to 12V DC. There were also changes to the rack, introducing a single rack bay instead of the triplet rack.
spacer

In 2016, Microsoft introduced the Project Olympus subgroup of OCP. Project Olympus uses a more traditional 19-inch rack and employs a rack PDU to distribute AC power to the servers.

One unique aspect of the PDU design is the universal input on the PDU, which uses a proprietary connector from Harting. The detachable AC input cord can be configured for different types of power input, allowing one rack PDU to be used at 208V three-phase Delta 60A and 415/240V three-phase Wye 32A. This simplifies deployment by reducing SKU count on rack PDUs and is especially helpful for consistency in global deployments.

Project Olympus was a departure from the power shelf topology, but illustrated that OCP could incorporate a variety of different ideas under its open-source umbrella.

spacer

As OCP has grown, changed and included new members and working groups, the focus on improving ORV2 has led to the Open Rack Version 3 (ORV3) standard. ORV3 seeks to address several needs expressed by the OCP community:

  • Higher power densities, up to 30 kW per rack enclosure
  • New busbar design to handle more power, with split zones
  • Toolless blind-mate connection of the power shelf to the busbar
  • Standards for cold plate liquid cooling and mounting cooling manifolds in the rack 
  • Flexibility to mount 21-inch OU or 19-inch RU equipment in the rack enclosure 
  • Options for side panels and doors
  • General effort to make the rack and power more flexible for wider-scale adoption
spacer

OCP is considered by many to be a hyperscale-only platform. It is hard to dispute that companies like Meta, Google and Microsoft drive such economies of scale that they essentially become the platform. That doesn’t mean that smaller enterprise users cannot benefit from OCP standards. Arguably, they can benefit even more by leveraging the lessons learned from larger-scale deployments.

This still presents challenges, because the OCP model requires a high degree of engineering and direct-to-ODM interaction. It is not common for enterprise customers to have large engineering teams that include mechanical, electrical and power engineers on staff. A data center manager more often has a small team of network infrastructure experts, software experts and a facility or power engineer. Although a capable team, they typically focus on evaluating commercial solutions rather than designing from scratch. This is where the traditional 19-inch rack and AC power standard has served well for years.

spacer

The basic 19-inch rack standard was defined by AT&T around 100 years ago to save space and homogenize installation of telecommunications equipment in telephone company central offices. When telco manufacturers embarked on higher-speed systems, they continued to rely on 19-inch racks. The budding computer industry eventually followed suit and provided AC power supplies for each server.

Today, the 19-inch rack and AC power standard is firmly established and well supported by many different OEM suppliers. Established server vendors such as HPE, Dell, Oracle, Cisco and Lenovo have countless 19-inch server options, refined over many generations. The 19-inch rack standard is supported by companies such as Eaton, APC, Legrand, Vertiv and Rittal, all offering a large selection of racks and related accessories. These racks come in various heights as tall as 58U, including the classic 42U, with 24-inch (600 mm) or 30-inch (750 mm) external cabinet width and various depth options.

Large AC power distribution systems start with a three-phase UPS on the facility side. A typical data center will have dual-feed power (A and B) throughout the facility, which is distributed to rack enclosures using panel boards, remote power panels (RPP) or overhead busway systems. Inside the rack enclosure, two AC rack PDUs distribute the A and B feed power to redundant AC server power supplies, with two to six power supplies per server.

spacer