Raritan International
Overview Many data center managers are doing a good job conserving energy – decreasing PUE, raising data center temperatures, using air-side economizers to reduce energy consumption for cooling – but average power consumption at the rack is still going up. In fact, the increased efficiency means more power is available for servers to support data center growth. Data centers are finding that they must deploy more and more power to their racks. This white paper addresses considerations surrounding the deployment of high power.
Trends in Data Center Power Deployment Data center managers are deploying more and more power to their IT equipment racks to keep up with power-hungry devices. From the chart below, nearly half (49%) of the data center managers polled had a maximum rack power density of 12kW or less. Their expectations were that two years later, only one-third (33%) would have a maximum rack power density of 12kW or less. Some data centers today have racks wired to provide as much as 30kVA.
Drivers for High Power Racks High power requirements at data center racks are driven by several factors, such as high-density racks filled with 1U “pizza box” servers. There are companies now deploying 1U servers in 54U racks. Another example is networking equipment such as Cisco® Nexus 7000 series systems. There are also blade server installations, such as multiple HP® c7000 chassis, in one rack. And network storage devices such as the Dell™ Compellent™ Storage Center FC enclosure which draws 450W for each 2U device.
Below is an ASHRAE chart showing the projected heat load, which is also the power consumption since each watt of energy consumed by IT equipment is converted to one watt of heat. Note that the vertical scale of the chart is logarithmic, so power demands are not leveling off but increasing dramatically.
Data Center Power Distribution Around the World Typical voltages in North America are 120V and 208V. Some typical voltages internationally are 100V (Japan), 230V (Europe) and 240V (Australia). Since IT equipment vendors want to be able to sell their products globally, virtually all IT equipment is designed with power supplies which automatically adjust to voltages up to 240V.
Both single-phase or three-phase circuits can be distributed to racks. In North America, three-phase circuits are typically 208V, though 400V is becoming more common. For the rest of the world, three-phase power distribution is 400V (Europe and most of Asia) and 415V (Australia). Since the maximum voltage conventional IT equipment accepts is 240V, it will be the job of the rack or cabinet PDU to take a 400V input and convert it to 230V or 240V at the PDU outlets.
In many parts of the world, electrical circuits are specified as being rated at 16A or 32A. This is the actual current that these electrical devices are allowed to carry safely. In North America, electrical equipment is typically specified as 15A, 20A, 30A, etc. However, the National Electrical Code (NEC) requires that these values be “derated” by 20% to provide some headroom. So in North America an electrical device specified as 20A is actually rated at 16A (20A x 80%).
Watts (W) is used to specify the actual power consumed (active power). Volt Amps (VA) is used to specify the power that is available (apparent power). Think of apparent power as the design specification. For example, you could have a rack wired for 5.0kVA that is actually drawing only 4.2kW. This white paper will follow this convention, but the terms kW and kVA are often used interchangeably.
Trends in Data Center Power Deployment Data center managers are deploying more and more power to their IT equipment racks to keep up with power-hungry devices. From the chart below, nearly half (49%) of the data center managers polled had a maximum rack power density of 12kW or less. Their expectations were that two years later, only one-third (33%) would have a maximum rack power density of 12kW or less. Some data centers today have racks wired to provide as much as 30kVA.
Drivers for High Power Racks High power requirements at data center racks are driven by several factors, such as high-density racks filled with 1U “pizza box” servers. There are companies now deploying 1U servers in 54U racks. Another example is networking equipment such as Cisco® Nexus 7000 series systems. There are also blade server installations, such as multiple HP® c7000 chassis, in one rack. And network storage devices such as the Dell™ Compellent™ Storage Center FC enclosure which draws 450W for each 2U device.
Below is an ASHRAE chart showing the projected heat load, which is also the power consumption since each watt of energy consumed by IT equipment is converted to one watt of heat. Note that the vertical scale of the chart is logarithmic, so power demands are not leveling off but increasing dramatically.
Data Center Power Distribution Around the World Typical voltages in North America are 120V and 208V. Some typical voltages internationally are 100V (Japan), 230V (Europe) and 240V (Australia). Since IT equipment vendors want to be able to sell their products globally, virtually all IT equipment is designed with power supplies which automatically adjust to voltages up to 240V.
Both single-phase or three-phase circuits can be distributed to racks. In North America, three-phase circuits are typically 208V, though 400V is becoming more common. For the rest of the world, three-phase power distribution is 400V (Europe and most of Asia) and 415V (Australia). Since the maximum voltage conventional IT equipment accepts is 240V, it will be the job of the rack or cabinet PDU to take a 400V input and convert it to 230V or 240V at the PDU outlets.
In many parts of the world, electrical circuits are specified as being rated at 16A or 32A. This is the actual current that these electrical devices are allowed to carry safely. In North America, electrical equipment is typically specified as 15A, 20A, 30A, etc. However, the National Electrical Code (NEC) requires that these values be “derated” by 20% to provide some headroom. So in North America an electrical device specified as 20A is actually rated at 16A (20A x 80%).
Watts (W) is used to specify the actual power consumed (active power). Volt Amps (VA) is used to specify the power that is available (apparent power). Think of apparent power as the design specification. For example, you could have a rack wired for 5.0kVA that is actually drawing only 4.2kW. This white paper will follow this convention, but the terms kW and kVA are often used interchangeably.
What Is High Power? High power consumption at a rack may take the form of a few devices, each of which consumes a lot of power, such as blade servers and blade chassis requiring 5kW or more per chassis, or many moderate power consumption devices, such as a 42U rack filled with 42 1U “pizza box” servers, each server requiring 200-300 watts. There are several ways of deploying power in these scenarios and an approach which works for a high-outlet-density situation may also work for a situation where a lot of power needs to be deployed to a few power hogs.
Some data center managers add power by running more circuits. But, in general, it does not make sense to run several whips (power cables) to devices with multiple power supplies such as blade servers. It is easier and more economical to run two high-power feeds, either by under-floor whips or an overhead system, to a pair of high-power rack PDUs. From the high-power rack PDUs, short cables can be run to the power supplies, making for a much cleaner, e.g., less under-floor air obstruction, and more manageable deployment. Economics also improve with savings in copper and component costs.
When considering power demand, it is important to determine and design for peak actual demand. Designing to IT equipment nameplate ratings is excessively high. Designing for average power consumption may not be sufficient for periods of peak demand.
High Power, High Outlet Density In the case of a large number of devices, each demanding a moderate amount of power, many power outlets will be necessary on the rack PDU.
A typical dense “pizza box” deployment would include two rack PDUs for redundant power where each PDU is loaded to 40%, so that if one power feed fails, the other feed will not exceed the NEC requirement of 80% (for North America). Typical outlets for “pizza box” servers are IEC C-13 (up to 250V, 10A international, 15A UL) and NEMA 5-20R (up to 125V, 20A, 16A rated). In this application, it is not uncommon to see a three-phase 208V 50A rack PDU with up to 54 outlets providing up to 14.4kW power per rack.
208V single phase vs. 208V three phase If each server consumes an average of 200W, then the total power consumption is 42 x 200W = 8.4kW. The fully populated rack in this case requires 8.4kW. Therefore, as you size a rack PDU to support this load, you’ll need to look for something that supports greater than 8.4kW. While rack PDUs are advertised in the market at a certain voltage, phase and amperage, kW ratings on rack PDUs typically already account for the NEC requirement of 80% load.
Because for three-phase power the sine waves are 120 degrees out of phase, calculating VA is slightly more complex than for single phase because we need to include the square root of 3, which is 1.732. The apparent power formula for three phase is V x DeratedA x 1.732 = VA. A three-phase Delta deployment provides three separate circuits and more than 70% more total power than a comparable single phase, single circuit.
Alternatively, if single-phase circuits are being run to the rack, then to support an 8.4kW load at the rack, you would need a rack PDU that provides a minimum of 60A. The math works as follows:
48A (80% rating of 60A) * 208V = 10kW
Finally, if you believe you need additional headroom for growth for potential increased server utilization that drives power consumption greater than an average of 200W, then an appropriate rack PDU could be 50A 208V three-phase which will support 14.4kW. The math works as follows:
40A (80% rating of 50A) * 208V * sq. rt. 3 (or 1.73) = 14.4kW
Some data center managers add power by running more circuits. But, in general, it does not make sense to run several whips (power cables) to devices with multiple power supplies such as blade servers. It is easier and more economical to run two high-power feeds, either by under-floor whips or an overhead system, to a pair of high-power rack PDUs. From the high-power rack PDUs, short cables can be run to the power supplies, making for a much cleaner, e.g., less under-floor air obstruction, and more manageable deployment. Economics also improve with savings in copper and component costs.
When considering power demand, it is important to determine and design for peak actual demand. Designing to IT equipment nameplate ratings is excessively high. Designing for average power consumption may not be sufficient for periods of peak demand.
High Power, High Outlet Density In the case of a large number of devices, each demanding a moderate amount of power, many power outlets will be necessary on the rack PDU.
A typical dense “pizza box” deployment would include two rack PDUs for redundant power where each PDU is loaded to 40%, so that if one power feed fails, the other feed will not exceed the NEC requirement of 80% (for North America). Typical outlets for “pizza box” servers are IEC C-13 (up to 250V, 10A international, 15A UL) and NEMA 5-20R (up to 125V, 20A, 16A rated). In this application, it is not uncommon to see a three-phase 208V 50A rack PDU with up to 54 outlets providing up to 14.4kW power per rack.
208V single phase vs. 208V three phase If each server consumes an average of 200W, then the total power consumption is 42 x 200W = 8.4kW. The fully populated rack in this case requires 8.4kW. Therefore, as you size a rack PDU to support this load, you’ll need to look for something that supports greater than 8.4kW. While rack PDUs are advertised in the market at a certain voltage, phase and amperage, kW ratings on rack PDUs typically already account for the NEC requirement of 80% load.
Because for three-phase power the sine waves are 120 degrees out of phase, calculating VA is slightly more complex than for single phase because we need to include the square root of 3, which is 1.732. The apparent power formula for three phase is V x DeratedA x 1.732 = VA. A three-phase Delta deployment provides three separate circuits and more than 70% more total power than a comparable single phase, single circuit.
Alternatively, if single-phase circuits are being run to the rack, then to support an 8.4kW load at the rack, you would need a rack PDU that provides a minimum of 60A. The math works as follows:
48A (80% rating of 60A) * 208V = 10kW
Finally, if you believe you need additional headroom for growth for potential increased server utilization that drives power consumption greater than an average of 200W, then an appropriate rack PDU could be 50A 208V three-phase which will support 14.4kW. The math works as follows:
40A (80% rating of 50A) * 208V * sq. rt. 3 (or 1.73) = 14.4kW
