Cooling Configurations for Data Center Cabinets and Racks
The heat loads within data center cabinets and racks continue to rise, prompting facility managers and IT staff to look for more energy efficient, cost effective, and reliable cooling solutions. Since there are many different cabinet configurations available for data center equipment cooling, it’s important to understand the various options and their benefits. In general, cabinet cooling configurations can be categorized as air cooling, liquid cooling, or a combination of air and liquid cooling. This article will illustrate several different configurations for air cooling, air and liquid cooling, and liquid cooling of cabinets, summarizing Chapter 4 of Liquid Cooling Guidelines for Datacom Equipment Centers.1 It will consider both the external interaction between facility infrastructure and cabinet/rack cooling, as well as available internal cooling systems for the cabinets/racks.
Figure 1 shows a purely air-cooled cabinet or rack. In this configuration, air is simply blown over the electronics to remove the waste heat. This is the most basic form of active cooling at the cabinet/rack level and is a configuration that most facility operators know well. Typically, air cooling is less expensive and easier to implement than liquid cooling. However, liquid cooling removes a much higher heat load and is more energy efficient than air cooling.
Figure 1: Air-cooled rack or cabinet
As defined by the American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE),2 a cabinet or rack is considered to be liquid-cooled if liquid must be circulated to and from the rack or cabinet for operation. There are often two liquid cooling loops. The first is the facility side loop, also known as the primary loop, which is coolant supplied by the facility to the data center. The second is the process side loop, also known as the secondary loop. This loop provides cooling to the cabinet/rack or data center equipment itself within the cabinet or rack. The coolant supply and return lines are routed under a raised floor, above the floor, or from the ceiling. Also, coolant supply and return connections can be at the base, top, or side of the cabinet or rack.
A combination air- and liquid-cooled rack is another cooling configuration available. (See Figure 2.) There are many ways this cooling method can be implemented. One way is to cool the electronics by blowing cool air into the front of the cabinet and removing waste heat via a liquid-to-air heat exchanger located at the rear of the cabinet. Another way to cool the electronics is to use a cold plate with circulating coolant located on specific “hot spots” with the air-cooling removing the remaining heat. These are just two of the possible combination cooling configurations.
Figure 2: Combination air- and liquid-cooled rack or cabinet
A cabinet or rack can also be purely liquid-cooled, as shown in Figure 3. The most common configuration is a completely enclosed cabinet with an air-to-liquid heat exchanger that uses cold air as the working fluid (i.e. – the coolant) for cooling the electronics. Alternatively, the electronics can all be cooled by running fluid to cold plates located in the data center equipment. In both of these cases, the heat is removed from the cabinet by the liquid.
Figure 3: Liquid-cooled rack or cabinet
Both the combination cooled and purely liquid-cooled options are susceptible to condensation if the coolant is below the room’s dew point. A Coolant Distribution Unit (CDU) can eliminate the risk of condensation by automatically ensuring that the coolant’s temperature is above the dew point. The CDU circulates chilled fluid through the cabinet, rack, or datacom equipment, essentially creating the secondary loop. Additional benefits of the CDU are that it isolates the electronics from the harsher facility water and minimizes the coolant volume near the technology. Figures 4 and 5 illustrate the use of an external CDU with the combination cooled implementation and the liquid-cooled implementation, respectively.
Figure 4: Combination air- and liquid-cooled rack or cabinet with external CDU
Figure 5: Liquid-cooled rack or cabinet with external CDU.
Alternatively, the CDU can be located at the cabinet or rack level, as seen in Figures 6 and 7. In these figures, a CDU is installed within the cabinet or rack to condition the fluid and distribute it to the electronics or datacom equipment. The internal CDU allows the operator to have greater cooling flexibility per cabinet or rack. An external CDU may cool multiple cabinets or racks and, therefore, requires that all of the cabinets and racks have the same cooling requirements. On the other hand, an internal CDU is a more costly solution that typically does not accommodate redundancy and creates a single point of failure.
Figure 6: Combination air- and liquid-cooled rack or cabinet with internal CDU.
Figure 7: Liquid-cooled rack or cabinet with internal CDU
While Figures 1-7 show how the building infrastructure interacts with the cabinet or rack, Figures 8-12 show configurations within the cabinet or racks themselves. Liquid cooling systems internal to the cabinet/rack remove heat by means of a fluid that has been conditioned previously by a CDU. Conditioned fluid is fluid that is delivered at a specific temperature. Possible fluids include water, an antifreeze mixture, dielectric fluid, or refrigerant. Filtered water that has added corrosion inhibitors and/or biocides is the most common fluid used with a CDU.
The first configuration, shown in Figure 8, uses a combination of air and liquid to cool the cabinet/rack. In this configuration, air is the only coolant and is blown over an air-to-liquid heat exchanger located somewhere along the airstream. The heat exchanger then serves to cool either the incoming or outgoing air. This reduces the heat load on the Computer Room Air Conditioning (CRAC) as well as reduces hot air recirculation. However, it is important to consider heat exchanger effectiveness and condensation in this cabinet configuration, especially when the heat exchanger is upstream from the datacom equipment. Also, the air-side pressure drop across the heat exchanger must be low if additional fans are not incorporated.
Figure 8: Open air-cooled datacom equipment in an air/liquid-cooled rack
Another configuration is a closed cabinet in which the air is also the only coolant entering the datacom equipment. (See Figure 9.) In this case, the entire heat load is removed by the conditioned fluid circulated to the heat exchanger from the CDU. The heat exchanger and fans can be located in various locations in the cabinet. However, they should be positioned to ensure that any condensation on the heat exchanger will not drip onto electronics. Unlike the hot aisle / cold aisle configurations, such as the one shown in Figure 8, this configuration prevents the mixing of hot and cold air and removes more heat.
Figure 9: Closed air-cooled datacom equipment in a liquid-cooled cabinet
The datacom equipment in the cabinet or rack can also be cooled purely by liquid, as seen in Figure 10. In this configuration, conditioned fluid from a CDU or from a facility water supply circulates coolant to all of the heat generating components in the datacom equipment. Figure 10 shows the optional liquid heat exchanger and pump (essentially an internal CDU) that would not be needed if the coolant was coming directly from a facility water supply and not from a CDU. However, it’s important to note that without a CDU in place, there is generally a larger volume of fluid volume near the electronics and therefore a higher risk of a catastrophic leak. Therefore, this configuration is rarely used.
Figure 10: Liquid-cooled datacom equipment in a liquid-cooled rack
Another configuration option, shown in Figure 11, supplies liquid cooling just to the hottest areas within the cabinet/rack (or areas where there are sensitive electronics) and supplies air cooling from the CRAC to the rest of the cabinet/rack. This method may work well in cabinets/racks where there are a variety of heat sources with different and/or complex geometries. As with the previous example show in Figure 10, there is an optional liquid heat exchanger and pump shown in Figure 11.
Figure 11: Open air- and liquid-cooled datacom equipment in an air/liquid cooled rack.
Lastly, the Figure 12 configuration is similar to the Figure 11 configuration except that it is a closed cabinet. It utilizes both liquid cooling and air cooling. Coolant supplied by the facility loop or a CDU to an air-to-liquid heat exchanger cools the air that cools the low heat density components. The liquid-to-liquid heat exchanger (optional) and its associated pump and fans or CDU coolant removes the heat from the high heat density components.
Figure 12: Closed air- and liquid-cooled datacom equipment in a liquid-cooled rack.
The majority of cabinet or rack cooling systems remove heat using some form of conditioned facility fluid in a cooled water loop, with assistance in some cases from the CRAC. This focus on liquid cooling is driven by the increasing heat loads of high-tech operating equipment. Liquid is more effective at removing these heat loads than air and, therefore, can help to protect the life-expectancy and efficiency of datacom equipment as well as lower the energy costs of cooling.
Liquid Cooling Guidelines for Datacom Equipment Centers can be purchased from ASHRAE for $50.00 by non-members and $40.00 by members by emailing email@example.com or visiting the ASHRAE bookstore at www.ashrae.org.
Note: Figures reprinted with permission from ASHRAE.
Read our press release "Lytron Acknowledged for Contributions to Datacom Liquid Cooling Guidelines."
1American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc. (2006). Liquid Cooling Guidelines for Datacom Equipment Centers, Atlanta.
2Ibid, page 37.