Tag: cooling

CDU Metrics & Control

Posted on by lechuck park

This image shows a CDU (Coolant Distribution Unit) Metrics & Control System diagram illustrating the overall structure. The system can be organized as follows:

System Structure

Upper Section: CDU Structure

  • First Loop: CPU with Coolant Distribution Unit
  • Second Main Loop: Row Manifold and Rack Manifold configuration
  • Process Chill Water Supply/Return: Process chilled water circulation system

Lower Section: Data Collection & Control Devices

  • Control Devices:
    • Pump (Pump RPM, Rate of max speed)
    • Valve (Valve Open %)
  • Sensor Configuration:
    • Temperature & Pressure Sensors on manifolds
  • Supply System:
    • Rack Water Supply/Return

Main Control Methods

1. Fixed Pressure Control (Fixed Pressure Drop)

  • Primary Method: Maintaining fixed pressure drop between rack supply-return
  • Alternatives: Fixed flow rate, fixed supply temperature, fixed return temperature, fixed speed control

2. Approach Temperature Control

  • Primary Method: Maintaining constant approach temperature
  • Alternatives: Fixed open, fixed secondary supply temperature control

Summary

This CDU system provides precise cooling control for data centers through dual management of pressure and temperature. The system integrates sensor feedback from manifolds with pump and valve control to maintain optimal cooling conditions across server racks.

#CDU #CoolantDistribution #DataCenterCooling #TemperatureControl #PressureControl #ThermalManagement

with Claude

CDU ( OCP Project Deschutes ) Numbers

Posted on by lechuck park

OCP CDU (Deschutes) Standard Overview

The provided visual summarizes the key performance metrics of the CDU (Cooling Distribution Unit) that adheres to the OCP (Open Compute Project) ‘Project Deschutes’ specification. This CDU is designed for high-performance computing environments, particularly for massive-scale liquid cooling of AI/ML workloads.

Key Performance Indicators

  • System Availability: The primary target for system availability is 99.999%. This represents an extremely high level of reliability, with less than 5 minutes and 15 seconds of downtime per year.
  • Thermal Load Capacity: The CDU is designed to handle a thermal load of up to 2,000 kW, which is among the highest thermal capacities in the industry.
  • Power Usage: The CDU itself consumes 74 kW of power.
  • IT Flow Rate: It supplies coolant to the servers at a rate of 500 GPM (approximately 1,900 LPM).
  • Operating Pressure: The overall system operating pressure is within a range of 0-130 psig (approximately 0-900 kPa).
  • IT Differential Pressure: The pressure difference required on the server side is 80-90 psi (approximately 550-620 kPa).
  • Approach Temperature: The approach temperature, a key indicator of heat exchange efficiency, is targeted at ≤3∘C. A lower value is better, as it signifies more efficient heat removal.

Why Cooling is Crucial for GPU Performance

Cooling has a direct and significant impact on GPU performance and stability. Because GPUs are highly sensitive to heat, if they are not maintained within an optimal temperature range, they will automatically reduce their performance through a process called thermal throttling to prevent damage.

The ‘Project Deschutes’ CDU is engineered to prevent this by handling a massive thermal load of 2,000 kW with a powerful 500 GPM flow rate and a low approach temperature of ≤3∘C. This robust cooling capability ensures that GPUs can operate at their maximum potential without being limited by heat, which is essential for maximizing performance in demanding AI workloads.

with Gemini

‘tightly fused’

Posted on by lechuck park

This illustration visualizes the evolution of data centers, contrasting the traditionally separated components with the modern AI data center where software, compute, network, and crucially, power and cooling systems are ‘tightly fused’ together. It emphasizes how power and advanced cooling are organically intertwined with GPU and memory, directly impacting AI performance and highlighting their inseparable role in meeting the demands of high-performance AI. This tight integration symbolizes a pivotal shift for the modern AI era.

LLM Efficiency with a Cooling

Posted on by lechuck park

This image demonstrates the critical impact of cooling stability on both LLM performance and energy efficiency in GPU servers through benchmark results.

Cascading Effects of Unstable Cooling

Problems with Unstable Air Cooling:

  • GPU Temperature: 54-72°C (high and unstable)
  • Thermal throttling occurs – where GPUs automatically reduce clock speeds to prevent overheating, leading to significant performance degradation
  • Result: Double penalty of reduced performance + increased power consumption

Energy Efficiency Impact:

  • Power Consumption: 8.16kW (high)
  • Performance: 46 TFLOPS (degraded)
  • Energy Efficiency: 5.6 TFLOPS/kW (poor performance-to-power ratio)

Benefits of Stable Liquid Cooling

Temperature Stability Achievement:

  • GPU Temperature: 41-50°C (low and stable)
  • No thermal throttling → sustained optimal performance

Energy Efficiency Improvement:

  • Power Consumption: 6.99kW (14% reduction)
  • Performance: 54 TFLOPS (17% improvement)
  • Energy Efficiency: 7.7 TFLOPS/kW (38% improvement)

Core Mechanisms: How Cooling Affects Energy Efficiency

  1. Thermal Throttling Prevention: Stable cooling allows GPUs to maintain peak performance continuously
  2. Power Efficiency Optimization: Eliminates inefficient power consumption caused by overheating
  3. Performance Consistency: Unstable cooling can cause GPUs to use 50% of power budget while delivering only 25% performance

Advanced cooling systems can achieve energy savings ranging from 17% to 23% compared to traditional methods. This benchmark paradoxically shows that proper cooling investment dramatically improves overall energy efficiency.

Final Summary

Unstable cooling triggers thermal throttling that simultaneously degrades LLM performance while increasing power consumption, creating a dual efficiency loss. Stable liquid cooling achieves 17% performance gains and 14% power savings simultaneously, improving energy efficiency by 38%. In AI infrastructure, adequate cooling investment is essential for optimizing both performance and energy efficiency.

With Claude

Data Center ?

Posted on by lechuck park

This infographic compares the evolution from servers to data centers, showing the progression of IT infrastructure complexity and operational requirements.

Left – Server

  • Shows individual hardware components: CPU, motherboard, power supply, cooling fans
  • Labeled “No Human Operation,” indicating basic automated functionality

Center – Modular DC

  • Represented by red cubes showing modular architecture
  • Emphasizes “More Bigger” scale and “modular” design
  • Represents an intermediate stage between single servers and full data centers

Right – Data Center

  • Displays multiple server racks and various infrastructure components (networking, power, cooling systems)
  • Marked as “Human & System Operation,” suggesting more complex management requirements

Additional Perspective on Automation Evolution:

While the image shows data centers requiring human intervention, the actual industry trend points toward increasing automation:

  1. Advanced Automation: Large-scale data centers increasingly use AI-driven management systems, automated cooling controls, and predictive maintenance to minimize human intervention.
  2. Lights-Out Operations Goal: Hyperscale data centers from companies like Google, Amazon, and Microsoft ultimately aim for complete automated operations with minimal human presence.
  3. Paradoxical Development: As scale increases, complexity initially requires more human involvement, but advanced automation eventually enables a return toward unmanned operations.

Summary: This diagram illustrates the current transition from simple automated servers to complex data centers requiring human oversight, but the ultimate industry goal is achieving fully automated “lights-out” data center operations. The evolution shows increasing complexity followed by sophisticated automation that eventually reduces the need for human intervention.

With Claude

Numbers about Cooling

Posted on by lechuck park

Numbers about Cooling – System Analysis

This diagram illustrates the thermodynamic principles and calculation methods for cooling systems, particularly relevant for data center and server room thermal management.

System Components

Left Side (Heat Generation)

  • Power consumption device (Power kW)
  • Time element (Time kWh)
  • Heat-generating source (appears to be server/computer systems)

Right Side (Cooling)

  • Cooling system (Cooling kW – Remove ‘Heat’)
  • Cooling control system
  • Coolant circulation system

Core Formula: Q = m×Cp×ΔT

Heat Generation Side (Red Box)

  • Q: Heat flow rate (J/s) = Power (kW)
  • V: Volumetric flow rate (m³/s)
  • ρ: Air density (approximately 1.2 kg/m³)
  • Cp: Specific heat capacity of air at constant pressure (approximately 1005 J/(kg·K))
  • ΔT: Temperature change

Cooling Side (Blue Box)

  • Q: Cooling capacity (kW)
  • m: Coolant circulation rate (kg/s)
  • Cp: Specific heat capacity of coolant (for water, approximately 4.2 kJ/kg·K)
  • ΔT: Temperature change

System Operation Principle

  1. Heat generated by electronic equipment heats the air
  2. Heated air moves to the cooling system
  3. Circulating coolant absorbs the heat
  4. Cooling control system regulates flow rate or temperature
  5. Processed cool air recirculates back to the system

Key Design Considerations

The cooling control system monitors critical parameters such as:

  • High flow rate vs. High temperature differential
  • Optimal balance between energy efficiency and cooling effectiveness
  • Heat load matching between generation and removal capacity

Summary

This diagram demonstrates the fundamental thermodynamic principles for cooling system design, where electrical power consumption directly translates to heat generation that must be removed by the cooling system. The key relationship Q = m×Cp×ΔT applies to both heat generation (air side) and heat removal (coolant side), enabling engineers to calculate required coolant flow rates and temperature differentials. Understanding these heat balance calculations is essential for efficient thermal management in data centers and server environments, ensuring optimal performance while minimizing energy consumption.

Components for AI Work

Posted on by lechuck park

This diagram visualizes the core concept that all components must be organically connected and work together to successfully operate AI workloads.

Importance of Organic Interconnections

Continuity of Data Flow

  • The data pipeline from Big Data → AI Model → AI Workload must operate seamlessly
  • Bottlenecks at any stage directly impact overall system performance

Cooperative Computing Resource Operations

  • GPU/CPU computational power must be balanced with HBM memory bandwidth
  • SSD I/O performance must harmonize with memory-processor data transfer speeds
  • Performance degradation in one component limits the efficiency of the entire system

Integrated Software Control Management

  • Load balancing, integration, and synchronization coordinate optimal hardware resource utilization
  • Real-time optimization of workload distribution and resource allocation

Infrastructure-based Stability Assurance

  • Stable power supply ensures continuous operation of all computing resources
  • Cooling systems prevent performance degradation through thermal management of high-performance hardware
  • Facility control maintains consistency of the overall operating environment

Key Insight

In AI systems, the weakest link determines overall performance. For example, no matter how powerful the GPU, if memory bandwidth is insufficient or cooling is inadequate, the entire system cannot achieve its full potential. Therefore, balanced design and integrated management of all components is crucial for AI workload success.

The diagram emphasizes that AI infrastructure is not just about having powerful individual components, but about creating a holistically optimized ecosystem where every element supports and enhances the others.

With Claude