Tag: ThermalManagement

Time Constant(Delay of the sensor)

Posted on by lechuck park

Image Interpretation: System Problems Due to Sensor Delay

This diagram explains system performance issues caused by the Time Constant (delay) of temperature sensors.

Top Section: Two Workload Scenarios

LLM Workload (AI Tasks)

  • Runs at 100% workload
  • Almost no delay (No Delay almost)
  • Result: Performance Down and Workload Cost waste

GPU Workload

  • Operating at 80°C
  • Thermal Throttling occurs
  • Transport Delay exists
  • Performance degradation starts at 60°C → Step down

Bottom Section: Core of the Sensor Delay Problem

Timeline:

  1. Sensor UP start (Temperature Sensor activation)
    • Big Delay due to Time Constant
  2. TC63 (After 10-20 seconds)
    • Sensor detects 63% temperature rise
    • Actual temperature is already higher
  3. After 30-40 seconds
    • Sensor detects 86% rise
    • Temperature Divergence, Late Cooling problem occurs

Key Issues

Due to the sensor’s Time Constant delay:

  • Takes too long to detect actual temperature rise
  • Cooling system activates too late
  • GPU already overheated, causing thermal throttling
  • Results in workload cost waste and performance degradation

Summary

Sensor delays create a critical gap between actual temperature and detected temperature, causing cooling systems to react too late. This results in GPU thermal throttling, performance degradation, and wasted computational resources. Real-time monitoring with fast-response sensors is essential for optimal system performance.

#ThermalManagement #SensorDelay #TimeConstant #GPUThrottling #DataCenter #PerformanceOptimization #CoolingSystem #AIWorkload #SystemMonitoring #HardwareEngineering #ThermalThrottling #LatencyChallenges #ComputeEfficiency #ITInfrastructure #TemperatureSensing

With Claude

GPU Throttling

Posted on by lechuck park

GPU Throttling Architecture Analysis

This diagram illustrates the GPU’s power and thermal management system.

Key Components

1. Two Throttling Triggers

  • Power Throttling: Throttling triggered by power limits
  • Thermal Throttling: Throttling triggered by temperature limits

2. Different Control Approaches

  • Power Limit (Budget) Controller: Slow, Linear Step Down
  • Thermal Safety Controller: Fast, Hard Step Down
    • This aggressive response is necessary because overheating can cause immediate hardware damage

3. Priority Gate

Receives signals from both controllers and determines which limitation to apply.

4. PMU/SMU/DVFS Controller

The Common Control Unit that manages:

  • PMU: Power Management Unit
  • SMU: System Management Unit
  • DVFS: Dynamic Voltage and Frequency Scaling

5. Actual Adjustment Mechanisms

  • Clock Domain Controller: Reduces GPU Frequency
  • Voltage Regulator: Reduces GPU Voltage

6. Final Result

Lower Power/Temp (Throttled): Reduced power consumption and temperature in throttled state

Core Principle

When the GPU reaches power budget or temperature limits, it automatically reduces performance to protect the system. By lowering both frequency and voltage simultaneously, it effectively reduces power consumption (P ∝ V²f).

Summary

GPU throttling uses two controllers—power (slow, linear) and thermal (fast, aggressive)—that feed into a shared PMU/SMU/DVFS system to dynamically reduce clock frequency and voltage. Thermal throttling responds more aggressively than power throttling because overheating poses immediate hardware damage risks. The end result is lower power consumption and temperature, sacrificing performance to maintain system safety and longevity.

#GPUThrottling #ThermalManagement #PowerManagement #DVFS #GPUArchitecture #HardwareOptimization #ThermalSafety #PerformanceVsPower #ComputerHardware #GPUDesign #SystemManagement #ClockSpeed #VoltageRegulation #TechExplained #HardwareEngineering

With Claude

Predictive 2 Reactions for AI HIGH Fluctuation

Posted on by lechuck park

Image Interpretation: Predictive 2-Stage Reactions for AI Fluctuation

This diagram illustrates a two-stage predictive strategy to address load fluctuation issues in AI systems.

System Architecture

Input Stage:

  • The AI model on the left generates various workloads (model and data)

Processing Stage:

  • Generated workloads are transferred to the central server/computing system

Two-Stage Predictive Reaction Mechanism

Stage 1: Power Ramp-up

  • Purpose: Prepare for load fluctuations
  • Method: The power supply system at the top proactively increases power in advance
  • Preventive measure to secure power before the load increases

Stage 2: Pre-cooling

  • Purpose: Counteract thermal inertia
  • Method: The cooling system at the bottom performs cooling in advance
  • Proactive response to lower system temperature before heat generation

Problem Scenario

The warning area at the bottom center shows problems that occur without these responses:

  • Power/Thermal Throttling
  • Performance degradation (downward curve in the graph)
  • System dissatisfaction state

Key Concept

This system proposes an intelligent infrastructure management approach that predicts rapid fluctuations in AI workloads and proactively adjusts power and cooling before actual loads occur, thereby preventing performance degradation.

Summary

This diagram presents a predictive two-stage reaction system for AI workload management that combines proactive power ramp-up and pre-cooling to prevent thermal throttling. By anticipating load fluctuations before they occur, the system maintains optimal performance without degradation. The approach represents a shift from reactive to predictive infrastructure management in AI computing environments.

#AIInfrastructure #PredictiveComputing #ThermalManagement #PowerManagement #AIWorkload #DataCenterOptimization #ProactiveScaling #AIPerformance #ThermalThrottling #SmartCooling #MLOps #AIEfficiency #ComputeOptimization #InfrastructureAsCode #AIOperations

With Claude

UPS & ESS

Posted on by lechuck park

UPS vs. ESS & Key Safety Technologies

This image illustrates the structural differences between UPS (Uninterruptible Power System) and ESS (Energy Storage System), emphasizing the advanced safety technologies required for ESS due to its “High Power, High Risk” nature.

1. Left Side: System Comparison (UPS vs. ESS)

This section contrasts the purpose and scale of the two systems, highlighting why ESS requires stricter safety measures.

  • UPS (Traditional System)
    • Purpose: Bridges the power gap for a short duration (10–30 mins) until the backup generator starts (Generator Wake-Up Time).
    • Scale: Relatively low capacity (25–500 kWh) and output (100 kW – N MW).
  • ESS (High-Capacity System)
    • Purpose: Stores energy for long durations (4+ hours) for active grid management, such as Peak Shaving.
    • Scale: Handles massive power (~100+ MW) and capacity (~400+ MWh).
    • Risk Factor: Labeled as “High Power, High Risk,” indicating that the sheer energy density makes it significantly more hazardous than UPS.

2. Right Side: 4 Key Safety Technologies for ESS

Since standard UPS technologies (indicated in gray text) are insufficient for ESS, the image outlines four critical technological upgrades (indicated in bold text).

① Battery Management System (BMS)

  • (From) Simple voltage monitoring and cut-off.
  • [To] Active Balancing & Precise State Estimation: Requires algorithms that actively balance cell voltages and accurately calculate SOC (State of Charge) and SOH (State of Health).

② Thermal Management System

  • (From) Simple air cooling or fans.
  • [To] Forced Air (HVAC) / Liquid Cooling: Due to high heat generation, robust air conditioning (HVAC) or direct Liquid Cooling systems are necessary.

③ Fire Detection & Suppression

  • (From) Detecting smoke after a fire starts.
  • [To] Off-gas Detection & Dedicated Suppression: Detects Off-gas (released before thermal runaway) to prevent fires early, using specialized suppressants like Clean Agents or Water Mist.

④ Physical/Structural Safety

  • (From) Standard metal enclosures.
  • [To] Explosion-proof & Venting Design: Enclosures must withstand explosions and safely vent gases.
  • [To] Fire Propagation Prevention: Includes fire barriers and BPU (Battery Protective Units) to stop fire from spreading between modules.

Summary

  • Scale: ESS handles significantly higher power and capacity (>400 MWh) compared to UPS, serving long-term grid needs rather than short-term backup.
  • Risk: Due to the “High Power, High Risk” nature of ESS, standard safety measures used in UPS are insufficient.
  • Solution: Advanced technologies—such as Liquid Cooling, Off-gas Detection, and Active Balancing BMS—are mandatory to ensure safety and prevent thermal runaway.

#ESS #UPS #BatterySafety #BMS #ThermalManagement #EnergyStorage #FireSafety #Engineering #TechTrends #OffGasDetection

WIth Gemini

TDP (Thermal Design power)

Posted on by lechuck park

TDP (Thermal Design Power) Interpretation

This image explains the concept and limitations of TDP (Thermal Design Power).

Main Process

Chip → Run Load → Generate Heat → TDP Measurement

  1. Chip: Processor/chip operates
  2. Load (Run): Executes specific workload
  3. Heat (make): Heat is generated (measured by number)
  4. ??? Watt: Displayed as TDP value

Role of TDP

  • Thermal Design Guideline: Reference for cooling system design
  • Cool Down: Serves as baseline for cooling solutions like fans and coolers

⚠️ Critical Limitations

Ambiguous Standard

  • “Typical high load” baseline is not standardized
  • Different measurement methods across vendors:
    • Intel’s TDP
    • NVIDIA’s TGP (Total Graphics Power)
    • AMD’s PPT (Package Power Tracking)

Problems with TDP

  1. Not Peak Power – Average value, not maximum power consumption
  2. Thermal Guideline, Not Electrical Spec – Just a guide for thermal management
  3. Poor Fit for Sustained Loads – Doesn’t properly reflect real high-load scenarios
  4. Underestimates Real-World Heat – Measured lower than actual heat generation

Summary

TDP is a thermal guideline for cooling system design, not an accurate measure of actual power consumption or heat generation. Different manufacturers use inconsistent standards (TDP/TGP/PPT), making comparisons difficult. It underestimates real-world heat and peak power, serving only as a reference point rather than a precise specification.

#TDP #ThermalDesignPower #CPUCooling #PCHardware #ThermalManagement #ComputerCooling #ProcessorSpecs #HardwareEducation #TechExplained #CoolingSystem #PowerConsumption #PCBuilding #TechSpecs #HeatDissipation #HardwareLimitations

With Claude

LLM goes with Computing-Power-Cooling

Posted on by lechuck park

LLM’s Computing-Power-Cooling Relationship

This diagram illustrates the technical architecture and potential issues that can occur when operating LLMs (Large Language Models).

Normal Operation (Top Left)

  1. Computing Requires – LLM workload is delivered to the processor
  2. Power Requires – Power supplied via DVFS (Dynamic Voltage and Frequency Scaling)
  3. Heat Generated – Heat is produced during computing processes
  4. Cooling Requires – Temperature management through proper cooling systems

Problem Scenarios

Power Issue (Top Right)

  • Symptom: Insufficient power (kW & Quality)
  • Results:
    • Computing performance degradation
    • Power throttling or errors
    • LLM workload errors

Cooling Issue (Bottom Right)

  • Symptom: Insufficient cooling (Temperature & Density)
  • Results:
    • Abnormal heat generation
    • Thermal throttling or errors
    • Computing performance degradation
    • LLM workload errors

Key Message

For stable LLM operations, the three elements of Computing-Power-Cooling must be balanced. If any one element is insufficient, it leads to system-wide performance degradation or errors. This emphasizes that AI infrastructure design must consider not only computing power but also adequate power supply and cooling systems together.

Summary

  • LLM operation requires a critical balance between computing, power supply, and cooling infrastructure.
  • Insufficient power causes power throttling, while inadequate cooling leads to thermal throttling, both resulting in workload errors.
  • Successful AI infrastructure design must holistically address all three components rather than focusing solely on computational capacity.

#LLM #AIInfrastructure #DataCenter #ThermalManagement #PowerManagement #AIOperations #MachineLearning #HPC #DataCenterCooling #AIHardware #ComputeOptimization #MLOps #TechInfrastructure #AIatScale #GreenAI

WIth Claude

Cooling for AI (heavy heater)

Posted on by lechuck park

AI Data Center Cooling System Architecture Analysis

This diagram illustrates the evolution of data center cooling systems designed for high-heat AI workloads.

Traditional Cooling System (Top Section)

Three-Stage Cooling Process:

  1. Cooling Tower – Uses ambient air to cool water
  2. Chiller – Further refrigerates the cooled water
  3. CRAH (Computer Room Air Handler) – Distributes cold air to the server room

Free Cooling option is shown, which reduces chiller operation by leveraging low outside temperatures for energy savings.

New Approach for AI DC: Liquid Cooling System (Bottom Section)

To address extreme heat generation from high-density AI chips, a CDU (Coolant Distribution Unit) based liquid cooling system has been introduced.

Key Components:

① Coolant Circulation and Distribution

  • Direct coolant circulation system to servers

② Heat Exchanges (Two Methods)

  • Direct-to-Chip (D2C) Liquid Cooling: Cold plate with manifold distribution system directly contacting chips
  • Rear-Door Heat Exchanger (RDHx): Heat exchanger mounted on rack rear door (immersion cooling)

③ Pumping and Flow Control

  • Pumps and flow control for coolant circulation

④ Filtration and Coolant Quality Management

  • Maintains coolant quality and removes contaminants

⑤ Monitoring and Control

  • Real-time monitoring and cooling performance control

Critical Differences

Traditional Method: Air cooling → Indirect, suitable for low-density workloads

AI DC Method: Liquid cooling → Direct, high-efficiency, capable of handling high TDP (Thermal Design Power) of AI chips

Liquid has approximately 25x better heat transfer efficiency than air, making it effective for cooling AI accelerators (GPUs, TPUs) that generate hundreds of watts to kilowatt-level heat.

Summary:

  1. Traditional data centers use air-based cooling (Cooling Tower → Chiller → CRAH), suitable for standard workloads.
  2. AI data centers require liquid cooling with CDU systems due to extreme heat from high-density AI chips.
  3. Liquid cooling offers direct-to-chip heat removal with 25x better thermal efficiency than air, supporting kW-level heat dissipation.

#AIDataCenter #LiquidCooling #DataCenterInfrastructure #CDU #ThermalManagement #DirectToChip #AIInfrastructure #GreenDataCenter #HeatDissipation #HyperscaleComputing #AIWorkload #DataCenterCooling #ImmersionCooling #EnergyEfficiency #NextGenDataCenter

With Claude