Human with AI

Posted on by lechuck park

This image titled “Human with AI” illustrates the collaborative structure between humans and AI.

Top: Human works

Humans operate through three stages:

  1. Experience – Collecting various experiences and information
  2. Thought – Thinking and judging by combining emotions, logic, and intuition
  3. Action – Executing final decisions

Bottom: AI Works

AI operates through similar three stages:

  1. Learning – Learning from databases and patterns
  2. Reasoning – Analyzing and judging through algorithms and calculations
  3. Inference – Deriving results based on statistics and probabilities

Core: Human-AI Collaboration Structure

The green arrow in the center with “Develop & Verification” represents the process where humans verify AI’s reasoning results and make final judgments (Thought) to connect them to actual actions (Action).

In other words, when AI analyzes data and presents reasoning results, humans review and verify them to ultimately decide whether to execute – representing a Human-in-the-loop system. AI assists decision-making, but the final judgment and action are under human responsibility.

Summary

This diagram illustrates a Human-in-the-loop AI system where AI processes data and provides reasoning, but humans retain final decision-making authority. Both humans and AI follow similar learning-thinking-acting cycles, but human verification serves as the critical bridge between AI inference and real-world action. This structure emphasizes responsible AI deployment with human oversight.

#HumanAI #AICollaboration #HumanInTheLoop #AIGovernance #ResponsibleAI #AIDecisionMaking #HumanOversight #AIVerification #HumanCenteredAI #AIEthics

With Claude

Who is the first wall?

Posted on by lechuck park

AI Scaling: The 6 Major Bottlenecks (2025)

1. Data

  • High-quality text data expected to be depleted by 2026
  • Solutions: Synthetic data (fraud detection in finance, medical data), Few-shot learning

2. LLM S/W (Algorithms)

  • Ilya Sutskever: “The era of simple scaling is over. Now it’s about scaling the right things”
  • Innovation directions: Test-time compute scaling (OpenAI o1), Mixture-of-Experts architecture, Hybrid AI

3. Computing → Heat

  • GPT-3 training required 1,024 A100 GPUs for several months
  • By 2030, largest training runs projected at 2-45GW scale
  • GPU cluster heat generation makes cooling a critical challenge

4. Memory & Network ⚠️ Current Critical Bottleneck

Memory

  • LLMs grow 410x/2yr, computing power 750x/2yr vs DRAM bandwidth only 2x/2yr
  • HBM3E completely sold out for 2024-2025. AI memory market projected to grow at 27.5% CAGR

Network

  • Speed of light limitation causes tens to hundreds of ms latency over distance. Critical for real-time applications (autonomous vehicles, AR)
  • Large-scale GPU clusters require 800Gbps+, microsecond-level ultra-low latency

5. Power 💡 Long-term Core Constraint

  • Sam Altman: “The cost of AI will converge to the cost of energy. The abundance of AI will be limited by the abundance of energy”
  • Power infrastructure (transmission lines, transformers) takes years to build
  • Data centers projected to consume 7.5% of US electricity by 2030

6. Cooling

  • Advanced technologies like liquid cooling required. Infrastructure upgrades take 1+ year

“Who is the first wall?”

Critical Bottlenecks by Timeline:

  1. Current (2025): Memory bandwidth + Data quality
  2. Short-to-Mid term: Power infrastructure (5-10 years to build)
  3. Long-term: Physical limit of the speed of light

Summary

The “first wall” in AI scaling is not a single barrier but a multi-layered constraint system that emerges sequentially over time. Today’s immediate challenges are memory bandwidth and data quality, followed by power infrastructure limitations in the mid-term, and ultimately the fundamental physical constraint of the speed of light. As Sam Altman emphasized, AI’s future abundance will be fundamentally limited by energy abundance, with all bottlenecks interconnected through the computing→heat→cooling→power chain.

#AIScaling #AIBottleneck #MemoryBandwidth #HBM #DataCenterPower #AIInfrastructure #SpeedOfLight #SyntheticData #EnergyConstraint #AIFuture #ComputingLimits #GPUCluster #TestTimeCompute #MixtureOfExperts #SamAltman #AIResearch #MachineLearning #DeepLearning #AIHardware #TechInfrastructure

With Claude

Data Center Shift with AI

Posted on by lechuck park

Data Center Shift with AI

This diagram illustrates how data centers are transforming as they enter the AI era.

📅 Timeline of Technological Evolution

The top section shows major technology revolutions and their timelines:

  • Internet ’95 (Internet era)
  • Mobile ’07 (Mobile era)
  • Cloud ’10 (Cloud era)
  • Blockchain
  • AI(LLM) ’22 (Large Language Model-based AI era)

🏢 Traditional Data Center Components

Conventional data centers consisted of the following core components:

  • Software
  • Server
  • Network
  • Power
  • Cooling

These were designed as relatively independent layers.

🚀 New Requirements in the AI Era

With the introduction of AI (especially LLMs), data centers require specialized infrastructure:

  1. LLM Model – Operating large language models
  2. GPU – High-performance graphics processing units (essential for AI computations)
  3. High B/W – High-bandwidth networks (for processing large volumes of data)
  4. SMR/HVDC – Switched-Mode Rectifier/High-Voltage Direct Current power systems
  5. Liquid/CDU – Liquid cooling/Cooling Distribution Units (for cooling high-heat GPUs)

🔗 Key Characteristic of AI Data Centers: Integrated Design

The circular connection in the center of the diagram represents the most critical feature of AI data centers:

Tight Interdependency between SW/Computing/Network ↔ Power/Cooling

Unlike traditional data centers, in AI data centers:

  • GPU-based computing consumes enormous power and generates significant heat
  • High B/W networks consume additional power during massive data transfers between GPUs
  • Power systems (SMR/HVDC) must stably supply high power density
  • Liquid cooling (Liquid/CDU) must handle high-density GPU heat in real-time

These elements must be closely integrated in design, and optimizing just one element cannot guarantee overall system performance.

💡 Key Message

AI workloads require moving beyond the traditional layer-by-layer independent design approach of conventional data centers, demanding that computing-network-power-cooling be designed as one integrated system. This demonstrates that a holistic approach is essential when building AI data centers.

📝 Summary

AI data centers fundamentally differ from traditional data centers through the tight integration of computing, networking, power, and cooling systems. GPU-based AI workloads create unprecedented power density and heat generation, requiring liquid cooling and HVDC power systems. Success in AI infrastructure demands holistic design where all components are co-optimized rather than independently engineered.

#AIDataCenter #DataCenterEvolution #GPUInfrastructure #LiquidCooling #AIComputing #LLM #DataCenterDesign #HighPerformanceComputing #AIInfrastructure #HVDC #HolisticDesign #CloudComputing #DataCenterCooling #AIWorkloads #FutureOfDataCenters

With Claude

The optimization

Posted on by lechuck park

This diagram illustrates the fundamental purpose and stages of optimization.

Basic Purpose of Optimization:

Optimization

  • Core Principle: Perform only necessary actions
  • Code Level: Remove unnecessary elements

Two Goals of Optimization:

1. More Speed

  • O(n): Algorithm (Logic) improvement
  • Techniques: Caching/Parallelization/Recursion optimization

2. Less Resource

  • Memory: Reduce memory usage
  • Management: Dynamic & Static memory optimization

Optimization Implementation Stages:

Stage 1: SW Level (Software Level)

  • Code-level optimization

Stage 2: HW Implementation (Hardware Implementation)

  • Offload heavy workloads to hardware
  • Applied when software optimization is insufficient

Optimization Process:

Input → Processing → Output → Verification

  1. Deterministic INPUT Data: Structured input (DB Schema)
  2. Rule-based: Apply rule-based optimization
  3. Deterministic OUTPUT: Predictable results
  4. Verification: Validate speed, resource usage through benchmarking and profiling

Summary:

Optimization aims to increase speed and reduce resources by removing unnecessary operations. It follows a staged approach starting from software-level improvements and extending to hardware implementation when needed. The process ensures predictable, verifiable results through deterministic inputs/outputs and rule-based methods.

#Optimization #PerformanceTuning #CodeOptimization #AlgorithmImprovement #SoftwareEngineering #HardwareAcceleration #ResourceManagement #SpeedOptimization #MemoryOptimization #SystemDesign #Benchmarking #Profiling #EfficientCode #ComputerScience #SoftwareDevelopment

With Claude

New For AI

Posted on by lechuck park

Analysis of “New For AI” Diagram

This image, titled “New For AI,” systematically organizes the essential components required for building AI systems.

Structure Overview

Top Section: Fundamental Technical Requirements for AI (Two Pillars)

Left Domain – Computing Axis (Turquoise)

  1. Massive Data
    • Processing vast amounts of data that form the foundation for AI training and operations
  2. Immense Computing
    • Powerful computational capacity to process data and run AI models

Right Domain – Infrastructure Axis (Light Blue)

3. Enormous Energy
Large-scale power supply to drive AI computing

  1. High-Density Cooling
    • Effective heat removal from high-performance computing operations

Central Link 🔗

Meaning of the Chain Link Icon:

  • For AI to achieve its performance, Computing (Data/Chips) and Infrastructure (Power/Cooling) don’t simply exist in parallel
  • They must be tightly integrated and optimized to work together
  • Symbolizes the interdependent relationship where strengthening only one side cannot unlock the full system’s potential

Bottom Section: Implementation Technologies (Stability & Optimization)

Learning & Inference/Reasoning (Learning and Inference Optimization)

Technologies to enhance AI model performance and efficiency:

  • Evals/Golden Set: Model evaluation and benchmarking
  • Safety Guardrails, RLHF-DPO: Safety assurance and human feedback-based learning
  • FlashAttention: Memory-efficient attention mechanism
  • Quant(INT8/FP8): Computational optimization through model quantization
  • Speculative/MTP Decoding: Inference speed enhancement techniques

Massive Parallel Computing (Large-Scale Parallel Computing)

Hardware and network technologies enabling massive computation:

  • GB200/GB300 NVL72: NVIDIA’s latest GPU systems
  • HBM: High Bandwidth Memory
  • InfiniBand, NVlink: Ultra-high-speed interconnect technologies
  • AI factory: AI-dedicated data centers
  • TPU, MI3xx, NPU, DPU: Various AI-specialized chips
  • PIM, CxL, UvLink: Memory-compute integration and next-gen interfaces
  • Silicon Photonics, UEC: Optical communication technologies

More Energy, Energy Efficiency (Energy Supply and Efficiency)

Technologies for stable and efficient power supply:

  • Smart Grid: Intelligent power grid
  • SMR: Small Modular Reactor (stable large-scale power source)
  • Renewable Energy: Renewable energy integration
  • ESS: Energy Storage System (power stabilization)
  • 800V HVDC: High-voltage direct current transmission (loss minimization)
  • Direct DC Supply: Direct DC supply (eliminating conversion losses)
  • Power Forecasting: AI-based power demand prediction and optimization

High Heat Exchange & PUE (Heat Exchange and Power Efficiency)

Securing cooling system efficiency and stability:

  • Liquid Cooling: Liquid cooling (higher efficiency than air cooling)
  • CDU: Coolant Distribution Unit
  • D2C: Direct-to-Chip cooling
  • Immersing: Immersion cooling (complete liquid immersion)
  • 100% Free Cooling: Utilizing external air (energy saving)
  • AI-Driven Cooling Optimization: AI-based cooling optimization
  • PUE Improvement: Power Usage Effectiveness (overall power efficiency metric)

Key Message

This diagram emphasizes that for successful AI implementation:

  1. Technical Foundation: Both Data/Chips (Computing) and Power/Cooling (Infrastructure) are necessary
  2. Tight Integration: These two axes are not separate but must be firmly connected like a chain and optimized simultaneously
  3. Implementation Technologies: Specific advanced technologies for stability and optimization in each domain must provide support

The central link particularly visualizes the interdependent relationship where “increasing computing power requires strengthening energy and cooling in tandem, and computing performance cannot be realized without infrastructure support.”

Summary

AI systems require two inseparable pillars: Computing (Data/Chips) and Infrastructure (Power/Cooling), which must be tightly integrated and optimized together like links in a chain. Each pillar is supported by advanced technologies spanning from AI model optimization (FlashAttention, Quantization) to next-gen hardware (GB200, TPU) and sustainable infrastructure (SMR, Liquid Cooling, AI-driven optimization). The key insight is that scaling AI performance demands simultaneous advancement across all layers—more computing power is meaningless without proportional energy supply and cooling capacity.

#AI #AIInfrastructure #AIComputing #DataCenter #AIChips #EnergyEfficiency #LiquidCooling #MachineLearning #AIOptimization #HighPerformanceComputing #HPC #GPUComputing #AIFactory #GreenAI #SustainableAI #AIHardware #DeepLearning #AIEnergy #DataCenterCooling #AITechnology #FutureOfAI #AIStack #MLOps #AIScale #ComputeInfrastructure

With Claude