Tag: NUMA

Parallel Processing ( Process – Data works)

Posted on by lechuck park

From Claude with some prompting
This image illustrates different architectures of Parallel Processing:

  1. Single Core CPU: A single CPU connected to memory via one memory channel. The memory is divided into Instruction (Computing) and Data sections.
  2. Multi Core CPU: A CPU with multiple cores connected to memory through multiple memory channels. The memory structure is similar to the single core setup.
  3. NUMA (Non-Uniform Memory Access): Multiple multi-core CPUs, each with local memory. CPUs can access memory attached to other CPUs, but with “More Hop Memory Access”.
  4. GPU (Graphics Processing Unit): Described as “Completely Independent Processing-Memory Units”. It uses High Bandwidth Memory and has a large number of processing units directly mapped to data.

The GPU architecture shows many small processing units connected to a shared high-bandwidth memory, illustrating its capacity for massive parallel processing.

This diagram effectively contrasts CPU and GPU architectures, highlighting how CPUs are optimized for sequential processing while GPUs are designed for highly parallel tasks.

Non-Uniform Memory Access

Posted on by lechuck park

From DALL-E with some prompting
The image depicts the NUMA (Non-Uniform Memory Access) architecture in computer systems. Key elements include:

  1. Operating System: Manages and controls processes running on the CPU.
  2. CPU: Central Processing Units where computing tasks are executed.
  3. NUMA Nodes: Guide CPUs to use the nearest memory, with each NUMA node having memory areas closer to specific CPUs.
  4. Memory Access Paths: “Short Path” indicates a fast and low-energy memory access that is closer, while “Long Path” represents a slower and more energy-consuming memory access that is farther away.

The structure illustrates that memory access times in a NUMA system are not uniform across all memory, suggesting that memory access optimization can enhance overall system performance.