Memory-Centric HPC

- Overview

Memory-centric computing, also known as processing in memory (PIM), is a computer architecture paradigm that allows data processing to take place in, near, or on devices that store or generate data. This paradigm can make computing more efficient by offering an alternative to the traditional processor-centric paradigm, which only performs data processing in the processor. 

Memory-centric computing aims to enable computation in and near all places where data is stored and generated. For example, data processing could take place near sensors, memory, or storage devices. 

The "memory wall" problem was identified in the 1990s to describe the issue of microprocessor performance improving faster than DRAM memory speed. This narrowing gap between the speed of CPU and memory means that processor cores will increasingly sit idle waiting for data. 

Memory-centric computing can improve complex event processing, deliver faster reporting and product releases, and provide quicker and more accurate decision-making.

- Memory (Bandwidth and Capacity) Wall

The "memory wall" problem, originally posed by Wulf and McKee in the 1990s, states that microprocessor performance is increasing much faster than DRAM memory speed. This trend makes the memory subsystem one of the most critical system-level performance bottlenecks. 

In addition to the memory "bandwidth" wall, computer system designers are also noticing the emergence of a new memory wall in the data center, one of memory "capacity," where peak imbalances in compute versus memory capacity require hyperscalers to overprovision each server's memory is sized for worst-case usage, resulting in significant memory underutilization. 

To overcome the memory wall problem, computer architects have conducted two important studies, one is memory processing (to solve the problem of memory bandwidth) and the other is memory decomposition (to solve the problem of memory capacity).

- Memory-centric Computing vs. Processor-centric Computing

Modern computing systems are processor-centric. Data processing (i.e. computation) occurs only in the processor (e.g. CPU, GPU, FPGA, ASIC). Therefore, data needs to be moved from where it is generated/captured (e.g., sensors) and where it is stored (e.g., storage and storage devices) to the processor before it can be processed. 

The processor-centric design paradigm significantly limits the performance and energy efficiency as well as scalability and sustainability of modern computing systems. 

Many studies have shown that even the most powerful processors and accelerators waste a significant portion (e.g., >60%) of the time waiting for data and energy moving data between storage/memory units to the processor. 

This is true even though most of the hardware space of such systems is dedicated to data storage and communication (e.g., multi-level caches, DRAM chips, storage systems, and interconnects). 

- The Promising Frontiers of Memory-Centric Computing

Memory-centric computing aims to bring computing power to and near wherever data is generated and stored. Therefore, it can greatly reduce the huge negative impact of data access and data movement on performance and energy by essentially avoiding data movement and reducing data access latency and energy. 

Many recent studies have shown that memory-centric computing can greatly improve system performance and energy efficiency. Major industrial suppliers and startups have also recently introduced memory chips with complex computing capabilities.

Memory-centric computing enables balanced and efficient system designs, where compute and memory access are fundamentally balanced and the processor-memory dichotomy is eliminated. These systems can provide higher performance and more efficient than existing processor-centric systems. They can also enable potential new applications and computing platforms. 

However, as with any new paradigm, memory-centric computing systems pose significant adoption challenges.