High Performance Computing Systems and Applications

- Overview

High performance computing (HPC) systems are groups of computers that work together to solve computationally intensive tasks. HPC systems can be custom-built supercomputers or clusters of individual computers. HPC systems can be run on-premises, in the cloud, or as a hybrid of both. 

HPC systems are built with high-performance processors, high-speed memory and storage, and other advanced components. In a cluster, each computer is called a node, and each node is responsible for a different task. The nodes are connected through a network and designed to function as a single, powerful computing resource.

Supercomputing is one of the most well-known HPC solutions. In a supercomputer, thousands of compute nodes work together to complete one or more tasks, which is called parallel processing. This is similar to having thousands of PCs networked together, combining compute power to complete tasks faster.

Please refer to the following for more information:

• Wikipedia: High Performance Computing

- Advanced AI Solutions and High Performance Architecture

Artificial intelligence (AI) solutions require purpose-built architecture to enable rapid model development, training, tuning and real-time intelligent interaction. 

High Performance Architecture (HPA) is critical at every stage of the AI ​​workflow, from model development to deployment. Without the right foundation, companies will struggle to realize the full value of their investments in AI applications.

HPA strategically integrates high-performance computing workflows, AI/ML development workflows, and core IT infrastructure elements into a single architectural framework designed to meet the intense data requirements of advanced AI solutions. 

• High-Performance Computing (HPC): Training and running modern AI engines requires the right amount of combined CPU and GPU processing power. 
• High-performance storage: Training AI/ML models requires the ability to reliably store, clean, and scan large amounts of data. 
• High-performance networks: AI/ML applications require extremely high-bandwidth and low-latency network connections. 
• Automation, orchestration, software and applications: HPA requires the right mix of data science tools and infrastructure optimization. 
• Policy and governance: Data must be protected and easily accessible to systems and users. 
• Talent and skills: Building AI models and maintaining HPA infrastructure requires the right mix of experts.

- High-performance Computing (HPC) vs. Supercomputers

High-performance computing (HPC) and supercomputers are both types of computing that use advanced techniques and systems to solve complex problems. 

HPC is sometimes used as a synonym for supercomputing. However, in other contexts, "supercomputer" is used to refer to a more powerful subset of "high-performance computers". 

Here are some differences between HPC and supercomputers:

• Hardware configuration and architecture: HPC systems are typically more flexible and scalable than supercomputers.
• Hardware: Supercomputers are made up of thousands of compute nodes that work together to complete tasks. HPC systems are built using massive clusters of servers with one or more central processing units (CPUs).
• Security: HPC is more of a headache for security. With connections that may run the gamut from secure onsite servers to publicly managed cloud instances and private cloud deployments, ensuring security across the HPC continuum is inherently challenging.

HPC systems are designed to handle large amounts of data, perform intricate simulations, and deliver results at high speeds. They are often measured in terms of floating-point operations per second (FLOPS). 

Supercomputers are a class of computing machines that are built to handle tasks of unprecedented complexity. Supercomputing clusters are used by AI researchers to test model designs, hyperparameters, and datasets. They can improve AI models and advance AI capabilities. 

HPC can be used for tasks such as: Handling the complex partial differential equations used to express the physics of weather. Supercomputers can be used for tasks such as: Real-time AI applications like autonomous cars and robotics

[Lake Brienz, Switzerland]

- Why Do We  Need the HPC?

Specialized computing resources were necessary to help researchers and scientists extracts insights from massive data sets. Scientist need HPC because they hit a tipping point. At some point in research, there is a need to:

• Expand the current study area (regional → national → global)
• Integrate new data
• Increase model resolution

But … processing on my desktop or a single server no longer works. Some typical computational barriers:

• Time – processing on local systems is too slow or not feasible
• CPU Capacity -- Can only run one model at a time
• Develop, implement, and disseminate state-of-the-art techniques and tools so that models are more effectively applied to today’s decision-making
• Management of Computer Systems – Science Groups don’t want to purchase and manage local computer systems – they want to focus on science

- Exascale Computing

- Memory Centric Computing

- Applications of HPC