This title appears in the Scientific Report :
2022
Please use the identifier:
http://dx.doi.org/10.5281/ZENODO.7391024 in citations.
Efficient Distributed GPU Programming for Exascale
Efficient Distributed GPU Programming for Exascale
Over the past years, GPUs became ubiquitous in HPC installations around the world. Today, they provide the majority of performance of some of the largest supercomputers (e.g. Summit, Sierra, JUWELS Booster). This trend continues in the recently deployed and upcoming Pre-Exascale and Exascale systems...
Saved in:
Personal Name(s): | Herten, Andreas (Corresponding author) |
---|---|
Oden, Lena / Hrywniak, Markus / Kraus, Jiri / Garcia De Gonzalo, Simon | |
Contributing Institute: |
Jülich Supercomputing Center; JSC |
Imprint: |
2022
|
DOI: |
10.5281/ZENODO.7391024 |
Conference: | Supercomputing Conference, Dallas (USA), 2022-11-14 - 2022-11-14 |
Document Type: |
Lecture |
Research Program: |
Domain-Specific Simulation & Data Life Cycle Labs (SDLs) and Research Groups Cross-Domain Algorithms, Tools, Methods Labs (ATMLs) and Research Groups Future Computing & Big Data Systems |
Publikationsportal JuSER |
Over the past years, GPUs became ubiquitous in HPC installations around the world. Today, they provide the majority of performance of some of the largest supercomputers (e.g. Summit, Sierra, JUWELS Booster). This trend continues in the recently deployed and upcoming Pre-Exascale and Exascale systems (LUMI, Leonardo; Frontier, Perlmutter): GPUs are chosen as the core computing devices to enter this next era of HPC.To take advantage of future GPU-accelerated systems with tens of thousands of devices, application developers need to have the proper skills and tools to understand, manage, and optimize distributed GPU applications. In this tutorial, participants will learn techniques to efficiently program large-scale multi-GPU systems. While programming multiple GPUs with MPI is explained in detail, also advanced tuning techniques and complementing programming models like NCCL and NVSHMEM are presented. Tools for analysis are shown and used to motivate and implement performance optimizations. The tutorial teaches fundamental concepts that apply to GPU-accelerated systems in general, taking the NVIDIA platform as an example. It is a combination of lectures and hands-on exercises, using Europe’s fastest supercomputer, JUWELS Booster, for interactive learning and discovery. |