MS6D - Tuning Low-Overhead Node-Level Load Balancing for Computational Science and Engineering Simulations on Current and Emerging Supercomputers
2023-06-28 14:00 - 16:00 (Europe/Stockholm) | Schwarzhorn
Minisymposium

Inter-node dynamic load balancing speeds up scientific simulations on a supercomputer. Given that nodes of emerging supercomputers have a much larger number and variety of processors than previous supercomputers, as well as the continued need for scalability of applications, intra-node load balancing is increasingly needed. Repurposing inter-node load balancing techniques for intra-node load balancing is not practical due to heavyweight processor virtualization and because of the costs incurred from load balancing itself, e.g., data movement from task migration, are more intricate and harder to predict. Node-level load balancing requires novel low-cost load balancing strategies and empirical auto-tuning of such strategies. This mini-symposium brings forward state-of-the-art research involving such node-level load balancing from the level of (1) numerical algorithms via the SLATE library, (2) adaptive and intelligent runtime systems via Charm++’s node-level load balancing, (3) compilers via affinity and loop transformations in LLVM’s OpenMP, and (4) low-level runtime libraries via discussion of the Argobots runtime system. By attending this mini-symposium, one can expect lively discussions, research exchanges, and ideas for future work between the audience and presenters on support for node-level load balancing and its synergy with other techniques within HPC software.

LLVM’s OpenMP Loop Transformations to Enhance Locality in OpenMP Loop Scheduling

This talk will present OpenMP loop transformations, e.g., tiling, in a way that complements existing and novel dynamic schedules for OpenMP loops and OpenMP tasking. It can be used for auto-tuning for obtaining the optimal composition of multiple loop transformations and parameters without the cost...

Michael Kruse (Argonne National Laboratory)
---------------------
Low-Level HPC System Software for Task Parallelism

This talk will discuss the process of enhancing node-level dynamic scheduling for MPI+X applications, expressed in higher-level models for task parallelism, such as OpenMP, and capable of targeting both CPU and GPU processors. The main focus of the talk will be on low-level HPC system software impl...

Vivek Sarkar (Georgia Institute of Technology)
---------------------
Tunable Tasking in PARSEC for Numerical Linear Algebra Routines Used in HPC

This talk will show that a specialization of node-level load balancing for each numerical linear algebra routine of the SLATE library is needed due to intricacies of each routine. In particular, it will show an auto-tuned low-overhead loop scheduling strategy with customized data layouts speeds up d...

George Bosilca (University of Tennessee)
---------------------
Intelligent Node-Level Locality-Aware Load Balancing and Optimizations with Charm++'s CkLoop

In this talk, we discuss novel load balancers in Charm++/CharmMPI and the associated node-level library called CkLoop. We also will discuss how the design of these load balancers carefully re-uses intelligent heuristics from inter-node load balancers of Charm++. Furthermore, we will show how inter- ...

Kavitha Chandrasekar (University of Illinois Urbana-Champaign) and Harshitha Menon (Lawrence Livermore National Laboratory)

Domain: Computer Science, Machine Learning, and Applied Mathematics

Session Chair: Vivek Kale (Sandia National Laboratories)
