LRZ Bootcamp With NIVIDA on Accelerated Distributed Computing

The LRZ (Leibniz Supercomputing Centre) BDAI Team, together with NVIDIA, is organizing a Bootcamp on Accelerated Distributed Computing - powered by NVIDIA on March 26, 2025, at 9 AM.

The key topics addressed are:
1. Training Parallelization - Various methods to split and optimize training across multiple systems, including data/model parallelism and advanced techniques like ZeRO and Mixture-of-Experts

2. Communication Systems - How components communicate within and between nodes using tools like NCCL

3. Implementation Tools - Frameworks like NeMo Megatron and Profiler

You can register yourself for the bootcamp under the following link, where you can also find a detailed agenda.