PUMPS+AI Summer School, 2025, July 14-18th

PUMPS+AI was selected an ACM Europe Seasonal School in 2023 and 2024, this year's selection is still pending.

 

The 15th edition of the Programming and Tuning Massively Parallel Systems + Artificial Intelligence summer school (PUMPS+AI) is aimed at enriching the skills of researchers, graduate students and instructors with cutting-edge technique and hands-on experience in developing applications for many-core processors with massively parallel computing resources like GPU accelerators, with a special focus on artificial intelligence.

This year's PUMPS+AI provides a sneak peak of new material to be released in the upcoming 5th edition of the textbook Programming Massively Parallel Processors. This material covers topics such as large language models, matrix multiplication optimization techniques, wavefront algorithms, and the latest multi-GPU libraries.

The Barcelona Supercomputing Center (BSC) does research around GPU architecture and programming languages for parallel computing. Please contact us for possible collaborations.

  • Summer School Co-Directors: Mateo Valero (BSC and UPC) and Wen-mei Hwu (University of Illinois at Urbana-Champaign / NVIDIA)
  • Local Organizers: Antonio J. Peña (Chair, BSC/UPC), Marc Jordà (BSC), Mar Bonet (BSC), Oscar Amoros (UPC), Shuxin Zheng (BSC/UPC)
  • Dates:
    • Applications due: 4th May
      • Due to space limitations, early application is strongly recommended. You may also be suggested to attend an online prerequisite training on basic CUDA programming before joining PUMPS.
    • Notification of acceptance: 12th May
  • Location: UPC Campus Nord, Barcelona (Spain)
  • Organized by:
    • Barcelona Supercomputing Center (BSC)
    • Universitat Politècnica de Catalunya (UPC)
    • University of Illinois at Urbana-Champaign (University of Illinois)
    • HiPEAC Network of Excellence (HiPEAC)
  • The following is a list of some of the topics that will be covered during the course:
    • Deep Learning / AI engine internals
    • High-level programming models
    • CUDA Algorithmic Optimization Strategies
    • Dealing with Sparse and Dynamic data
    • Efficiency in Large Data Traversal
    • Reducing Output Interference
    • Controlling Load Imbalance and Divergence
    • Acceleration of Collective Operations
    • Dynamic Parallelism and HyperQ
    • Debugging and Profiling CUDA Code
    • Multi-GPU Execution
    • Architecture Trends and Implications
    • Introduction to OmpSs and to the Paraver analysis tool
    • OmpSs: Leveraging GPU/CUDA Programming
    • Hands-on Labs: CUDA Optimizations on Scientific Codes; OmpSs Programming and Tuning
  • Instructors:
  • Prerequisites for the course are:
    • Basic CUDA knowledge is required to attend the course. Applicants that cannot certify their experience in CUDA programming will be asked to take a short on-line course covering the necessary introductory topics
    • C, C++, Java, or equivalent programming knowledge. Skills in parallel programming will be helpful

Preliminary Overview

  • Registration for the course is free for attendees from academia and public institutions. Please note that travel, lodging, and meals are not covered. Applicants from non-academic institutions (companies), please contact us by email at pumps at bsc.es for sponsorship possibilities.
  • By the end of the summer school, participants will:
    • Be able to design algorithms (including deep learning / AI) that are suitable for accelerators.
    • Understand the most important architectural performance considerations for developing parallel applications.
    • Be exposed to computational thinking skills for accelerating applications in science and engineering.
    • Engage computing accelerators on science and engineering breakthroughs.
  • Programming Languages: CUDA, MPI, NCCL, NVSHMEM OmpSs
  • Hands-on Labs: Afternoon labs with teaching assistants for each audience/level.
    • Participants are expected to bring their own laptops to access the servers with GPU accelerators.
    • The afternoon lab sessions will provide hands-on experience with various languages and tools covered in the lectures and will comprise a brief introduction to the programming assignments, followed by independent work periods. Teaching assistants will be available in person and on the web to help with assignments.