November 5th – Memory and Communication Walls in the Exascale Era of Computing

This week, we have a presentation by Mert Hidayetoglu, a PhD candidate in the area of high performance computing.

Memory and Communication Walls in the Exascale Era of Computing

As we are going into the exascale era of computing, we are racing to demonstrate technological capabilities for solving critical problems for the well-being of citizens of all countries today, such as understanding climate change or simulating COVID-19 mechanisms at molecular level. In fact, the exascale computer merely celebrates a psychological milestone because only a handful of applications will enjoy exascale deployment on early systems. Because, a plethora of today’s relevant scientific, AI, and graph-analytics workloads involve irregular and sparse computational patterns, and they therefore suffer from memory-wall and communication-wall bottlenecks. As a result, these applications utilize only a tiny portion of the theoretical performance of large-scale computing systems in practice. In this talk, I will explain memory-wall and communication-wall bottlenecks, and present Tiled SpMM and hierarchical communication techniques for overcoming those on supercomputers with multi-GPU node architecture.

Mert Hidayetoglu is a PhD candidate at ECE Illinois with research at the intersection of large-scale applications, high-performance computing, and software systems. His dissertation focuses on addressing inefficiencies with unstructured data accesses, communications, and computations on supercomputers with multi-GPU node architecture. His SC20 paper on X-ray imaging won the best paper award and his HPEC’20 paper on sparse DNN inference won MIT/Amazon/IEEE Graph Challenge. He is a 2021 recipient of ACM/IEEE-CS George Michael Memorial HPC Fellowship.