Wen mei W. Hwu, David B. Kirk, Izzat El Hajj

Research output: Chapter in Book/Report/Conference proceedingChapter


This chapter introduces the book by first giving an account of the historic events that pushed heterogeneous parallel computing into the mainstream. It distinguishes between latency-oriented multicore CPUs and throughput oriented many-thread GPUs, the two main styles of computing devices in modern heterogeneous computing systems. The chapter then goes into the main reasons for applications to demand more speed and the Amdahl’s law, an important observation about the parallelization of real application. It then explains how the book addresses the main challenges in parallel algorithms and parallel programming and how the skills that have been learned from the book based on CUDA, the language of choice for programming examples, and exercises in this book can be generalized into other parallel programming interfaces. Finally, the chapter highlights the overarching goals for the book and the contents and rationales of the rest of the chapters.

Original languageEnglish (US)
Title of host publicationProgramming Massively Parallel Processors
Subtitle of host publicationa Hands-on Approach, Fourth Edition
Number of pages20
ISBN (Electronic)9780323912310
ISBN (Print)9780323984638
StatePublished - Jan 1 2022


  • Amdahl’s law
  • GPU computing
  • MPI
  • OpenMP
  • Parallel computing
  • computational thinking
  • data characteristics
  • data parallelism
  • embarrassingly parallel applications
  • heterogeneous computing
  • high-performance computing
  • latency-oriented design
  • memory bandwidth
  • parallel algorithms
  • parallel patterns
  • parallel programming
  • parallel programming languages
  • parallel programming models
  • performance optimization
  • programmer productivity
  • scalability
  • supercomputing
  • synchronization
  • throughput-oriented design
  • work distribution
  • work efficiency

ASJC Scopus subject areas

  • General Computer Science


Dive into the research topics of 'Introduction'. Together they form a unique fingerprint.

Cite this