An architectural framework for runtime optimization

Matthew C. Merten, Andrew R. Trick, Ronald D. Barnes, Erik M. Nystrom, Christopher N. George, John C. Gyllenhaal, Wen Mei W. Hwu

Research output: Contribution to journalArticlepeer-review


Wide-issue processors continue to achieve higher performance by exploiting greater instruction-level parallelism. Dynamic techniques such as out-of-order execution and hardware speculation have proven effective at increasing instruction throughput. Runtime optimization promises to provide an even higher level of performance by adaptively applying aggressive code transformations on a larger scope. This paper presents a new hardware mechanism for generating and deploying runtime optimized code. The mechanism can be viewed as a filtering system that resides in the retirement stage of the processor pipeline, accepts an instruction execution stream as input, and produces instruction profiles and sets of linked, optimized traces as output. The code deployment mechanism uses an extension to the branch prediction mechanism to migrate execution into the new code without modifying the original code. These new components do not add delay to the execution of the program except during short bursts of reoptimization. This technique provides a strong platform for runtime optimization because the hot execution regions are extracted, optimized, and written to main memory for execution and because these regions persist across context switches. The current design of the framework supports a suite of optimizations, including partial function inlining (even into shared libraries), code straightening optimizations, loop unrolling, and peephole optimizations.

Original languageEnglish (US)
Pages (from-to)567-589
Number of pages23
JournalIEEE Transactions on Computers
Issue number6
StatePublished - Jun 2001


  • Code layout
  • Dynamic optimization
  • Hardware profiling
  • Low-overhead profiling
  • Partial function inlining
  • Postlink optimization
  • Program hot spot
  • Runtime optimization
  • Trace formation and optimization

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture
  • Computational Theory and Mathematics


Dive into the research topics of 'An architectural framework for runtime optimization'. Together they form a unique fingerprint.

Cite this