Characterizing the caching and synchronization performance of a multiprocessor operating system

Good cache memory performance is essential to achieving high CPU utilization in shared-memory multiprocessors. While the performance of caches is determined by both application and operating system (OS) references, most research has focused on the cache performance of applications alone. This is partially due to the difficulty of measuring OS activity and, as a result, the cache performance of the OS is largely unknown. In this paper, we characterize the cache performance of a commercial System V UNIX running on a four-CPU multiprocessor. The related issue of the performance impact of the OS synchronization activity is also studied. For our study, we use a hardware monitor that records the cache misses in the machine without perturbing it. We study three multi-processor workloads: a parallel compile, a multiprogrammed load, and a commercial database. Our results show that OS misses occur frequently enough to stall CPUs for 17-21% of their non-idle time. Further, if we include application misses induced by OS interference in the cache, then the stall time reaches 25%. A detailed analysis reveals three major sources of OS misses: instruction fetches, process migration, and data accesses in block operations. As for synchronization behavior, we find that OS synchronization has low overhead if supported correctly and that OS locks show good locality and low contention.