Leveraging Service Meshes as a New Network Layer

As modern cloud services have scaled out, applications have moved from relatively monolithic designs to highly modularized fleets of microservices that communicate among each other to perform application-level tasks. These microservices effectively form a network at the application layer, and service mesh frameworks have recently emerged to factor out microservices' common communication functionality. This paper seeks to highlight the emergence of service meshes as what is effectively a new layer in the networking stack, and the associated new challenges and opportunities. As a case study, we leverage the fact that service meshes can be better informed about application needs, and design a system that utilizes provenance tracing within the service mesh to perform cross-layer prioritization of latency-sensitive requests, within an application serving a mix of workloads. Broadly speaking, we believe that as applications factor out communication into service meshes, an exciting new domain is opening that can utilize techniques from the networking community to improve application performance.