Advanced flow-control mechanisms for the sockets direct protocol over InfiniBand

P. Balaji, S. Bhagvat, D. K. Panda, R. Thakur, W. Gropp

Research output: Chapter in Book/Report/Conference proceedingConference contribution


The Sockets Direct Protocol (SDP) is an industry standard to allow existing TCP/IP applications to be executed on high-speed networks such as InfiniBand (IB). Like many other high-speed networks, IB requires the receiver process to inform the network interface card (NIC), before the data arrives, about buffers in which incoming data has to be placed. To ensure that the receiver process is ready to receive data, the sender process typically performs flow-control on the data transmission. Existing designs of SDP flow-control are naive and do not take advantage of several interesting features provided by IB. Specifically, features such as RDMA are only used for performing zero-copy communication, although RDMA has more capabilities such as sender-side buffer management (where a sender process can manage SDP resources for the sender as well as the receiver). Similarly, IB also provides hardware flow-control capabilities that have not been studied in previous literature. In this paper, we utilize these capabilities to improve the SDP flow-control over IB using two designs: RDMA-based flow-control and NIC-assisted RDMA-based flow-control. We evaluate the designs using micro-benchmarks and real applications. Our evaluations reveal that these designs can improve the resource usage of SDP and consequently its performance by an order-of-magnitude in some cases. Moreover we can achieve 10-20% improvement in performance for various applications.

Original languageEnglish (US)
Title of host publication2007 International Conference on Parallel Processing, ICPP
StatePublished - 2007
Externally publishedYes
Event36th International Conference on Parallel Processing in Xi'an, ICPP - Xi'an, China
Duration: Sep 10 2007Sep 14 2007

Publication series

NameProceedings of the International Conference on Parallel Processing
ISSN (Print)0190-3918


Other36th International Conference on Parallel Processing in Xi'an, ICPP

ASJC Scopus subject areas

  • Hardware and Architecture
  • General Engineering


Dive into the research topics of 'Advanced flow-control mechanisms for the sockets direct protocol over InfiniBand'. Together they form a unique fingerprint.

Cite this