Falcon: Fair and Efficient Online File Transfer Optimization

Md Arifuzzaman, Brian Bockelman, James Basney, Engin Arslan

Research output: Contribution to journalArticlepeer-review


Research networks provide high-speed wide-area network connectivity between research and education institutions to facilitate large-scale data transfers. However, scalability issues of legacy transfer applications such as scp and FTP hinder the effective utilization of these networks. Although researchers extended the legacy transfer applications to increase their performance by exploiting I/O and network parallelism, these solutions necessitate users to fine-tune parallelism level, a task that is challenging even for experienced users due to the dynamic nature of networks. In this article, we propose an online optimization algorithm, Falcon, to tune the degree of parallelism for file transfers to maximize transfer throughput while keeping system overhead at a minimum. As research networks are shared infrastructures, we introduce a game theory-inspired novel utility function to evaluate the performance of various parallelism levels such that competing transfers are guaranteed to converge to a fair and stable solution. We assessed the performance of Falcon in isolated and production high-speed networks and found that it can discover optimal transfer parallelism in as little as 20 seconds and outperform the state-of-the-art solutions by more than $2\times$2×. Moreover, Falcon is guaranteed to converge to Nash Equilibrium when multiple transfers compete for the same resources with the help of its game theory-inspired utility function. Finally, we demonstrate that Falcon can also be used as a central transfer scheduler to speed up convergence time, increase stability, and enforce system/user-level resource limitations in shared networks.

Original languageEnglish (US)
Pages (from-to)2265-2278
Number of pages14
JournalIEEE Transactions on Parallel and Distributed Systems
Issue number8
StatePublished - Aug 1 2023


  • Online transfer optimization
  • file transfer tuning
  • high-speed networks
  • throughput optimization in research networks

ASJC Scopus subject areas

  • Signal Processing
  • Hardware and Architecture
  • Computational Theory and Mathematics


Dive into the research topics of 'Falcon: Fair and Efficient Online File Transfer Optimization'. Together they form a unique fingerprint.

Cite this