Fast, low-memory algorithm for construction of nanosecond level snapshots of financial markets

Robert Sinkovits, Tao Feng, Mao Ye

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a fast, low-memory algorithm for constructing an order-by-order level snapshot of financial markets with nanosecond resolution. This new implementation is 20-30x faster than an earlier version of the code. In addition, since message data are retained only for as long as it they are needed, the memory footprint is greatly reduced. We find that even the heaviest days of trading spanning the NASDAQ, NYSE and BATS exchanges can now easily be handled using compute nodes with very modest memory (~ 4 GB). A tradeoff of this new approach is that the ability to efficiently manage large numbers of small files is more critical. We demonstrate how we can accommodate these new I/O requirements using the solid-state storage devices (SSDs) on SDSC's Gordon system.

Original languageEnglish (US)
Title of host publicationProceedings of the XSEDE 2014 Conference
Subtitle of host publicationEngaging Communities
PublisherAssociation for Computing Machinery
ISBN (Print)9781450328937
DOIs
StatePublished - 2014
Event2014 Annual Conference on Extreme Science and Engineering Discovery Environment, XSEDE 2014 - Atlanta, GA, United States
Duration: Jul 13 2014Jul 18 2014

Publication series

NameACM International Conference Proceeding Series

Other

Other2014 Annual Conference on Extreme Science and Engineering Discovery Environment, XSEDE 2014
Country/TerritoryUnited States
CityAtlanta, GA
Period7/13/147/18/14

Keywords

  • High-frequency trading
  • Parallel computing
  • Performance tuning

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'Fast, low-memory algorithm for construction of nanosecond level snapshots of financial markets'. Together they form a unique fingerprint.

Cite this