An architecture for distributed real-time large-scale information processing for intelligence analysis

Eugene S. Santos, Eunice E. Santos, Eugene S. Santos

Research output: Contribution to journalConference articlepeer-review


Given a massive and dynamic space of information (nuggets) and a query to be answered, how can the correct (answer) nuggets be retrieved in an effective and efficient manner? We present a large-scale distributed real-time architecture based on anytime intelligent foraging, gathering, and matching (I-FGM) on massive and dynamic information spaces. Simply put, we envision that when given a search query, large numbers of computational processes are alerted or activated in parallel to begin identifying and retrieving the appropriate information nuggets. In particular, our approach aims to provide an anytime capability which functions as follows: Given finite computational resources, I-FGM will proceed to explore the information space and, over time, continuously identify and update promising candidate nugget, thus, good candidates will be available at anytime on request. With the computational costs of evaluating the relevance of a candidate nugget, the anytime nature of I-FGM will provide increasing confidence on nugget selections over time by providing admissible partial evaluations. When a new promising candidate is identified, the current set of selected nuggets is re-evaluated and updated appropriately. Essentially, I-FGM will guide its finite computational resources in locating the target information nuggets quickly and iteratively over time. In addition, the goal of I-FGM is to naturally handle new nuggets as they appear. A central element of our framework is to provide a formal computational model of this massive data-intensive problem.

Original languageEnglish (US)
Pages (from-to)161-171
Number of pages11
JournalProceedings of SPIE - The International Society for Optical Engineering
StatePublished - Aug 16 2004
Externally publishedYes
EventIntelligent Computing: Theory and Applications II - Orlando, FL, United States
Duration: Apr 12 2004Apr 13 2004


  • Data clustering
  • Distributed artificial intelligence
  • Intelligent information retrieval
  • Multi-agent systems
  • Pattern matching

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering


Dive into the research topics of 'An architecture for distributed real-time large-scale information processing for intelligence analysis'. Together they form a unique fingerprint.

Cite this