Mining flipping correlations from large datasets with taxonomies

Marina Barsky, Sangkyum Kim, Tim Weninger, Jiawei Han

Research output: Contribution to journalArticle

Abstract

In this paper we introduce a new type of pattern - a flipping correlation pattern. The flipping patterns are obtained from contrasting the correlations between items at different levels of abstraction. They represent surprising correlations, both positive and negative, which are specific for a given abstraction level, and which "flip" from positive to negative and vice versa when items are generalized to a higher level of abstraction. We design an efficient algorithm for finding flipping correlations, the Flipper algorithm, which outperforms näive pattern mining methods by several orders of magnitude. We apply Flipper to real-life datasets and show that the discovered patterns are non-redundant, surprising and actionable. Flipper finds strong contrasting correlations in itemsets with low-to-medium support, while existing techniques cannot handle the pattern discovery in this frequency range.

Original languageEnglish (US)
Pages (from-to)370-381
Number of pages12
JournalProceedings of the VLDB Endowment
Volume5
Issue number4
DOIs
StatePublished - Dec 2011

Keywords

  • Flipping correlation
  • Itemset mining

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Mining flipping correlations from large datasets with taxonomies'. Together they form a unique fingerprint.

  • Cite this