TY - JOUR
T1 - Re-examination of interestingness measures in pattern mining
T2 - A unified framework
AU - Wu, Tianyi
AU - Chen, Yuguo
AU - Han, Jiawei
N1 - Funding Information:
The work was supported in part by the U.S. National Science Foundation NSF IIS-08-42769, NSF IIS-09-05215, NSF DMS-05-03981, and NSF DMS-08-06175. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the funding agencies. This paper is a major-value added extension of the paper: Tianyi Wu, Yuguo Chen and Jiawei Han, Association Mining in Large Databases: A Re-Examination of Its Measures”, Proc. 2007 Int. Conf. on Principles and Practice of Knowledge Discovery in Databases (PKDD’07), Warsaw, Poland, Sept. 2007, pp. 621–628.
Copyright:
Copyright 2010 Elsevier B.V., All rights reserved.
PY - 2010/11
Y1 - 2010/11
N2 - Numerous interestingness measures have been proposed in statistics and data mining to assess object relationships. This is especially important in recent studies of association or correlation pattern mining. However, it is still not clear whether there is any intrinsic relationship among many proposed measures, and which one is truly effective at gauging object relationships in large data sets. Recent studies have identified a critical property, null-(transaction) invariance, for measuring associations among events in large data sets, but many measures do not have this property. In this study, we re-examine a set of null-invariant interestingnessmeasures and find that they can be expressed as the generalized mathematical mean, leading to a total ordering of them. Such a unified framework provides insights into the underlying philosophy of the measures and helps us understand and select the proper measure for different applications. Moreover, we propose a new measure called Imbalance Ratio to gauge the degree of skewness of a data set. We also discuss the efficient computation of interesting patterns of different null-invariant interestingness measures by proposing an algorithm, GAMiner, which complements previous studies. Experimental evaluation verifies the effectiveness of the unified framework and shows that GAMiner speeds up the state-of-the-art algorithm by an order of magnitude.
AB - Numerous interestingness measures have been proposed in statistics and data mining to assess object relationships. This is especially important in recent studies of association or correlation pattern mining. However, it is still not clear whether there is any intrinsic relationship among many proposed measures, and which one is truly effective at gauging object relationships in large data sets. Recent studies have identified a critical property, null-(transaction) invariance, for measuring associations among events in large data sets, but many measures do not have this property. In this study, we re-examine a set of null-invariant interestingnessmeasures and find that they can be expressed as the generalized mathematical mean, leading to a total ordering of them. Such a unified framework provides insights into the underlying philosophy of the measures and helps us understand and select the proper measure for different applications. Moreover, we propose a new measure called Imbalance Ratio to gauge the degree of skewness of a data set. We also discuss the efficient computation of interesting patterns of different null-invariant interestingness measures by proposing an algorithm, GAMiner, which complements previous studies. Experimental evaluation verifies the effectiveness of the unified framework and shows that GAMiner speeds up the state-of-the-art algorithm by an order of magnitude.
KW - Association rules
KW - Frequent pattern
KW - Generalized mean
KW - Interestingness measure
KW - Null-invariant measure
UR - http://www.scopus.com/inward/record.url?scp=77956212881&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=77956212881&partnerID=8YFLogxK
U2 - 10.1007/s10618-009-0161-2
DO - 10.1007/s10618-009-0161-2
M3 - Article
AN - SCOPUS:77956212881
SN - 1384-5810
VL - 21
SP - 371
EP - 397
JO - Data Mining and Knowledge Discovery
JF - Data Mining and Knowledge Discovery
IS - 3
ER -