Complex heterogeneity learning: A theoretical and empirical study

Data heterogeneity such as task heterogeneity, view heterogeneity, and instance heterogeneity often co-exist in many real-world applications including insider threat detection, traffic prediction, brain image analysis, quality control in manufacturing processes, etc. However, most of the existing techniques might not take fully advantage of the rich heterogeneity. To address this problem, we propose a novel graph-based approach named M³ to simultaneously model triple heterogeneity in a principled framework. The main idea is to employ the hybrid graphs to jointly model the task relatedness, view consistency, and bag-instance correlation by enhancing the labeling consistency between nearby nodes on the graphs. Furthermore, we analyze the generalization performance of the proposed method based on Rademacher complexity, which sheds light on the benefits of jointly modeling multiple types of heterogeneity. The resulting optimization problem is challenging since the objective function is non-smooth and non-convex. We propose an iterative algorithm based on block coordinate descent and bundle method to solve the problem. Experimental results on various datasets demonstrate the effectiveness of the proposed method.