Driven by the dramatic growth of data both in terms of the size and sources, learning from heterogeneous data is emerging as an important research direction for many real applications. One of the biggest challenges of this type of problem is how to meaningfully integrate heterogeneous data to considerably improve the generality and quality of the learning model. In this paper, we first present a unified learning framework that aims to leverage the structural information from two types of data heterogeneity: view heterogeneity (as in multi-view learning) and worker heterogeneity (as in crowdsourcing). The objective follows the principles of view consistency and worker consensus by minimizing the loss term with a regularized prediction tensor. We then propose to relax and solve the optimization framework with an iterative updating method. We also prove that the gradient of the most time-consuming updating block is separable with respect to the workers, which leads to a randomized algorithm with faster speed and better convergence. Finally, we compare the proposed method with several state-of-the-arts and demonstrate its effectiveness on various data sets.