Structured learning with constrained conditional models

Ming Wei Chang, Lev Ratinov, Dan Roth

Research output: Contribution to journalArticlepeer-review


Making complex decisions in real world problems often involves assigning values to sets of interdependent variables where an expressive dependency structure among these can influence, or even dictate, what assignments are possible. Commonly used models typically ignore expressive dependencies since the traditional way of incorporating non-local dependencies is inefficient and hence leads to expensive training and inference. The contribution of this paper is two-fold. First, this paper presents Constrained Conditional Models (CCMs), a framework that augments linear models with declarative constraints as a way to support decisions in an expressive output space while maintaining modularity and tractability of training. The paper develops, analyzes and compares novel algorithms for CCMs based on HiddenMarkovModels and Structured Perceptron. The proposed CCM framework is also compared to task-tailored models, such as semi-CRFs. Second, we propose CoDL, a constraint-driven learning algorithm, which makes use of constraints to guide semi-supervised learning.We provide theoretical justification for CoDL along with empirical results which show the advantage of using declarative constraints in the context of semi-supervised training of probabilistic models.

Original languageEnglish (US)
Pages (from-to)399-431
Number of pages33
JournalMachine Learning
Issue number3
StatePublished - Sep 2012
Externally publishedYes


  • Information extraction
  • Natural language processing
  • Semi-supervised learning

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence


Dive into the research topics of 'Structured learning with constrained conditional models'. Together they form a unique fingerprint.

Cite this