Many successful object detection systems characterize object classes with a statistical profile over a large number of local features. We present an enhancement to this method that learns to assemble local features into features that capture more global properties such as body shape and color distribution. The system then learns to combine these estimated global features to improve object detection accuracy. In our approach, each candidate object detection from an off-the-shelf gradient-based detection system is transformed into a conditional random field. This CRF is used to extract a most likely object silhouette, which is then processed into features based on color and shape. Finally, we show that on the difficult Pascal VOC 2007 data set, detection rates can be improved by combining these global features with the local features from a state-of-the-art gradient based approach.