Color Attributes for Object Detection

State-of-the-art object detectors typically use shape information as a low level feature representation to capture the local structure of an object while ignoring the color information. On the other hand, and in contrast to object detection, color has been shown to yield excellent results in combination with shape features for image classification. The few approaches which do apply color for object detection focus on a single class such as pedestrians. However, the problem of generic object detection is more challenging and the contribution of color to object detection on standard benchmark datasets such as the PASCAL VOC is yet to be investigated.

In this work we investigate extending color information in two existing methods for object detection, specifically the part-based detection framework and the Efficient Subwindow Search approach. We show that the early fusion of shape and color, as is popular in image classification, leads to a significant drop in performance for object detection. Moreover, such approaches also provide sub-optimal results for object categories with varying importance of color and shape. Therefore, we propose the use of color attributes as an explicit color representation for object detection. Color attributes are compact, computationally efficient, and when combined with traditional shape features improve the performance significantly. The proposed approach is tested the PASCAL VOC 2007 and 2009 datasets. We also introduce a new dataset consisting of cartoon character images in which color plays a pivotal role.

Visualization of learned part-based models with color attributes. Both the HOG and color attribute components of our trained models are shown. Each cell is represented by the color which is obtained by multiplying the SVM weights for the 11 CN bins with a color representative of the color names. Top row: the HOG and color attribute models for pottedplant and horse. Bottom row: Marge and Tweety models. In the case of horse, the brown color of the horse together with a person sitting on top of it is prominent. Similarly, the model is able to capture the blue hair of Marge and orange feet of Tweety.

Code Available

Code to run Color-HOG detector with trained models for the PASCAL VOC 2007 available here: (Color-HOG detector code )

Cartoon Dataset

please email at fahad@cvc.uab.es to obtain the dataset:

Literature

Fahad Shahbaz Khan, Rao Muhammad Anwer, Joost van de Weijer, Andrew D. Bagdanov, Maria Vanrell and Antonio M. Lopez  Color Attributes for Object Detection , Proc. CVPR 2012 , Rhode Islands, USA, 2012. "(Poster)