This paper addresses the problem of automatic image annotation for semantic retrieval of images. We propose an image classification system that is capable of recognizing several image categories. The system is based on the support vector machine and a set of image features that includes MPEG-7 visual descriptors and a custom feature. The system is evaluated on a large dataset consisting of 14400 images in four categories - landscape, cityscape, vehicle and portrait. We find that the proposed edge direction histogram and the MPEG-7 edge histogram perform better than other features in this application. Experiment results indicate that the pair- wise SVM approach performs better than the one-versus-all SVM approach. The pair-wise method with confidence score voting has better classification rates compared to the pair-wise method with majority voting.