In content-based image retrieval (CBIR), feature aggregation is an approach to obtain image similarity by combining multiple feature distances. Most existing feature aggregation methods focus on heuristic-based or linear combination functions, which cannot sufficiently explore the interdependencies between features. Instead, a single aggregation function is always applied to all query images without considering the special features of each query image. In this paper, aggregation is formulated as a classification problem in a feature similarity space and solved by support vector machines (SVMs). The new method can learn an aggregation function for each query image and extend the linear aggregation to a nonlinear one using the kernel trick. Experiments demonstrate that the image retrieval performance of the proposed method is superior.