A Probabilistic Approach to Cross-region Matching based Image Retrieval
RIS ID
130873
Abstract
With deep convolutional features, cross-region matching (CRM) has recently shown superior performance on image retrieval. It evaluates image similarity by comparing image regions at different locations and scales, and is therefore more robust to geometric variance of objects. This paper firstly scrutinizes CRM based image retrieval to provide a rigorous probabilistic interpretation by following the probability ranking principle. In addition to manifesting the assumptions implicitly taken by CRM, our interpretation highlights a fundamental issue hindering the performance of CRM—when comparing two image regions, CRM ignores modeling the distribution of the visual concept class associated with an image region, making the similarity comparison less precise. Taking advantage of the unprecedented representation capability of deep convolutional features, this paper proposes one approach to tackle that issue. It treats locally clustered image regions as a pseudo-labeled class sharing the same visual concept, and utilizes them to model the distribution of the visual concept class associated with an image region. Both non-parametric and parametric methods are developed for this purpose, with careful probabilistic justification. Extensive experimental study on multiple benchmark datasets demonstrates the superior performance of the proposed pseudo-label approach to CRM and other comparable methods, with the maximum improvement of more than 10 percentage points over CRM.
Grant Number
ARC/DE160100241
Publication Details
Gao, Z., Wang, L. & Zhou, L. (2019). A Probabilistic Approach to Cross-region Matching based Image Retrieval. IEEE Transactions on Image Processing, 28 (3), 1191-1204.