Current content-based image retrieval (CBIR) methods are inefficient due to several fundamental problems such as (1) the sparsity and reliability of tags, (2) the representation of image semantics, and (3) the fusion of textual and visual relevance. This analytical paper starts with a survey of related works and proposes a framework to address these issues.

The authors note that recent works have proposed a variety of feature-based methods, ranging from color histograms to elementary feature descriptors (HTML tags). There are two general strategies: early fusion (which integrates individual features before measuring the visual similarities) and late fusion (which uses individual features to compute similarities separately, and then aggregates them). Efficient algorithms to hash and measure similarities have been developed, along with metrics for machine learning and text tagging. Hypergraphs for cross-media retrieval and text-query ranking have been widely used in recent works.

Using semantic hypergraphs, the authors have proposed a method centered on tag compilation, semantics modeling, and relevance fusion. Image search yields images often containing unreliable or missing tags. Based on performance measure metrics, an optimization problem is set up to fill the metric and address unreliable tags. Using semantic hypergraphs of images in a database, the fusion of features is achieved to build a “relevance” list to set up an algorithm to trade off between textual and visual relevance.

The authors report test results for the proposed method using two datasets of 55,615 and 25,000 images, 1,000 and 457 tags, and 81 and 38 semantic concepts. The paper is well written and contains 47 references. It should interest readers working in CBIR.