Hierarchical Deep Click Feature Prediction for Fine-Grained Image Recognition

The click feature of an image, defined as the user click frequency vector of the image on a predefined word vocabulary, is known to effectively reduce the semantic gap for fine-grained image recognition. Unfortunately, user click frequency data are usually absent in practice. It remains challenging to predict the click feature from the visual feature, because the user click frequency vector of an image is always noisy and sparse. In this paper, we devise a Hierarchical Deep Word Embedding (HDWE) model by integrating sparse constraints and an improved RELU operator to address click feature prediction from visual features. HDWE is a coarse-to-fine click feature predictor that is learned with the help of an…

• NN
  National Natural Science Foundation of China

  Awards: 61602136, 61622205, 61601158, 61836002
• AR
  Australian Research Council

  Awards: DP-180103424, FL-170100117