Cross-modal Retrieval with Correspondence Autoencoder

The problem of cross-modal retrieval, e.g., using a text query to search for images and vice-versa, is considered in this paper. A novel model involving correspondence autoencoder (Corr-AE) is proposed here for solving this problem. The model is constructed by correlating hidden representations of two uni-modal autoencoders. A novel optimal objective, which minimizes a linear combination of representation learning errors for each modality and correlation learning error between hidden representations of two modalities, is used to train the model as a whole. Minimization of correlation learning error forces the model to learn hidden representations with only common information in different modalities, while…

• NN
  National Natural Science Foundation of China

  Award: 61273365
• BU
  Beijing University of Posts and Telecommunications
• MO
  Ministry of Science and Technology of the People's Republic of China

  Award: 2012AA011103