RiNALMo: general-purpose RNA language models can generalize well on structure prediction tasks

While RNA has recently been recognized as an interesting small-molecule drug target, many challenges remain to be addressed before we take full advantage of it. This emphasizes the necessity to improve our understanding of its structures and functions. Over the years, sequencing technologies have produced an enormous amount of unlabeled RNA data, which hides a huge potential. Motivated by the successes of protein language models, we introduce RiboNucleic Acid Language Model (RiNALMo) to unveil the hidden code of RNA. RiNALMo is the largest RNA language model to date, with 650M parameters pre-trained on 36M non-coding RNA sequences from several databases. It can extract hidden knowledge and capture the…

• NR
  National Research Foundation
• AF
  Agency for Science, Technology and Research

  Award: I23D1AG079
• NR
  National Research Foundation Singapore

  Award: NRF-CRP27-2021RS-0001
• NS
  National Supercomputing Centre Singapore