ProteinBERT: a universal deep-learning model of protein sequence and function
Hebrew University of Jerusalem · Ben-Gurion University of the Negev
Abstract
SUMMARY: Self-supervised deep language modeling has shown unprecedented success across natural language tasks, and has recently been repurposed to biological sequences. However, existing models and pretraining methods are designed and optimized for text analysis. We introduce ProteinBERT, a deep language model specifically designed for proteins. Our pretraining scheme combines language modeling with a novel task of Gene Ontology (GO) annotation prediction. We introduce novel architectural elements that make the model highly efficient and flexible to long sequences. The architecture of ProteinBERT consists of both local and global representations, allowing end-to-end processing of these types of inputs and…
Citation impact
- FWCI
- 73.37
- Percentile
- 100%
- References
- 32
Authors
5Topics & keywords
- Computer science
- Artificial intelligence
- Deep learning
- Annotation
- Task (project management)
- Function (biology)
- Language model
- Scheme (mathematics)
- Quality Education