Distributed Foundation Models for Multi-Modal Learning in 6G Wireless Networks

Benefiting from the ability to process and integrate data from various modalities, multi-modal foundation models (FMs) facilitate potential applications across a range of fields, including computer vision (CV), natural language processing (NLP), and diverse multi-modal applications such as imagetext retrieval. Currently, FMs are deployed on computing clusters for training and inference to meet their considerable computational demands. In the foreseeable future, the parameter size of FMs is expected to evolve further, posing challenges to both computation resources and energy supply. Fortunately, leveraging the next-generation wireless networks (6G) to aggregate substantial computation resources and multi-modal…

• NS
  National Science Foundation

  Award: CNS-2107216,CNS-2128368,CMMI-2222810,ECCS-2302469
• NN
  National Natural Science Foundation of China

  Award: 62325108,62341131,U23A20281,61971257