Once-for-All: Train One Network and Specialize it for Efficient\n Deployment

We address the challenging problem of efficient inference across many devices\nand resource constraints, especially on edge devices. Conventional approaches\neither manually design or use neural architecture search (NAS) to find a\nspecialized neural network and train it from scratch for each case, which is\ncomputationally prohibitive (causing $CO_2$ emission as much as 5 cars'\nlifetime) thus unscalable. In this work, we propose to train a once-for-all\n(OFA) network that supports diverse architectural settings by decoupling\ntraining and search, to reduce the cost. We can quickly get a specialized\nsub-network by selecting from the OFA network without additional training. To\nefficiently train OFA networks,…