GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding.

Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, Zhifeng Chen

Browse the full ICLR paper archive.