Softermax: Hardware/Software Co-Design of an Efficient Softmax for Transformers.

Jacob R. Stevens, Rangharajan Venkatesan, Steve Dai, Brucek Khailany, Anand Raghunathan

Browse the full DAC paper archive.