Research engineer optimizing attention for large language model training
Advances attention optimization for large language model training, after deep learning algorithm engineering at NVIDIA. Commits to Dao-AILab/flash-attention, a library for exact attention, and NVIDIA/Megatron-LM, a library for training transformer models.