Triton Tutorial Practice: 05 Layer Normalization

學習目標

• Implementing backward pass in Triton.
• Implementing parallel reduction in Triton.

Vector-Jacobian Products

https://arxiv.org/abs/2104.00219

The Fundamentals of Autograd

what-is-automatic-differentiation

Reference

• https://isamu-website.medium.com/understanding-triton-tutorials-part-2-f6839ce50ae7
• https://arxiv.org/abs/2104.00219