Derivatives
A derivative measures the instantaneous rate of change of a function with respect to one variable. It tells you how much the output shifts when you nudge the input, and is the foundation of gradient-based learning.
What is Derivatives?
A derivative measures the instantaneous rate of change of a function with respect to one variable. It tells you how much the output shifts when you nudge the input, and is the foundation of gradient-based learning.
A derivative measures the instantaneous rate of change of a function with respect to one variable. It tells you how much the output shifts when you nudge the input, and is the foundation of gradient-based learning.
Where is it used?
Backpropagation computes derivatives of the loss with respect to every weight in an LLM; PyTorch's autograd builds a computation graph and applies the chain rule automatically.
How to build it
Use `torch.autograd` by setting `requires_grad=True` on a tensor, computing a loss, calling `.backward()`, and inspecting `.grad` to see the derivative.
Code
A practical example: