Dot Product
The dot product of two equal-length vectors is the sum of their element-wise products, yielding a scalar. It measures how much two vectors align and is the core of attention.
What is Dot Product?
The dot product of two equal-length vectors is the sum of their element-wise products, yielding a scalar. It measures how much two vectors align and is the core of attention.
The dot product of two equal-length vectors is the sum of their element-wise products, yielding a scalar. It measures how much two vectors align and is the core of attention.
Where is it used?
Self-attention computes the dot product between query and key vectors to produce attention scores; cosine similarity in embedding search is a normalized dot product.
How to build it
Compute `torch.dot(a, b)` on two vectors, compare with `(a * b).sum()`, and observe that orthogonal vectors produce a dot product of zero.
Code
A practical example: