Key
The key is a vector representing what a token 'offers' for matching, computed by projecting the hidden state with W_K. The dot product of a query with each key produces attention scores.
What is Key?
The key is a vector representing what a token 'offers' for matching, computed by projecting the hidden state with W_K. The dot product of a query with each key produces attention scores.
The key is a vector representing what a token 'offers' for matching, computed by projecting the hidden state with W_K. The dot product of a query with each key produces attention scores.
Where is it used?
During the KV-cache at LLM inference, keys and values for past tokens are cached so only the new token's key is computed, dramatically speeding up autoregressive generation.
How to build it
Compute `K = x @ W_K`, then `scores = Q @ K.T / sqrt(d_k)`, and inspect which keys score highest for a given query to see what the token attends to.