Vectors
A vector is an ordered 1D array of numbers representing magnitude and direction in a space. In ML, a vector typically holds a feature set, an embedding, or a set of weights for one neuron.
What is Vectors?
A vector is an ordered 1D array of numbers representing magnitude and direction in a space. In ML, a vector typically holds a feature set, an embedding, or a set of weights for one neuron.
A vector is an ordered 1D array of numbers representing magnitude and direction in a space. In ML, a vector typically holds a feature set, an embedding, or a set of weights for one neuron.
Where is it used?
Token embeddings in GPT are 768-dim (or 4096-dim for Llama-2 7B) vectors; the hidden state at every layer is a vector that flows through the residual stream.
How to build it
Create vectors with `torch.tensor([1.0, 2.0, 3.0])`, compute the L2 norm with `torch.norm(v)`, and measure the angle between two vectors using `torch.cosine_similarity`.
Code
A practical example: