Vector Space
A vector space is the mathematical space in which embeddings live — every token or sentence is a point in d_model-dimensional space, and directions in this space can encode semantic relationships.
What is Vector Space?
A vector space is the mathematical space in which embeddings live — every token or sentence is a point in d_model-dimensional space, and directions in this space can encode semantic relationships.
A vector space is the mathematical space in which embeddings live — every token or sentence is a point in d_model-dimensional space, and directions in this space can encode semantic relationships.
Where is it used?
In word2vec's space, `king - man + woman ≈ queen` demonstrates that directions encode gender, royalty, and other concepts; the same geometry underlies LLM hidden states.
How to build it
Load GloVe vectors, compute `word2vec['king'] - word2vec['man'] + word2vec['woman']`, and find the nearest neighbor by cosine similarity to verify analogies emerge in the space.