Image Embeddings
Image embeddings are dense vector representations of an image produced by a vision encoder, capturing visual semantics that can be projected into the LLM's token embedding space.
What is Image Embeddings?
Image embeddings are dense vector representations of an image produced by a vision encoder, capturing visual semantics that can be projected into the LLM's token embedding space.
Image embeddings are dense vector representations of an image produced by a vision encoder, capturing visual semantics that can be projected into the LLM's token embedding space.
Where is it used?
CLIP and SigLIP produce image embeddings used by LLaVA and Qwen-VL; the projection layer maps 1024-dim CLIP features to 4096-dim Llama hidden states.
How to build it
Load `CLIPModel.from_pretrained("openai/clip-vit-large-patch14")`, run `model.get_image_features(pixel_values=image)`, and project with `nn.Linear(1024, d_model)` into the LLM space.