Prompt Tokens
Prompt tokens are the input tokens provided by the user that condition the model's generation. They include system instructions, conversation history, and the current query, all converted to token IDs by the tokenizer.
What is Prompt Tokens?
Prompt tokens are the input tokens provided by the user that condition the model's generation. They include system instructions, conversation history, and the current query, all converted to token IDs by the tokenizer.
Prompt tokens are the input tokens provided by the user that condition the model's generation. They include system instructions, conversation history, and the current query, all converted to token IDs by the tokenizer.
Where is it used?
In every LLM interaction, the prompt tokens consume part of the context window. GPT-4, Claude, and LLaMA all process the full prompt in the first forward pass. Token counting APIs help users stay within limits before sending a request.
How to build it
Tokenize the full prompt: `prompt_ids = tokenizer(prompt, return_tensors='pt').input_ids`. Count: `n_prompt = prompt_ids.shape[1]`. Ensure `n_prompt + max_new_tokens <= context_window` before generation.