# Special Token Design

> Define and apply typed special tokens (system, user, assistant, tool_call, tool_result) consistently — the model's instruction-following depends on the exact delimiters it was trained on.

- **Category**: Language Models
- **Subcategory**: Tokenization
- **Canonical URL**: https://designpattern.fyi/patterns/special-token-design/

---

## Description
**Intent**: LLMs learn conversation structure from the special tokens present during fine-tuning. Misusing or omitting them at inference breaks the model's ability to follow its own format — degrading output quality silently.

**Context**: A chat model fine-tuned with specific role delimiters (e.g. <|im_start|>system, <|im_start|>user) expects those exact tokens at inference. Calling the model with raw text, wrong delimiters, or custom invented tokens means it can't locate the system prompt boundary, user query, or assistant turn — and instruction following degrades.

**Solution**: Study the model's official chat template and reproduce it exactly using tokenizer.apply_chat_template() (HuggingFace) or the documented format. Define explicit typed roles for every message boundary. For tool-calling models, use the documented tool_call and tool_result token types — not ad-hoc JSON embedded in user messages. Never invent special tokens at inference time that the model wasn't trained to recognize.



## Use Cases
Any model served via the chat completion API. Multi-turn conversation systems. Tool-calling and function-calling agents. Any deployment where system prompt injection and role boundary separation affect instruction-following quality.





## Trade-offs


### Advantages

- Correct structure the model was trained to expect — maximizes instruction-following quality

- Role separation makes multi-turn context unambiguous to the model

- Documented chat templates are reproducible and model-version-stable




### Considerations & Drawbacks

- Chat templates are model-specific and change between versions — must be tracked per deployment

- Wrong chat template degrades output with no error signal — it just looks worse

- Custom fine-tuning with different special tokens requires updating all downstream inference code