context-compression

# Context Compression Context compression is the process of reducing the size of textual context provided to a language model while retaining the information most essential to the task. As conversations grow longer and retrieved documents grow larger, compression becomes critical for staying within token limits and keeping inference costs manageable without sacrificing answer quality. ## Workflow 1. **Measure the Token Budget**: Determine the model's total context window (e.g., 4K, 32K, 128K tokens) and subtract the tokens reserved for the system prompt, instructions, and the model's generation output. The remainder is your available context budget. If the raw context already fits, compression may be unnecessary. 2. **Score Information Density**: Analyze each paragraph, sentence, or chunk of the raw context and assign an information-density score based on how many task-relevant facts it contains per token. Sentences that are purely stylistic, redundant, or off-topic receive low scores. This can be done heuristically (keyword overlap with the query) or via a lightweight classifier. 3. **Select a Compression Strategy**: Choose the most appropriate technique based on the compression ratio needed and the nature of the content: - *Extractive summarization* — select the most important sentences verbatim. - *Abstractive summarization* — rewrite content in fewer words while preserving meaning. - *Key-point extraction* — pull out only named entities, facts, and figures. ...