prompt-repetitionlisted

# Prompt Repetition ## Problem Being Solved LLMs are trained as **Causal Language Models**, where each token attends only to **previous tokens**. This leads to: 1. **Context-Question Problem**: The question is unknown when processing context 2. **Options-First MCQ Problem**: Cannot fully understand the question context when viewing answer choices 3. **Position/Index Problem**: Attention weights weaken for specific position information in long lists **Prompt repetition** enables the second pass to reference the entire first pass, effectively **mimicking some benefits of bidirectional attention**. --- ## When to use this skill - **When using lightweight models**: claude-haiku, gemini-flash, gpt-4o-mini, etc. - **Options-First MCQ**: Multiple choice where answer choices appear before the question - **Context + Question**: Searching for specific information in long contexts - **Index/Position Tasks**: Position-based queries in inventories or lists - **NPC Dialogue**: Maintaining consistency for game AI characters - **Non-Reasoning Tasks**: Tasks that do not use Chain-of-Thought --- ## How It Works ### Limitations of Causal Attention ``` [Context] → [Question] ↓ Cannot reference Question content when processing Context tokens Attention weights for Context are already finalized by the time Question tokens appear ``` ### How Prompt Repetition Solves This ``` [First Pass] [Second Pass] Context → Question → Context' → Question'