algo-nlp-summarization

# Text Summarization ## Overview Text summarization condenses documents while preserving key information. Extractive: selects and concatenates important sentences from the original. Abstractive: generates new text that paraphrases the content. Extractive is simpler and more faithful; abstractive is more fluent but may hallucinate. ## When to Use **Trigger conditions:** - Condensing long documents, reports, or article collections - Building automated summary pipelines for content curation - Comparing extractive vs abstractive approaches for a use case **When NOT to use:** - When full document understanding is needed (summarization loses detail) - For structured data extraction (use NER or information extraction) ## Algorithm ``` IRON LAW: Abstractive Summarization Can HALLUCINATE Abstractive models may generate fluent text containing facts NOT in the source. Always verify key claims in abstractive summaries against the original document. For high-stakes use cases (legal, medical), prefer extractive or use abstractive with factual consistency checking. ``` ### Phase 1: Input Validation Determine: input length, target summary length (ratio or word count), single-doc vs multi-doc, domain. **Gate:** Input text available, target length defined. ### Phase 2: Core Algorithm **Extractive (TextRank/LexRank):** 1. Split document into sentences 2. Build similarity graph (sentence nodes, cosine similarity edges) 3. Run PageRank on sentence graph 4. Select top-k sentences by rank,...