← ClaudeAtlas

kb-reviewlisted

Read-only review of a retrieval-augmented setup, Amazon Bedrock Knowledge Bases first and hand-built RAG pipelines second: sources, chunking, metadata and tenant filtering, embedding model, vector store, sync, retrieval quality and cost per query. Builds a golden question set, measures recall@5 and MRR against the live knowledge base, and leaves the eval set behind as JSONL. Use when someone asks whether the knowledge base is set up right, why answers miss the right document, or what a RAG pipeline costs to run.
timurgaleev/vibestack · ★ 6 · AI & Automation · score 79
Install: claude install-skill timurgaleev/vibestack
## When to invoke Use when: "knowledge base review", "rag review", "bedrock knowledge base", "chunking strategy", "retrieval quality", "recall at k", "vector store review", "why does the bot not find the document". ## Preamble ```bash eval "$(~/.vibestack/bin/vibe-slug 2>/dev/null)" 2>/dev/null || SLUG="unknown" _LEARN_FILE="${VIBESTACK_HOME:-$HOME/.vibestack}/projects/${SLUG:-unknown}/learnings.jsonl" if [ -f "$_LEARN_FILE" ]; then _LEARN_COUNT=$(wc -l < "$_LEARN_FILE" 2>/dev/null | tr -d ' ') echo "LEARNINGS: $_LEARN_COUNT entries loaded" if [ "$_LEARN_COUNT" -gt 5 ] 2>/dev/null; then ~/.vibestack/bin/vibe-learnings-search --limit 5 2>/dev/null || true fi else echo "LEARNINGS: none yet" fi ``` {{include lib/snippets/session-host.md}} {{include lib/snippets/decision-brief.md}} {{include lib/snippets/working-protocols.md}} {{include lib/snippets/state-protocols.md}} ## User-invocable When the user types `/kb-review`, run this skill. --- ## Step 1: Inventory Find out what the knowledge base is made of before judging any of it. Two places to look: infrastructure code in the repo, and the live account. **Repo first.** Grep for the Terraform resources and for the SDK calls a hand-built pipeline would use: ```bash grep -rnE 'aws_bedrockagent_(knowledge_base|data_source)|aws_opensearchserverless_collection|pgvector|s3_vectors|S3_VECTORS' --include='*.tf' --include='*.py' --include='*.ts' --include='*.yaml' --include='*.yml' . 2>/dev/null | head -40 grep -