← ClaudeAtlas

deepeval-evaluationlisted

Use for any question or task about testing an LLM or RAG app with pytest, including hallucination checks in CI and answers that invent facts missing from the retrieved documents: DeepEval LLMTestCase, built-in metrics such as G-Eval, answer relevancy, faithfulness, contextual recall and precision, hallucination, bias and toxicity, deepeval test run with assert_test, and the Confident AI dashboard.
testland/qa-skills · ★ 0 · AI & Automation · score 72
Install: claude install-skill testland/qa-skills
# deepeval-evaluation ## Overview [de-gh]: https://github.com/confident-ai/deepeval [de-start]: https://deepeval.com/docs/getting-started [de-halluc]: https://deepeval.com/docs/metrics-hallucination [de-multiturn]: https://deepeval.com/docs/evaluation-multiturn-test-cases [de-role]: https://deepeval.com/docs/metrics-role-adherence [de-retention]: https://deepeval.com/docs/metrics-knowledge-retention [de-completeness]: https://deepeval.com/docs/metrics-conversation-completeness Per [de-start][de-start], DeepEval is "an open-source LLM eval package" enabling "evaluation of LLM applications locally through test cases and metrics." The model: each test constructs an `LLMTestCase`, applies one or more `Metric` instances, and either asserts (`assert_test`) or batch-evaluates (`evaluate`). Pytest discovery + reporting works unchanged. ## When to use - The repo already uses pytest; LLM tests should live alongside unit tests. - The user works with RAG and needs Faithfulness / Contextual-* metrics without writing them from scratch. - The team wants a managed dashboard (Confident-AI) for regression tracking + prompt-vs-prompt comparison. - Programmatic test-case authoring (data-driven from a CSV/JSONL) is needed and pytest fixtures fit better than YAML config. ## Step 1 - Install Per [de-gh][de-gh] and [de-start][de-start]: ```bash pip install -U deepeval ``` Optional Confident-AI login (for dashboard): ```bash deepeval login ``` Per [de-start][de-start]: after login