evaluate-rag

Guides evaluation of RAG pipeline retrieval and generation quality. Use when evaluating a retrieval-augmented generation system, measuring retrieval quality, assessing generation faithfulness or relevance, generating synthetic QA pairs for retrieval testing, or optimizing chunking strategies.

Try your first task

Scope an initial RAG evaluation

Start a fresh conversation.

Use the evaluate-rag skill to review my completed RAG error analysis at <path-to-error-analysis> and representative full traces at <path-to-rag-traces>. Determine whether the observed failures primarily involve retrieval, generation, or both, then outline a small first evaluation plan that measures first-pass retrieval with Recall@k and evaluates generation faithfulness and relevance separately. Replace both angle-bracketed values with local paths.

Required inputs

A completed error analysis of RAG pipeline traces
Representative full RAG traces showing retrieved content and generated answers
Local paths replacing <path-to-error-analysis> and <path-to-rag-traces>

Suggested template by Skill Leaderboard, grounded in the linked source. Task evidence ↗

Inside SKILL.md

177 lines

Section content has not been verified for this skill yet.

Full source remains withheld under the current source-display policy. Read the author’s source ↗