rag-architect

name: rag-architectdescription: >  Design RAG pipelines: chunking, retrieval evaluation, and architecture. Use when building a  RAG system, selecting a chunking strategy, choosing a vector database, optimizing retrieval  quality, or evaluating with RAGAS metrics.license: MIT + Commons Clausemetadata:   version: 1.1.0  author: borghei  category: engineering  domain: ai-ml  tier: POWERFUL  updated: 2026-06-17  tags:     - rag    - retrieval    - chunking    - vector-database    - embeddings

RAG Architect

The agent designs, implements, and optimizes production-grade RAG pipelines, from document chunking through evaluation.

Core Capabilities

• Chunking strategy selection — match corpus characteristics to fixed-size, sentence, paragraph, semantic, recursive, or document-aware chunking with sized parameters.
• Embedding & vector-DB choice — pick an embedding model (local vs API) and vector store (Pinecone, Weaviate, Qdrant, Chroma, pgvector) by scale, latency, and cost.
• Retrieval design — dense, sparse (BM25), or hybrid retrieval with Reciprocal Rank Fusion plus cross-encoder reranking when precision must exceed 0.85.
• Query transformations — HyDE, multi-query, and step-back techniques for style mismatch and ambiguous queries.
• Guardrails — PII detection, hallucination/NLI checks, source attribution, confidence scoring, and injection prevention.
• Evaluation — RAGAS faithfulness/relevance plus IR metrics (Precision@K, Recall@K, MRR, NDCG) with failure analysis.
• Production patterns — caching, streaming, fallbacks, incremental re-indexing, and cost control.

When to Use

• Building a RAG system end to end.
• Selecting a chunking strategy or choosing a vector database.
• Optimizing retrieval quality or adding reranking.
• Evaluating a pipeline with RAGAS or IR metrics.

Clarify First

Before designing the pipeline, settle the inputs below. When one is missing or vague and would change the result, ask about it; when it wouldn't, use a sensible default and say which one you used.

• Corpus characteristics: size, document structure, and domain (drives the chunking-strategy selection and parameters).
• Scale / latency / cost constraints: query volume and budget (selects the embedding model and vector DB).
• Retrieval precision target: the accuracy bar (precision >0.85 forces hybrid retrieval + cross-encoder reranking).
• Query types: ambiguous, multi-hop, or style-mismatched (decides which query transforms: HyDE / multi-query / step-back).

Ask about the two or three inputs that matter most for this request, not the whole list. If the user wants a result straight away, produce it and state your assumptions at the top.

Tools

Python tools live at the skill root (no scripts/ dir). Full flags/output formats: references/tool-cli-reference.md.

References

Load the reference that matches the task — keep this file lean and pull detail on demand:

• references/rag-design-guide.md — the 8-step workflow, every selection matrix (chunking, embedding, vector DB, retrieval, query transforms), context-window optimization, RAGAS targets, guardrails, a worked YAML pipeline example, production patterns, common pitfalls, troubleshooting table, and success criteria. Read when designing or debugging a pipeline.
• references/tool-cli-reference.md — full flag/parameter tables, examples, and output formats for chunking_optimizer.py, retrieval_evaluator.py, and rag_pipeline_designer.py. Read before running the scripts.
• references/chunking_strategies_comparison.md — deep comparison of the five chunking strategies with size distributions, quality metrics, and domain recommendations. Read when choosing a chunking strategy.
• references/embedding_model_benchmark.md — benchmark of OpenAI, open-source, specialized, and domain-specific embedding models. Read when selecting an embedding model.
• references/rag_evaluation_framework.md — full evaluation framework: retrieval/generation/end-to-end dimensions, offline/online/human methodologies, metric implementations. Read when building an evaluation harness.

Scope & Limitations

This skill covers:

• End-to-end RAG pipeline architecture design: chunking, embedding, vector storage, retrieval, reranking, and evaluation.
• Quantitative chunking analysis across four strategy families (fixed-size, sentence, paragraph, semantic).
• Retrieval quality evaluation using standard IR metrics (Precision@K, Recall@K, MRR, NDCG) with a built-in TF-IDF baseline.
• Automated pipeline design with component selection, cost projection, and Mermaid architecture diagrams.

This skill does NOT cover:

• LLM prompt engineering or generation-side optimization -- see engineering/prompt-engineer-toolkit.
• Database schema design for metadata stores alongside vector databases -- see engineering/database-designer.
• Production observability, alerting, and SLO dashboards for deployed pipelines -- see engineering/observability-designer.
• Agent orchestration or multi-step reasoning workflows that sit on top of RAG retrieval -- see engineering/agent-workflow-designer.

Integration Points