senior-prompt-engineer

name: senior-prompt-engineerdescription: >  Prompt engineering and LLM evaluation. Use when optimizing prompts, designing prompt  templates, evaluating LLM outputs, building agentic systems, implementing RAG, creating few-  shot examples, or designing structured-output workflows.license: MIT + Commons Clausemetadata:   version: 1.1.0  author: borghei  category: engineering  domain: prompt-engineering  updated: 2026-06-17  tags: [prompt-optimization, llm-evaluation, agents, prompt-engineering]

Senior Prompt Engineer

Prompt engineering patterns, LLM evaluation frameworks, and agentic system design. Provides static (deterministic) analysis tools to optimize prompts, evaluate RAG retrieval and generation quality, and validate/visualize agent workflows — plus deep reference libraries of prompt patterns, evaluation metrics, and agent architectures.

Core Capabilities

• Prompt optimization — token counting and cost estimation, clarity/structure scoring, ambiguity and redundancy detection, and generation of optimized prompt versions.
• Few-shot & structured output design — extract/manage few-shot examples, design diverse example sets (simple/edge/complex/negative), and enforce reliable JSON/XML schema outputs.
• RAG evaluation — context relevance, answer faithfulness, groundedness (ROUGE-L), and retrieval metrics (Precision@K, MRR, NDCG) over pre-retrieved contexts.
• Agentic system design — validate agent configs, visualize flows (ASCII/Mermaid), estimate token cost per run, and apply ReAct / Plan-Execute / Tool-Use / multi-agent patterns.
• Pattern library — 10 prompt patterns, evaluation frameworks (A/B testing, benchmarks, human eval), and agent architectures with pseudocode.

When to Use

• Optimizing an existing prompt's performance or reducing token costs.
• Designing prompt templates, few-shot examples, or structured-output workflows.
• Evaluating LLM outputs or RAG retrieval/generation quality.
• Building or validating agentic systems and tool-calling workflows.

Clarify First

What shapes this skill's output most is the target model, the job the prompt has to do, and the data you can test it against. Settle the inputs below before producing anything. When one is missing or vague and would change the result, ask about it; when it wouldn't, use a sensible default and say which one you used.

• Target model: token counts and cost estimates depend on it. prompt_optimizer.py --model and agent_orchestrator.py --estimate-cost use different characters-per-token ratios for Claude and GPT models, and a pattern that suits one model family may not suit another.
• Task and output contract: whether the prompt classifies, extracts, generates or drives an agent decides which of the 10 patterns fits and whether a JSON or XML schema has to be enforced. A missing output format is one of the first issues the optimizer flags.
• Evaluation data: without sample inputs, expected outputs or pre-retrieved contexts, the work stays at static clarity and structure scoring. rag_evaluator.py needs contexts and questions before it can report relevance, faithfulness or Precision@K, so find out what exists before promising those numbers.
• Volume and budget: the daily run count drives the monthly cost projection (--runs), and a tight budget shifts the trade-off between more few-shot examples and a shorter prompt.

Ask about the two or three inputs that matter most for this request, not the whole list. If the user wants a result straight away, produce it and state your assumptions at the top.

Tools

References

Load the reference that matches the task — keep this file lean and pull detail on demand:

• references/tools-and-workflows.md — full tool usage with sample outputs, the prompt-optimization / few-shot / structured-output workflows, common-patterns and command quick references, troubleshooting table, success criteria, and complete per-script parameter/output-format reference. Read when running any tool or executing a workflow.
• references/prompt_engineering_patterns.md — 10 prompt patterns (zero/few-shot, CoT, role, structured output, self-consistency, ReAct, tree-of-thoughts, RAG) with example inputs and expected outputs. Read when choosing or applying a prompt technique.
• references/llm_evaluation_frameworks.md — evaluation metrics, text-generation and RAG-specific scoring, human-eval frameworks, A/B testing, benchmark datasets, and pipeline design. Read when measuring quality or comparing prompts.
• references/agentic_system_design.md — agent architectures (ReAct, Plan-and-Execute, Tool Use, multi-agent, memory/state) and design patterns with pseudocode. Read when building agents or tool-calling systems.

Scope & Limitations

This skill covers:

• Static prompt analysis: token counting, clarity scoring, structure detection, and optimization suggestions
• RAG evaluation: context relevance, answer faithfulness, groundedness, and retrieval metrics (Precision@K, ROUGE-L, MRR, NDCG)
• Agent workflow design: configuration validation, ASCII/Mermaid visualization, and token cost estimation
• Few-shot example extraction and management from existing prompts

This skill does NOT cover:

• Live LLM calls or runtime prompt testing --- all analysis is static/deterministic (see senior-ml-engineer for LLM integration)
• Vector database setup or embedding generation --- RAG evaluator scores pre-retrieved contexts only (see senior-data-engineer for pipeline orchestration)
• Fine-tuning, RLHF, or model training workflows (see senior-ml-engineer for model deployment)
• Production monitoring, A/B test execution, or real-time drift detection (see senior-data-scientist for experiment design)

Integration Points