generate-synthetic-data

Create diverse synthetic test inputs for LLM pipeline evaluation using dimension-based tuple generation. Use when bootstrapping an eval dataset, when real user data is sparse, or when stress-testing specific failure hypotheses. Do NOT use when you already have 100+ representative real traces (use stratified sampling instead), or when the task is collecting production logs.

Try your first task

Draft synthetic evaluation scenarios

Start a fresh conversation.

Use the generate-synthetic-data skill for <application-description>. Based on <known-failure-prone-areas> and any supplied feedback or traces, propose exactly three failure-focused dimensions and draft 20 diverse tuples for my review. Do not generate final natural-language queries or run the pipeline yet. Replace the angle-bracketed placeholders with my application and known-risk details.

Required inputs

A description of the LLM application
Known failure-prone areas, user feedback, or representative traces
A person with domain knowledge who can review whether the tuples are realistic

Suggested template by Skill Leaderboard, grounded in the linked source. Task evidence ↗

Inside SKILL.md

131 lines

Section content has not been verified for this skill yet.

Full source remains withheld under the current source-display policy. Read the author’s source ↗