Start a fresh conversation. Replace the highlighted placeholders.
Use cdf-auto-labeling as the parent router to plan a minimal annotation pipeline for [domain, modality, intended consumer and required annotations]. Confirm [input/output paths, VLM/LLM endpoint URLs and model names, model-cache path, GPU IDs and reasoning-model max_tokens cap]; ask about missing or ambiguous values. Read scenario-planning and cookbook-authoring references, verify selected stage packages/images exist, and identify the closest shipped cookbook. Return the minimal stage subset, adaptation plan and blockers. Do not customize a cookbook until its shipped example runs clean, or promise an end-to-end run without its services. Keep secrets and absolute home paths out of cookbooks; execute nothing in this planning task.
Required inputs
Confirmed paths, endpoint/model settings, cache, GPUs and token cap
User-approved starter prompt based on a pinned source version; check current skill requirements before running. Task evidence ↗
SKILL.md
name: cdf-auto-labelingdescription: >- Use when a user needs to get started with CDF Auto-Labeling, plan a scenario, run or debug a shipped cookbook, author prompts or cookbooks, migrate a pipeline, configure a stage, or pull the prebuilt stage containers from NGC (nvcr.io). Confirm critical inputs (data path, output path, endpoints) and ask when any are missing. This is a router: read the matching reference instead of inventing a workflow.license: Apache-2.0owner: NVIDIAservice: cosmos-data-factoryversion: 1.1.0reviewed: 2026-09-14author: "NVIDIA <opensource@nvidia.com>"metadata: author: "NVIDIA <opensource@nvidia.com>" tags: [getting-started, onboarding, new-domain, quickstart, new-use-case]
CDF Auto-Labeling
Use this skill when a user wants to kick off CDF Auto-Labeling on their
own data, domain, or use case, or when the request matches a shipped cookbook,
stage, authoring, or migration task. This is a router: sequence the specialized
references instead of duplicating their detail.
Routing (Read First)
Request looks like
Read
New user, clean checkout, first validated run, "how do I get started"
This file, then the matching reference below
Choose annotation targets / stage subset for a domain
Confirm the critical run inputs with the user before doing anything else, and
ask a concise question whenever one is missing or ambiguous - never guess or
silently invent a default. At minimum confirm: input data path, output path,
VLM/LLM endpoint URLs and model names, model cache path, GPU ids, and (for
reasoning-capable models) the max_tokens cap. Restate the confirmed values
back to the user before the first execution.
Verify the environment: repository cloned, make targets available, the
model cache path exists, the VLM/LLM endpoints are reachable, and a GPU is
available. State any missing prerequisite as a blocker instead of assuming it.
If the stage images are not present locally, offer to pull the prebuilt
images from NGC with make pull instead of building them. Use
ngc-containers.
Run a shipped example first to confirm the stack works end to end before
customizing. Pick the closest operator pipeline - video data augmentation,
event-and-person-attribute-search, or event-verification-reasoning - and
run its committed cookbook. Use the matching operator reference.
Plan the target scenario: define modality, domain, intended consumer, and
required annotations, and get a minimal stage subset. Use
scenario-planning.
Adapt the closest shipped cookbook to the new domain rather than authoring
from scratch. Use cookbook-authoring.
Author the domain prompts and question banks. Use
prompt-authoring.
Configure the per-stage settings for the domain (detector classes or SAM3
prompts, endpoints, windowing, max_tokens). Use the relevant stage
reference, starting with
detection-and-tracking.
Adopting an existing external annotation or dataset-generation repository into
CDF instead of starting from a shipped cookbook is a migration task; use
pipeline-migration for that path.
Examples
New user, new domain: "I cloned the repo and have my own warehouse-safety video.
How do I produce auto-labels for my domain?"
Guided path:
Confirm env (model cache, VLM/LLM endpoints, GPU), then prove the stack on a
shipped example before customizing:
make run SCRIPT=workflow-runner:main \
ARGS='--cookbook-file cookbooks/video_data_augmentation/configs/pipeline_video.yaml --container-dry-run'
Plan the domain (scenario-planning) -> subset
detection_and_tracking -> captioning -> visual_qa -> reasoning -> training_export
(add grounding_2d for caption→boxes or referring_expressions for boxes→phrases;
use grounding-2d /
referring-expressions).
Copy the closest cookbook to cookbooks/warehouse_safety/configs/pipeline.yaml
and adapt inputs, detector classes/SAM3 prompts, prompts, and question banks.
Dry-run the new cookbook, then run for real and validate outputs:
make run SCRIPT=workflow-runner:main \
ARGS='--cookbook-file cookbooks/warehouse_safety/configs/pipeline.yaml --container-dry-run'
Guardrails
Do not guess or fabricate the critical inputs enumerated in step 1; if any is
missing or ambiguous, ask the user and confirm before executing.
Do not customize a cookbook before a shipped example runs clean; a broken base
makes domain debugging ambiguous.
Keep the first custom pipeline minimal - only the stages needed for the
requested annotations - and expand later.
Verify that every selected stage's service package and image exist in the
current branch before promising an end-to-end run.
Do not put secrets, tokens, or absolute home paths in committed cookbooks; use
placeholders such as <model-cache> and env vars for endpoint keys.
For reasoning-capable models (for example Gemini 3 Flash), raise max_tokens
on the visual_qa and reasoning LLM substages to avoid the thinking-token
tax; keep the default cap for non-reasoning models.
Do not rely on non-CDF pipelines, commands, or file locations. A first run
must be reproducible through workflow-runner:main inside this repo.