tao-artifacts
Four typed artifacts flow through every TAO job. Their schemas live here and
nowhere else — producers (model/data skills) and consumers (platform skills)
both validate against this skill's references/.
Quick Start — validate an artifact
python - <<'PY'
import json, yaml, jsonschema, pathlib
ref = pathlib.Path("${TAO_SKILL_BANK_PATH:?}/skills/core/tao-artifacts/references")
schema = json.loads((ref / "spec_bundle.schema.json").read_text())
bundle = yaml.safe_load(open("/path/to/bundle.yaml")) # or a dict built in-context
jsonschema.validate(bundle, schema) # raises on violation
print("bundle OK")
PY
Validate the bundle before the verify-before-launch gate; validate a
job-record only when debugging (the writer script already enforces the schema).
The two rules the schemas enforce structurally
- Nested, not dotted. A
spec is a nested dict mirroring the container's
config shape — {"train": {"num_epochs": 12}}. Any key containing . at
any depth is rejected ({"train.num_epochs": 12} is the #1 authoring
mistake). Note the distinction: declared_inputs[].spec_key and
gpu_spec_key are dotted/indexed pointers into the spec
(dataset.train_data_sources[0].image_dir) — dots are correct there.
- Mode discrimination.
mode: config requires spec + config_format
and a command containing {config_path}, and forbids args.
mode: args requires args and forbids spec. There is no other mode.
Optional action lifecycle
Use execution when an action needs more than its primary command. This is the
shared model-to-platform seam; do not add a model-specific Docker, Kubernetes,
or SLURM renderer merely to carry runtime environment, attestations,
post-processing, or helper dependencies.
- The producing model/data skill owns
environment, ordered pre_commands,
ordered post_commands, distributed-launch intent, and completion evidence.
- The platform owns container mounts, scheduler/container syntax, task/rank
binding, timeouts, log paths, and preservation of the real child exit code.
environment is non-secret. Credential values continue to use the selected
platform's secret/sidecar contract and never enter a spec-bundle.- Commands, environment values, and string values in
spec may use
{config_path}, {job_id}, and {results_dir}. The platform binds them
only after the job record has been opened; the job record's results_dir is
authoritative over any pre-review display path. Persist hashes of both the
producer bundle and the bound runtime config.
supporting_files names checked-in orchestration helpers relative to the
producing skill root. The platform stages the closed set, verifies every
declared SHA256, and rejects traversal, undeclared siblings, or overwrite of
a different bundle. Supporting files orchestrate an action; they must never
shadow or patch code inside the selected image.- A
torchrun declaration expresses process topology, not SLURM/Kubernetes
syntax. Each platform maps it to its native distributed launcher.
Fixed status vocabulary
Every job state anywhere in the pipeline is exactly one of:
PENDING · RUNNING · COMPLETE · ERROR · CANCELED · UNKNOWN
Platform-native sub-states (ImagePullBackOff, PENDING-because-resources,
Insufficient-GPU, slurm COMPLETING…) are never new states — they ride in
the transition's message field. Terminal = COMPLETE | ERROR | CANCELED.
This is what lets the in-turn poll loop and the detached poller share one code
path across docker/slurm/kubernetes/brev.
Ordering invariants (enforced at the seam, stated here)
- The verify-before-launch gate runs on the spec-bundle, before any job id
exists.
tao_job_record.py open writes PENDING + the resolved results_dir
first and returns the id — the only handle a launch can use. A submit
that skipped the gate has no id, so it cannot launch.transitions is append-only; .tao/ lives outside every synced results
tree.