stepfun-asr

name: stepfun-asrdisable-model-invocation: truedescription: >-  Transcribes Chinese/English audio with StepFun's stepaudio-3-asr-max via its SSE endpoint (not  /v1/audio/transcriptions) — one call handles long-form audio with no chunking. Use when migrating  from step-asr/stepaudio-2.5-asr, or hitting the misleading "model not supported" error (actually  wrong endpoint). Triggers on 阶跃 ASR, 语音识别. Not for TTS with the sibling model (use stepfun-tts).

StepFun stepaudio-3-asr-max

Transcribe audio with StepFun's stepaudio-3-asr-max (StepAudio 3, released 2026-09-15, verified 2026-09-16; supersedes stepaudio-2.5-asr on the same endpoint). Long audio in one call, no chunking — but only if the request hits the right endpoint with the right body shape. The wrong endpoint returns an error that looks identical to "model doesn't exist", which is the #1 reason this skill exists.

Companion: for TTS with stepaudio-3-tts (the sibling model), use the stepfun-tts skill — they share an API key but live on different endpoints with different body shapes.

Why this skill exists — three traps that cost hours

1. Wrong endpoint, wrong error. stepaudio-3-asr-max does not live on /v1/audio/transcriptions (that endpoint serves the older step-asr family). It lives on /v1/audio/asr/sse — SSE streaming, JSON body, base64 audio. Sending it to the wrong endpoint returns {"error":{"message":"model stepaudio-3-asr-max not supported"}}, which is identical in structure to a genuinely nonexistent model name. People waste hours filing whitelist tickets.

2. Subscription endpoint vs wallet endpoint. Step Plan supports stepaudio-2.5-asr at https://api.stepfun.com/step_plan/v1/audio/asr/sse; the wallet endpoint serves stepaudio-3-asr-max. Choose the endpoint and supported model together. Do not infer that Plan is text-only or that changing only a key changes the billing account.

3. SSE error events are real. Censorship can fire on the ASR side too (rarely). Don't assume only transcript.text.delta and transcript.text.done events arrive — handle type: error events in the stream or you'll silently drop them.

Config and auth

API key resolves in this order (fail-fast, no defaults):

1. $STEPFUN_API_KEY environment variable
2. ${CLAUDE_PLUGIN_DATA}/config.json with {"api_key": "..."} (cross-session persistence)

For an authorized active Pro/Plan subscription, use the Plan route first. Select the wallet route when the Plan attempt fails, the required capability or recognition quality is unavailable there, or the task requires higher concurrency. Record the reason and actual route; this script never retries against the wallet automatically. Read current support at Step Plan audio integration.

Run a Plan call with the dedicated environment variable:

python3 scripts/asr_transcribe.py audio.wav --key-env STEPFUN_PLAN_TOKEN \
  --endpoint https://api.stepfun.com/step_plan/v1/audio/asr/sse \
  --model stepaudio-2.5-asr --json

An explicitly selected STEPFUN_PLAN_TOKEN never falls back to the wallet key or plugin configuration. Existing calls without these options keep their wallet endpoint and stepaudio-3-asr-max default. Billing follows the selected endpoint.

First-time setup:

mkdir -p "${CLAUDE_PLUGIN_DATA}" && cat > "${CLAUDE_PLUGIN_DATA}/config.json" <<EOF
{"api_key": "<paste Step API Key here>"}
EOF

Reuse an existing authorized Step API Key or its configured environment first. If none is available, ask for configuration; do not guess or use a placeholder. Get keys at https://platform.stepfun.com/ → API Keys. Use the subscription endpoint for Plan Credit and the wallet endpoint for pay-as-you-go billing.

Quick start — wallet route without an active Plan

python3 scripts/asr_transcribe.py /path/to/audio.mp3

Output: plain text transcription on stdout.

For machine-readable output with usage / timing:

python3 scripts/asr_transcribe.py /path/to/audio.mp3 --json

For non-Chinese audio:

python3 scripts/asr_transcribe.py /path/to/audio.mp3 --language en

Per-word timestamps need no flag — --json always carries segments:

"segments": [{"text": "Understand ", "start_ms": 228, "end_ms": 1108}, ...]

One entry per word, monotonic. A few words share their predecessor's timestamp (the server flushes in blocks), which is fine for locating a moment but not for forced alignment.

To use an older model:

python3 scripts/asr_transcribe.py /path/to/audio.mp3 --model stepaudio-2.5-asr

The script handles base64 encoding, the nested {audio: {data, input: {transcription, format}}} body, SSE parsing, and the misleading-endpoint pitfall. Prefer it over hand-rolled HTTP calls unless integrating into a larger pipeline.

Decision table

Speaker labels — scripts/asr_file.py

stepaudio-3-asr-max on /v1/audio/asr/sse has no speaker capability at all (14 candidate request fields measured inert). Diarization lives on the async file endpoint:

python3 scripts/asr_file.py https://example.com/talk.mp3
# [   6.61-   8.43] speaker_0: Hello. Hello. Oh,
# [   8.21-  10.11] speaker_1: hello! I didn't know you were there.

Verified end-to-end 2026-09-18 on a two-speaker sample: correct turn boundaries, per-word timestamps inside each utterance, up to 10 speakers per task. Uses stepaudio-2.5-asr — v3 is not served on this endpoint.

The hard constraint: it fetches a URL and nothing else. Base64 is rejected and so is StepFun's own stepfile:// file store, so there is no way to feed it a local file without first putting that file somewhere publicly fetchable. Treat that as the caller's decision. references/known_issues.md has the three dead ends and the retry/redirect behaviour.

Parameters are free — never omit one silently

Sending more request parameters costs nothing: billing is per audio-hour. So the default is send everything useful, and every field we do not send has to carry a written reason in REQUEST_PARAMS at the top of scripts/asr_transcribe.py.

python3 scripts/check_params.py            # diff official field table vs REQUEST_PARAMS
python3 scripts/check_params.py --selftest # calibrate the check before trusting it

Two guards, one per direction:

• Request side — check_params.py fetches the official field table and fails if it lists a field REQUEST_PARAMS does not mention. --selftest calibrates both ways: the real manifest must pass (no false alarms), and a manifest with enable_timestamp removed must be caught (the actual historical gap — per-word timestamps were missing for months because only the response field table was ever read).
• Response side — the parser reports unhandled_response_fields for anything the server sends that it does not consume, because that is what the timestamp gap looked like from this side: start_time/end_time arrived on every delta and were thrown away.

Supported audio formats

The script auto-detects from extension; pass --format to override:

For mp4/m4a/webm/etc., transcode to one of the above first via ffmpeg. Production pipelines often pre-transcode everything to OGG/Opus 16kHz mono to minimize base64 payload size.

Capacity and performance

v3 spot measurements (verified 2026-09-16): 10s clip → 1.1s, 53s real-world clip → 2.6s (~20× RTF). v2.5-era baseline for reference (2026-04-23, same endpoint): 32K context window, ~85-101× RTF on 17.4 min audio, single-call ceiling ≈ 30 min — treat 30 min as the working ceiling for v3 until re-probed, and re-measure before quoting long-audio numbers.

Common error patterns

More edge cases in references/known_issues.md.

Design invariants (do not break)

1. Always pass through SSE — don't try to buffer the response with a non-streaming client. The model emits transcript.text.delta for long audio; transcript.text.done carries the authoritative full text and usage. Reject the SSE format entirely and you'll get nothing.
2. Take final text from transcript.text.done.text — concatenated deltas can drift on edge cases. Deltas are for progressive UI; the done event is the source of truth.
3. Handle error events in the stream — don't treat the SSE stream as if only success events arrive. A blocked-content event mid-stream returns type: error with no done event. transcribe()'s return dict always carries an errors list of the raw error-event payloads (empty if none fired), even when ok is True, so a caller wanting the old strict semantics (any error event ⇒ failure) must check errors itself.
4. Fail-fast on missing API key — never default to a placeholder or empty string. The script does this; preserve the behavior in any wrapper.

When to read references

• references/api_reference.md — exact JSON request body, all fields, all SSE event types, response examples. Read when writing raw HTTP calls instead of using the bundled script.
• references/known_issues.md — repetition hallucination details, endpoint and subscription gotchas, ASR-side censorship handling, pricing opacity. Read when debugging anomalous output or evaluating whether to migrate from step-asr-1.1.

Pricing (verified 2026-11-03, volatile)

The official pricing page lists the StepAudio3 series as free through 2026-11-08 23:59:59 Beijing time; stepaudio-3-asr-max becomes 7.50 元/音频小时 from 2026-11-09 00:00. Do not use that future rate as a current charge or assume the promotion continues after its stated end. Step Plan ASR uses stepaudio-2.5-asr and consumes subscription Credit. Verify current prices and actual account usage before quoting costs; a Studio points display does not by itself establish which funding account was charged. Source: https://platform.stepfun.com/docs/zh/guides/pricing/details.

Next Step

ASR transcripts almost always benefit from one of two downstream steps. After producing the transcript, suggest the natural next skill:

Transcription complete. The output is raw text from the model — common next steps:

Options:
A) transcript-fixer — clean up ASR errors (homophones, segmentation, filler words). Recommended if the recording is a real-world conversation, podcast, or interview rather than read-aloud text
B) meeting-minutes-taker — turn the transcript into structured minutes with decisions, action items, and speaker attribution. Recommended if the recording is a meeting
C) No thanks — the raw transcript is what I needed

Skip the suggestion when the user has already specified the downstream tool, or when the transcription was clearly a one-off lookup (e.g., "what does this 15-second clip say?").