baoyu-url-to-markdown

name: baoyu-url-to-markdowndescription: Fetch any URL and convert to markdown using baoyu-fetch CLI (Chrome CDP with site-specific adapters). Built-in adapters for X/Twitter, YouTube transcripts, Hacker News threads, and generic pages via Defuddle. Handles login/CAPTCHA via interaction wait modes. Use when user wants to save a webpage as markdown.version: 1.61.0metadata:   openclaw:     homepage: https://github.com/JimLiu/baoyu-skills#baoyu-url-to-markdown    requires:       anyBins:         - bun

URL to Markdown

User Input Tools

When this skill prompts the user, follow this tool-selection rule (priority order):

1. Prefer built-in user-input tools exposed by the current agent runtime — e.g., AskUserQuestion, request_user_input, clarify, ask_user, or any equivalent.
2. Fallback: if no such tool exists, emit a numbered plain-text message and ask the user to reply with the chosen number/answer for each question.
3. Batching: if the tool supports multiple questions per call, combine all applicable questions into a single call; if only single-question, ask them one at a time in priority order.

Concrete AskUserQuestion references below are examples — substitute the local equivalent in other runtimes.

CLI Setup

The CLI source is vendored in {baseDir}/scripts/lib; scripts/package.json installs only third-party runtime dependencies.

1. {baseDir} = this SKILL.md's directory
2. ${BUN} = bun if installed; otherwise suggest installing Bun
3. If {baseDir}/scripts/node_modules is missing: ${BUN} install --cwd {baseDir}/scripts
4. ${READER} = {baseDir}/scripts/baoyu-fetch (substitute everywhere below)

Preferences (EXTEND.md)

First one found wins:

Found → read and apply. Not found → BLOCKING first-time setup: ask the questions in references/config/first-time-setup.md (Media, Output, Save — all in one call) before any conversion, write EXTEND.md, confirm "Preferences saved to [path]", then continue. Never create EXTEND.md with silent defaults.

Value priority: CLI arguments → EXTEND.md → skill defaults.

Usage

${READER} <url>                                          # headless, markdown to stdout
${READER} <url> --output article.md --download-media     # save with local media
${READER} <url> --wait-for interaction --output a.md     # login/CAPTCHA: auto-detect and continue
${READER} <url> --wait-for force --output a.md           # manual: Enter to continue
${READER} <url> --format json --output a.json
${READER} <url> --adapter youtube --output transcript.md

Options

Quality Gate

Headless capture is provisional: some sites silently return low-quality content with exit code 0. Inspect the saved markdown after every headless run, using references/quality-gate.md for the checklist, recovery steps, and capture modes.

Output Path

baoyu-fetch does not generate paths; the agent builds one and passes it to --output:

1. Base dir: EXTEND.md default_output_dir or ./url-to-markdown/
2. Domain from URL (e.g., example.com)
3. Slug from URL path or page title (kebab-case, 2-6 words)
4. Path: {base_dir}/{domain}/{slug}/{slug}.md — one directory per URL so media stays isolated
5. On conflict: {slug}-YYYYMMDD-HHMMSS/{slug}-YYYYMMDD-HHMMSS.md

Downloaded media goes into subdirectories next to the markdown file.

Adapters & Media

references/adapters.md: adapter catalog (X, YouTube, Hacker News, generic), media download flow (ask / always / never), JSON output schema. Read it for adapter-specific questions or media prompts.

Environment Variables

Troubleshooting: Chrome not found → --browser-path. Timeout → raise --timeout. Login/CAPTCHA → --wait-for interaction. Debug → --debug-dir to inspect captured HTML and network logs.

Extension Support

Custom configurations via EXTEND.md — see Preferences above.