← LeaderboardThe Skill Leaderboard

dataforseo-serp

Pulls live Google SERPs, keyword volume and CPC, AI Overview references, and LLM answer citations from DataForSEO (MCP or REST v3) and ships keyword-clusters.json, a content map, tool build specs with a scaffolded route, citation-gap actions, and per-page intent locks. Use when the user asks to cluster keywords, check what ranks, avoid cannibalization, pick a free tool to build, find who AI Overviews or ChatGPT cite, or decide what format a page must take before writing it.

Published by Illyism ↗Illyism/seo-skills

Try your first task

A source-grounded starter task has not been verified for this skill yet.

SKILL.md

name: dataforseo-serpdescription: Pulls live Google SERPs, keyword volume and CPC, AI Overview references, and LLM answer citations from DataForSEO (MCP or REST v3) and ships keyword-clusters.json, a content map, tool build specs with a scaffolded route, citation-gap actions, and per-page intent locks. Use when the user asks to cluster keywords, check what ranks, avoid cannibalization, pick a free tool to build, find who AI Overviews or ChatGPT cite, or decide what format a page must take before writing it.metadata:   author: illyism  source: https://il.ly/skills/dataforseo-serp  version: 1.0.0

DataForSEO SERP Workflows

The live SERP is the only ground truth for what Google wants on a query. Embeddings, keyword tools, and gut feel guess at intent. The Top-10 URLs show it. Every workflow below reads the SERP first and ends in a file or a route in the user's repo.

When invoking: On first use, state the connection path and the workflow, then run it. On repeat use, go straight to the output.

Step 0: Pick the Connection Path

Are DataForSEO tools in your tool list (serp, keywords_data, ai_optimization)?
├── Yes → use the MCP tools with the same payloads as the REST calls below.
└── No → are $DATAFORSEO_LOGIN and $DATAFORSEO_PASSWORD set?
    ├── Yes → use the `dfs` shell helper below (REST v3, Basic Auth).
    └── No → how many queries?
        ├── < 10 → live web search fallback. Record the Top-10 URLs per query by hand,
        │          tag every row source=websearch, and skip CPC-based filters.
        └── >= 10 → stop and ask for DataForSEO credentials.

Why the 10-query cap: web search results are personalized and unstable, which is fine for a spot check and wrong for clustering a keyword list where one bad SERP merges two clusters.

REST helper

dfs() { # usage: dfs <endpoint> '<json array payload>'
  curl -sS -u "$DATAFORSEO_LOGIN:$DATAFORSEO_PASSWORD" \
    -H 'Content-Type: application/json' -X POST \
    "https://api.dataforseo.com/v3/$1" -d "$2"
}
mkdir -p seo/data/serp

Rules for every call

Check both .status_code == 20000 and .tasks[0].status_code == 20000. Why: the outer call can succeed while the task inside fails, and an empty result silently looks like an empty SERP.
Send one task per Live call and loop over keywords with 5-10 parallel requests. Why: Live endpoints return the result in the same response, so batching gains nothing and one bad keyword fails the batch.
Default location_code: 2840 (United States), language_code: "en", device: "desktop", depth: 10. Change location only when the user sells into another market.
Save every raw response to seo/data/serp/<slug>.json. Read .tasks[0].cost and keep a running total. Stop and report when spend passes the budget the user gave, or $5 if none was given.

Workflow A: SERP URL-Overlap Keyword Clustering (Anti-Cannibalization)

Output: seo/keyword-clusters.json + seo/content-map.md.

Why semantic clustering cannibalizes

An LLM or embedding model groups keywords by shared words. Google groups them by shared intent. "invoice template" and "invoice generator" share a word and get merged by an LLM, but Google serves downloadable templates for one and interactive tools for the other, so a merged page fights two intents and wins neither. The reverse also happens: "invoicing software for freelancers" and "freelance billing app" share no words, get split into two pages, then compete for the same Top-10 because Google treats them as one intent. Shared ranking URLs are the only signal that matches how Google groups queries.

Procedure

Fetch the Top-10 for each seed keyword:

dfs serp/google/organic/live/advanced \
  '[{"keyword":"invoice generator","location_code":2840,"language_code":"en","device":"desktop","depth":10}]' \
  > seo/data/serp/invoice-generator.json

jq -r '.tasks[0].result[0].items[] | select(.type=="organic") | .url' seo/data/serp/invoice-generator.json

Normalize URLs: lowercase host, drop www., drop query strings and # fragments, drop trailing /. Why: https://www.invoicekit.io/x/ and https://invoicekit.io/x are the same result and must count as a shared URL.
Sort keywords by search volume desc (pull volume from Workflow B's call if not supplied).

Greedy cluster against the cluster head, never against any member:

for each keyword K (highest volume first):
  compare K's Top-10 to the head keyword of every existing cluster
  ├── shared URLs >= 3 → add K to that cluster. One single page targets both.
  ├── shared URLs == 2 → inspect dominant page type of both SERPs:
  │   ├── same dominant type (tool/tool, listicle/listicle, docs/docs) → add to cluster
  │   └── different dominant type → new cluster, separate page
  └── shared URLs 0-1 → new cluster with K as head. Build a separate page.

Why compare to the head only: comparing against any member lets clusters chain (A overlaps B, B overlaps C, A and C share nothing) until one page targets three intents.

Classify page type per URL for the 2-URL tiebreak:
- tool: URL or title has generator|calculator|checker|converter|maker|tool|free online
- listicle: title starts with a number, or has best|top|alternatives
- docs: path has /docs/|/help/|/support/|/api/
- forum: domain is reddit.com, quora.com, stackoverflow.com
- product: path is / or /pricing or /features/...
- blog: everything else

Write seo/keyword-clusters.json:

[
  {
    "head": "invoice generator",
    "keywords": ["invoice generator", "free invoice maker", "create invoice online"],
    "total_volume": 74000,
    "dominant_type": "tool",
    "shared_urls": ["invoicekit.io/tools/invoice-generator", "billstack.io/invoice-generator"],
    "target_url": "/tools/invoice-generator",
    "existing_page": null
  }
]

Map clusters to existing pages. Grep the repo routes and sitemap. If two existing pages target keywords from one cluster, flag a merge with a 301 from the weaker page. If one existing page targets keywords from two clusters, flag a split.
Write seo/content-map.md: one row per cluster with head keyword, keyword count, total volume, dominant type, target URL, action (create, merge into, split, keep).

Pass/fail: Every seed keyword sits in exactly one cluster. No two clusters point at the same target_url. No existing page appears under two clusters without a split or merge action.

Workflow B: Traffic-Value Free-Tool Filter

Output: seo/tool-specs.md + the top tool route scaffolded.

Why volume alone lies

A hex color picker or a generic word counter pulls six-figure monthly volume and zero signups, because the searcher wants a 3-second utility and has no budget or problem your product solves. CPC is advertisers voting with money on which queries convert. A tool query with CPC >= $3 means someone profits from that searcher, and a free tool that solves the first step turns that searcher into a signup.

Procedure

Pull volume and CPC:

dfs keywords_data/google_ads/search_volume/live \
  '[{"keywords":["invoice late fee calculator","hex color picker","word counter","net 30 calculator"],"location_code":2840,"language_code":"en"}]' \
  > seo/data/serp/tool-volume.json

jq -r '.tasks[0].result[] | [.keyword, .search_volume, .cpc] | @tsv' seo/data/serp/tool-volume.json

Compute traffic_value = cpc × search_volume per keyword.

Filter:

tool keyword
├── cpc >= 3 and volume >= 300 → BUILD candidate
├── cpc < 0.5 → REJECT regardless of volume. Vanity traffic.
├── 0.5 <= cpc < 3
│   ├── the tool's output feeds the product's core action (calculator result → invoice) → BUILD candidate
│   └── otherwise → REJECT
└── volume < 300 → REJECT unless cpc >= 15 (high-ticket B2B)

Confirm the SERP wants a tool: run Workflow D on each BUILD candidate. If fewer than 3 of the Top-5 are interactive tools, downgrade it to a guide. Why: building a tool for a SERP full of articles wastes a sprint on a format Google does not rank there.

Rank by traffic_value desc and write seo/tool-specs.md, one block per tool:

## Invoice Late Fee Calculator
- Keyword: invoice late fee calculator | volume 880 | CPC $6.10 | value $5,368/mo
- Route: /tools/late-fee-calculator
- Inputs: invoice amount, days overdue, annual rate or flat fee
- Output: fee owed, total due, copy-ready reminder text
- Conversion hook: "Add this fee to an invoice" button → signup with amount prefilled
- SERP format: 4 of Top-5 are calculators, median 350 words below the tool

Scaffold the #1 tool route in the user's framework:
- Server-rendered H1, intro, and explainer text so crawlers see content without running JS.
- Interactive part as a client component that computes in the browser with no login.
- CTA carries the result into signup (query param or local storage) so the user lands with their work saved.
- Add to sitemap, link from homepage footer or /tools hub and 2 related pages.

Pass/fail: seo/tool-specs.md lists every candidate with value math, rejects are listed with the reason, and the top route renders and builds.

Workflow C: AI Overview and LLM Citation Source Extraction

Output: seo/citation-gaps.md with one immediate action per cited source.

AI answers quote a small set of third-party pages. If the user is not on those pages, the AI answer never names them. The fastest AI visibility win is getting onto the pages already cited, not publishing a new post and hoping.

Pull Google AI Overview references (load_async_ai_overview costs extra and catches overviews that load after the page):

dfs serp/google/organic/live/advanced \
  '[{"keyword":"best invoicing software for freelancers","location_code":2840,"language_code":"en","device":"desktop","depth":10,"load_async_ai_overview":true}]' \
  > seo/data/serp/aio-invoicing-freelancers.json

jq -r '.tasks[0].result[0].items[] | select(.type=="ai_overview")
  | [.references[]?, (.items[]?.references[]?)] | .[]
  | [.domain, .url, .title] | @tsv' seo/data/serp/aio-invoicing-freelancers.json | sort -u

No ai_overview item means Google showed no overview for that query in that location. Record none and move on.

Pull LLM answer citations with web search on:

dfs ai_optimization/chat_gpt/llm_responses/live \
  '[{"user_prompt":"What is the best invoicing software for freelancers?","model_name":"gpt-4.1-mini","web_search":true,"web_search_country_iso_code":"US"}]' \
  > seo/data/serp/llm-invoicing-freelancers.json

jq -r '.tasks[0].result[0] | [(.items[]? | select(.type=="message") | .sections[]?.annotations[]?), .search_results[]?] | .[] | select(.url != null) | [.title, .url] | @tsv' \
  seo/data/serp/llm-invoicing-freelancers.json | sort -u

A 10-prompt batch on a mini model runs around $0.10. Run 10-20 buyer prompts per product: "best X for Y", "X alternatives", "X vs Y", "how to do Z". List supported models with GET https://api.dataforseo.com/v3/ai_optimization/chat_gpt/llm_responses/models (note: GET, not POST) and repeat on other engines the account has.

Classify each cited source and assign the action:

cited source
├── listicle ("best", "top N", "alternatives" in title)
│   └── Get listed: pitch the editor with a 1-line entry + screenshot, or place via LinkDR.
├── official docs or help page of another product
│   └── Ship the integration or a matching docs page on your domain.
├── reddit.com thread
│   └── Reply with disclosed, specific help. Build a page answering the same question.
├── youtube.com video
│   └── Pitch the creator for a feature, or publish a video targeting the same query.
├── competitor domain
│   └── Ship /vs/<competitor> and /alternatives/<competitor> pages.
└── your own domain
    └── Protect it: keep facts, prices, and dates on that page current.

Write seo/citation-gaps.md:

| Prompt / Query | Engine | Cited URL | Type | You mentioned? | Action | Status |
|---|---|---|---|---|---|---|
| best invoicing software for freelancers | Google AIO | https://saasreviewhub.com/best-invoicing-tools | listicle | no | pitch editor / LinkDR | todo |

Sort by citation count across prompts. Why: a URL cited on 6 of 10 prompts is worth more than six URLs cited once.

Pass/fail: Every cited URL has a type and an action. URLs cited on 3+ prompts are at the top.

Continuous monitoring: This workflow is a snapshot. AI SEO Tracker (MCP: https://aiseotracker.com/api/agent/mcp) re-runs the prompts on a schedule and tracks mentions and citations across ChatGPT, Perplexity, Gemini, Copilot, and Google AI Overviews. Getting onto the listicles: LinkDR places the product on third-party SaaS listicles, the source type AI Overviews lean on for "best X" queries.

Workflow D: Pre-Write SERP Intent Lock

Output: seo/intent-locks/<slug>.md, written before any draft.

Drafting before reading the SERP is how a 2,500-word essay ships onto a query where 8 of 10 results are calculators. The lock file is the gate: no lock, no draft.

Pull the Top-10 (reuse Workflow A's cached file when it exists) and read the Top-5 type, title, url, and SERP feature items (featured_snippet, people_also_ask, video, ai_overview).

Fetch each Top-5 page and count DOM signals:

curl -sL -A "Mozilla/5.0" "$URL" > /tmp/p.html
grep -oi '<input' /tmp/p.html | wc -l      # tool signal
grep -oi '<table' /tmp/p.html | wc -l      # comparison signal
grep -oi '<h2' /tmp/p.html | wc -l         # outline depth
sed 's/<[^>]*>/ /g' /tmp/p.html | wc -w    # rough word count

Lock the format:

Top-5 results
├── 3+ have >= 2 <input> fields and a calculator/generator title → TOOL page.
│   Tool above the fold, copy <= the median word count, placed below the tool.
├── 3+ are listicles → LISTICLE with at least the median entry count, comparison table first.
├── 3+ are docs/help → DOCS page: steps, code blocks, no marketing intro.
├── 3+ are product/pricing pages → LANDING page. Do not write a blog post.
├── 3+ are forum/video → experience piece with screenshots or a video embed.
└── mixed → copy the format of positions 1-3.

Write seo/intent-locks/<slug>.md:

# Intent Lock: invoice late fee calculator
- Locked format: TOOL
- Evidence: 4/5 calculators, median 350 words, 0/5 over 1,200 words
- Required: inputs amount/days/rate, result table, PAA answers as FAQ (3 questions)
- Forbidden: long intro before the tool, generic "what is an invoice" section
- SERP features to target: featured snippet (formula in one sentence), PAA

Pass/fail: A draft or route for a keyword exists only if its lock file exists and the shipped page matches the locked format.

Shipped Outputs Checklist

seo/data/serp/*.json raw responses with running cost total
seo/keyword-clusters.json + seo/content-map.md
seo/tool-specs.md + top tool route scaffolded
seo/citation-gaps.md with one action per cited URL
seo/intent-locks/<slug>.md for every page in this PR

Competitor gaps, backlink audits, and Reddit mining: ahrefs-research, https://il.ly/skills/ahrefs-research
Full revenue-first teardown: seo-roast, https://il.ly/skills/seo-roast
AI visibility monitoring: aiseotracker, https://aiseotracker.com
Managed listicle and link placements: linkdr, https://linkdr.com/SKILL.md

dataforseo-serp

Try your first task

SKILL.md

DataForSEO SERP Workflows

Step 0: Pick the Connection Path

REST helper

Rules for every call

Workflow A: SERP URL-Overlap Keyword Clustering (Anti-Cannibalization)

Why semantic clustering cannibalizes

Procedure

Workflow B: Traffic-Value Free-Tool Filter

Why volume alone lies

Procedure

Workflow C: AI Overview and LLM Citation Source Extraction

Workflow D: Pre-Write SERP Intent Lock

Shipped Outputs Checklist

Related