For the complete documentation index, see llms.txt. This page is also available as Markdown.

ScraperAPI Plugin

The ScraperAPI Plugin combines 22 MCP tools and 19 agent skills that give your AI agent everything it needs to scrape the web with ScraperAPI. It is available on the Claude Plugin Hub and ClawHub.

Installation

Prerequisites:

  • ScraperAPI account.

  • Node.js (for Claude Code installs).

  • Claude Code or OpenClaw installed.

# Install directly from GitHub
npx skills add scraperapi/scraperapi-skills

What's included

  • Onboarding - entry point. Routes the agent to the right product.

  • Product primitives - skills for Standard API, Async Jobs, Crawler, DataPipeline and MCP.

  • SDK guides - Python, Node.JS, Ruby, PHP and Java.

  • Tools - scraperapi-cli and n8n community node.

  • Workflows - scraper builder, lead enrichment, research agent, market research, SEO audit, SERP intelligence and price monitoring.

How it Works

When your agent detects that a task matches the skill’s triggers (scraping, Google/Amazon/Walmart/eBay searches, Redfin queries, crawling, etc.), it loads the skill’s guidance into the system prompt. The skill contains:

  • A list of all available ScraperAPI tools and their parameters.

  • A decision tree for selecting the right tool.

  • Best practices for cost, pagination, localization and result formatting.

  • Guidance on handling large outputs efficiently.

Prompt Example

Real-time News & Market Data

- "I need latest Google news for the current price of Gold."

Response

Claude now factors in the SKILL.md file when making a decision. Instead of manually parsing the HTML or relying on raw search results, it sends a request to ScraperAPI's Google News Structured Data endpoint. That endpoint is specifically designed to return clean, structured JSON data for Google News results

ScraperAPI handles:

  • Bypassing the Bot protection mechanisms.

  • Fetching the Google News results.

  • Structuring the response into machine-readable data.

GoldPriceResponse.md

Last updated