Quick Start

The langchain-scraperapi package adds three ready-to-use LangChain tools backed by ScraperAPI:

Tool Class	Use it to
ScraperAPITool	Grab the HTML/text/markdown of any web page.
ScraperAPIGoogleSearchTool	Get structured Google Search SERP data (JSON or CSV format).
ScraperAPIAmazonSearchTool	Get structured Amazon Product-Search data (JSON or CSV format).

Installation

We’ll be using Python in this guide. Lets install the langchain-scraperapi package using pip:

pip install -U langchain-scraperapi

Setup

If you don't have an account with us yet, head over to scraperapi.com to create one and grab your API key from the Dashboard area. You will need to set it as an environment variable inside the python script:

import os
os.environ["SCRAPERAPI_API_KEY"] = "API_KEY"

ScraperAPITool

Scrape HTML, text, or markdown from any webpage:

import os
os.environ["SCRAPERAPI_API_KEY"] = "API_KEY"
from langchain_scraperapi.tools import ScraperAPITool

tool = ScraperAPITool()

# Get text content
result = tool.invoke({
    "url": "https://quotes.toscrape.com/",
    "output_format": "text",
    "render": True
})
print(result)

Parameters	Details
URL (required)	Target page URL
OUTPUT_FORMAT	"text" | "markdown" (default returns HTML)
COUNTRY_CODE	country_code – e.g. "us", "de"

For the full list of supported parameters, please visit this page.

ScraperAPIGoogleSearchTool

Get Google Search results in a structured JSON or CSV format:

import os
os.environ["SCRAPERAPI_API_KEY"] = "API_KEY"
from langchain_scraperapi.tools import ScraperAPIGoogleSearchTool

google_search = ScraperAPIGoogleSearchTool()

results = google_search.invoke({
    "query": "what is langchain",
    "num": 20,
    "output_format": "json"
})
print(results)

Parameters	Details
QUERY (required)	Search terms
OUTPUT_FORMAT	"json" (default) or "csv"
COUNTRY_CODE	country_code – e.g. "us", "de"

For the full list of supported parameters, please visit this page.

ScraperAPIAmazonSearchTool

Get Amazon Product Search results in a structured JSON or CSV format:

import os
os.environ["SCRAPERAPI_API_KEY"] = "API_KEY"
from langchain_scraperapi.tools import ScraperAPIAmazonSearchTool

amazon_search = ScraperAPIAmazonSearchTool()

products = amazon_search.invoke({
    "query": "noise cancelling headphones",
    "tld": "co.uk",
    "page": 2
})
print(products)

Parameters	Details
QUERY (required)	Product Search terms
OUTPUT_FORMAT	"json" (default) or "csv"
COUNTRY_CODE	country_code – e.g. "us", "de"

For the full list of supported parameters, please visit this page.