Use the Langchain ScraperAPI package

You are building your own LLM? And want to use Langchain? Great, then you should use the Langchain ScraperAPI package to access all data without getting blocked.

Github Repo Langchain ScraperAPI

What you can do?

Tool class	Use it to
ScraperAPITool	Grab the HTML/text/markdown of any web page
ScraperAPIGoogleSearchTool	Get structured Google Search SERP data
ScraperAPIAmazonSearchTool	Get structured Amazon product-search data

Installation

pip install -U langchain-scraperapi

Setup

Create an account at https://www.scraperapi.com/ and get an API key, then set it as an environment variable:

import os
os.environ["SCRAPERAPI_API_KEY"] = "your-api-key"

Learn more how to integrate this in the official Langchain ScraperAPI Github Repo or read our guide how to integrate it:

Integration Plan