Quickstart

Parameters

scrape (required)	Tells the LLM to scrape a URL from the internet using ScraperAPI.
url (requierd)	URL you wish to scrape.
render (optional)	Defaults to False. Set to True if the page requires JavaScript rendering to display its contents.
country_code (optional)	Activate country geotargeting (e.g. “us”, “es”, “uk”, etc.)
premium (optional)	Set to True to use residential IPs with your scrapes.
ultra_premium (optional)	Activates advanced bypass mechanisms when set to True. Can not be combined with premium.
device_type (optional)	Defaults to desktop. Set to mobile to use mobile user agents with the scrapes.

Prompt Templates

• Please scrape this URL <URL>. If you receive a 500 server error, identify the website's geo-targeting and add the corresponding country_code to overcome geo-restrictions. If errors continues, upgrade the request to use premium proxies by adding premium=true. For persistent failures, activate ultra_premium=true to use enhanced anti-blocking measures.

• Can you scrape URL <URL> to extract <SPECIFIC_DATA>? If the request returns missing/incomplete<SPECIFIC_DATA>, set render=true to enable JS Rendering.

Example Prompt (Claude)

"Please scrape this URL https://www.lowes.com/pd/Kozyard-12-ft-x-16-ft-Gazebo-Dark-Brown-Metal-Square-Screened-Gazebo-with-Steel-Roof/5014900669. If you receive a 500 server error identify the website's geo-targeting and add the corresponding country_code to overcome geo-restrictions. If errors continues, upgrade the request to use premium proxies by adding premium=true. For persistent failures, activate ultra_premium=true to use enhanced anti-blocking measures. Get me the price of the product."

Claude's Response

The LLM first attempted to scrape the product page with a flat request (no extra parameters), but it was unable to find the price information requested in the prompt. It then identified that the price was likely being loaded dynamically via JavaScript and determined that enabling JS Rendering (render=true) would be the next step.

The LLM then retried the request, this time with render=true and successfully retrieved the price information.

The full set of price options, product details, and availability were successfully extraced once JavaScript Rendering was applied.

Price Information:
Current Price: $2,058.00
Additional Pricing Options:
MyLowe's Rewards Credit Card: $1,955.10 (when you choose 5% savings on eligible purchases)
84-month financing: $35/month payments available
Buy Now, Pay Later (Lowe's Pay): $123.60 with 18 monthly payments
Product Details:
Model: Kozyard KZAHG1216BRW
Item #: 5682350
Size: 12-ft x 16-ft Alexander Dark Brown Gazebo with Hardtop Roof
Rating: 3.8/5 stars (15 reviews)
Features: Includes screen, hardtop roof, curtains, and mosquito netting
The gazebo is currently available for pickup (ready by Thu, Aug 21) and delivery (as soon as tomorrow) at the Presque Isle Lowe's location, with 18 units available for both options.

This illustrates why ScraperAPI is essential to the process. Without the smart proxy, user agent rotation, and JavaScript Rendering, the LLM would not have been able to access the webpage undisrupted and capture the price details successfully.