At ScraperAPI, we ensure that the data you need is handled with precision and delivered in a timely and efficient manner.
Depending on the outcome of each request, the API returns specific status codes. In case of failure, we will retry for up to 70 seconds to maximize the chances of a successful response before returning an error.
For certain supported websites, we offer an autoparse feature that returns parsed data in JSON format, streamlining your data processing.
All content is standardized to UTF-8 encoding, so you receive consistent results no matter the encoding format of the original site.
For more detailed insights, please refer to the linked articles below.
The API will return a specific status code after every request depending on whether the request was successful, failed or some other error occurred. ScraperAPI will retry failed requests for up to 70 seconds to try and get a successful response from the target URL before responding with a 500 error indicating a failed request.
Note: To avoid timing out your request before the API has had a chance to complete all retries, remember to set your timeout to 70 seconds.
In cases where a request fails after 70 seconds of retrying, you will not be charged for the unsuccessful request (you are only charged for successful requests, 200 and 404 status codes).
Errors can occasionally occur, so it's important to ensure your code handles them appropriately. Configuring your system to retry failed requests often leads to success. If you find that a request is persistently failing, double-check your request configuration to ensure it’s correct. If you’re repeatedly encountering anti-bot bans, please create a ticket with our support team and we will try to find a solution to bypass the restrictions.
If you receive a successful 200 status code response from the API but the response contains a CAPTCHA, please contact our support team and they will add it to our CAPTCHA detection database. Once included in our CAPTCHA database the API will treat it as a ban in the future and automatically retry the request.
Below are the possible status codes you will receive:
200
Successful response.
400
Bad request. Please check your request structure.
404
Requested page does not exist.
410
Requested page is no longer available.
500
After retrying for 70 seconds, the API was unable to receive a successful response.
429
You are sending requests too fast, and exceeding your concurrency limit.
403
You have used up all your API credits.
If you have specific project needs, that require you to parse the data you get back from us or transform it into a more readable or structured format, you are in the right place.
The portfolio of output formarts that our product supports is growing, as we aim to ensure we meet your diverse project requirements and the evolving industry standards. The subpages below go into detail about our existing output formats, where they apply, and which ones are best suited for your needs.
👉 Simply parse your data with one parameter - autoparse=true
👉 Get your parsed results in structured JSON or CSV for those specific domains with choosing output_format=json or output_format=csv
Read more about available domains here JSON Response - Autoparse 📜
👉 Get response in LLM friendly structure with the output formats markdown or text for every URL on the web
Read more here LLM Output Formats 💻
For selected domains we offer a parameter that parses the data and returns structured JSON format.
You enable the parsing simply by adding autoparse=true to your request.
Available domains:
Search Result
Product Pages
Product Pages
Products Pages
'For Sale' Listing
News Results
Search Results
Category Pages
Search Results
'For Rent' Listing
Job Results
Offers
Search Results
Listing Search Page
Shopping Results
Product Reviews
Google Maps
API REQUEST
PROXY MODE
In addition to parsing the data, you can choose between two different formats how you want to receive your structured response.
output_format=json
output_format=csv
Both options are available for the listed results above and can be used with the API in combination with autoparse=true parameter or with the Structured Data Collection Method.
Request:
Response:
Request:
Response:
input
name
brand
brand_url
pricing
list_price
shipping_price
availability_status
images
product_category
average_rating
feature_bullets
total_reviews
customization_options
seller_id
seller_name
ships_from
sold_by
B0DGHPQJLP
Apple iPhone 16 Pro 128 GB: 5G Handy mit Kamerasteuerung, 4K 120 fps Dolby Vision und einem großen Sprung bei der Batterielaufzeit. Funktioniert mit AirPods, Titan Natur
Besuche den Apple-Store
1.042,52 €
1.199,00 €
GRATIS
Auf Lager
["https://m.media-amazon.com/images/I/318JqEQUsPL.jpg","https://m.media-amazon.com/images/I/21bPL-xQTrL.jpg","https://m.media-amazon.com/images/I/31CK+Sv8xPL.jpg","https://m.media-amazon.com/images/I/31uzqWpN6ZL.jpg","https://m.media-amazon.com/images/I/51BLlolRXuL.jpg","https://m.media-amazon.com/images/I/31nafjXCXwL.jpg"]
4.4
["BEEINDRUCKENDES TITAN DESIGN – Das iPhone 16 Pro hat ein robustes und leichtes Design aus Titan mit einem größeren 6,3\" Super Retina XDR Display. Es ist extrem widerstandsfähig und hat einen Ceramic Shield der neuesten Generation auf der Vorderseite, der 2x härter ist als jedes andere Smartphone Glas.","ÜBERNIMM DIE KAMERASTEUERUNG – Mit der Kamerasteuerung kannst du einfacher und schneller auf Kameratools wie Zoom oder Tiefenschärfe zugreifen und das perfekte Foto in Rekordzeit aufnehmen.","...]
832
{"Farbe":[{"asin":"B0DGHPQJLP","is_selected":true,"value":"Titan Natur","image":"https://m.media-amazon.com/images/I/11LYpzRb1cL.jpg"},{"asin":"B0DGHH9JY3","is_selected":false,"value":"Titan Schwarz","image":"https://m.media-amazon.com/images/I/11EC0wYqODL.jpg"},{"asin":"B0DGHS5NND","is_selected":false,"value":"Titan Weiß","image":"https://m.media-amazon.com/images/I/011TB187wYL.jpg"},{"asin":"B0DGHZQXTJ","is_selected":false,"value":"Titan Wüstensand","image":"https://m.media-amazon.com/images/I/11lV2YEmZ9L.jpg"}]}
Amazon
Amazon
To properly train LLMs, a lot of high quality unbiased data is needed. There is a lot of public data that is relevant for LLMs, but at times, that data can be too noisy and too large. Luckily, we have a solution. One that gathers large-scale data and cleans it by removing irrelevant or duplicate content. The result - structured format responses, that can be used to train LLMs effectively. Simply add the parameter output_format=text or output_format=markdown to the request structure. Here are some examples:
API REQUEST
ASYNC REQUEST
PROXY MODE - COMING SOON!
Markdown:
Text:
Regardless of any tags in the HTML response body that might specify a different encoding (for example ISO-8859-2), ScraperAPI processes and delivers all content in UTF-8 encoding. This standardization offers several key advantages:
🌐Uniform Data Handling: UTF-8 encoding avoids issues related to special characters and symbols, making data processing smoother and reducing errors.
🔗Compatibility Across Systems: UTF-8 is widely supported across various platforms and programming languages, ensuring compatibility and reducing integration challenges.
🛠️Easier Debugging: Consistent encoding simplifies troubleshooting and debugging, as you can expect uniform data format in all your responses.
⚡Streamlined Development: Developers can work with a single encoding format, reducing the need for additional encoding/decoding steps and simplifying the development process.
You only need to refer to the Content-Type header in the response to verify this:
Content-type: text/html; charset=utf-8
