Input and Parameters
Last updated
Last updated
It is possible to scrape up to 10,000 URL, ASINs or keywords within one project. Depending on the project type, the type of Input can be different. E.g. Walmart category IDs for Category search or Amazon AINS for Amazon product data.
Input options are:
Input field: Copy paste the URLs or search terms into the input field. ‼️ Each term or URL must be in a separate line. ‼️
CSV file: Simply drag and drop the file into the field. The file should not contain a header row or any commas.
Webhook: Add a webhook URL where we poll the entries like URLs or search terms from. The file will be validated on the fly. This gives you a more flexible and dynamic way to update the pages and data you need.
Each term should be in one line without commas.
Amazon
Product Data
ASIN = Amazon Standard Identification Number. Please not that ASIN's are market specific (TLD). You can usually find the ASINs in the URL of an Amazon product (example: B07FTKQ97Q)
Offers
ASIN = Amazon Standard Identification
Product Reviews
ASIN = Amazon Standard Identification
Search Results
Amazon Search query string
Search Results
Query string / keywords that a user wants to search for.
Jobs
Query string / keywords that a user wants to search for.
News
Query string / keywords that a user wants to search for.
Shopping
Query string / keywords that a user wants to search for.
Walmart
Product Data
Walmart Product ID = You can find the product ID in the URL. Example: 5253396052 (/ip/5253396052)
Category Pages
Walmart category ID = You can find the walmart category ID in the URL. Example: 3944_1089430_37807 (/browse/3944_1089430_37807)
Search Results
Query string / keywords that a user wants to search for.
We have multiple parameters available that will improve the success of a job. Some parameters will increase the price for your job. Be aware that not all parameters might be in your plan! Read more about the credit costs here.
Activate premium residential and mobile IPs: Activate premium proxies. A simple request costs 10 API credits and 25 in combination with javascript rendering.
Activate advanced bypass mechanism: Activate advanced bypass for very difficult domains. A request costs 30 API credits.
Disable follow redirects
Activate javascript rendering: Activate if the targeted URLs require rendering javascript. We will fetch these pages using a headless browser.
Activate to parse results: Activate to receive your results as JSON for certain domains instead of plain HTML. The domains are Google, Amazon and Walmart.
Activate retrying 404 responses: Activate to retry URLs if they responded with 404 error.
Country code: Activate country geo-targeting to use country specific proxies, for example US proxies. This parameter does not increase the cost.
Device type: Default is device_type=desktop. The device type will be overridden if you use keep_headers=true and send your own user agent in the request header.
