Input and Parameters
Last updated
Was this helpful?
Last updated
Was this helpful?
It is possible to scrape up to 10,000 URL, ASINs or keywords within one project. Depending on the project type, the type of Input can be different. E.g. Walmart category IDs for Category search or Amazon AINS for Amazon product data.
Input options are:
Input field: Copy paste the URLs or search terms into the input field. ‼️ Each term or URL must be in a separate line. ‼️
CSV file: Simply drag and drop the file into the field. The file should not contain a header row or any commas.
Webhook: Add a webhook URL where we poll the entries like URLs or search terms from. The file will be validated on the fly. This gives you a more flexible and dynamic way to update the pages and data you need.
Each term should be in one line without commas.
Search Results
Query string / keywords that a user wants to search for.
Maps
Query string / keywords that a user wants to search for.
Jobs
Query string / keywords that a user wants to search for.
News
Query string / keywords that a user wants to search for.
Shopping
Query string / keywords that a user wants to search for.
Amazon
Product page data
ASIN = Amazon Standard Identification Number. Please not that ASIN's are market specific (TLD). You can usually find the ASINs in the URL of an Amazon product (example: B07FTKQ97Q)
Search page results
Amazon Search query string
Offers
ASIN = Amazon Standard Identification
Walmart
Product page data
Walmart Product ID = You can find the product ID in the URL. Example: 5253396052 (/ip/5253396052)
Category Pages
Walmart category ID = You can find the walmart category ID in the URL. Example: 3944_1089430_37807 (/browse/3944_1089430_37807)
Search page results
Query string / keywords that a user wants to search for.
Ebay
Product page data
Ebay product ID = You can find the 12 digit ID in the URL. Example: 116506192782
(/itm/116506192782?)
Search page results
Query string / keywords that a user wants to search for.
Redfin
Single for Sale listings
URLs of the on sale listings
Single for Rent listings
URLs of the for rent listings
Listing search page
URLs of search page
We have multiple parameters available that will improve the success of a job. Some parameters will increase the price for your job. Be aware that not all parameters might be in your plan! Read more about the credit costs here.
Activate premium residential and mobile IPs: Activate premium proxies. A simple request costs 10 API credits and 25 in combination with javascript rendering.
Activate advanced bypass mechanism: Activate advanced bypass for very difficult domains. A request costs 30 API credits.
Disable follow redirects
Activate javascript rendering: Activate if the targeted URLs require rendering javascript. We will fetch these pages using a headless browser.
Activate to parse results: Activate to receive your results as JSON for certain domains instead of plain HTML. The domains are Google, Amazon and Walmart.
Activate retrying 404 responses: Activate to retry URLs if they responded with 404 error.
Country code: Activate country geo-targeting to use country specific proxies, for example US proxies. This parameter does not increase the cost.
Device type: Default is device_type=desktop. The device type will be overridden if you use keep_headers=true and send your own user agent in the request header.
