Try our Async scraper
If you're experiencing low success rates with synchronous API requests, consider trying our Async scraper. This product is designed to improve success rates by submitting a job of scraping, rather than making requests to our endpoint and waiting for a response.
The Async scraper works on your requested URLs until it manages to get a successful response back (when applicable) and returns the data to you. This method is especially helpful when scraping difficult sites that have protection measures in place and may require more time to successfully scrape.
If a high success rate is more important to you than response time (for example, if you need a set of data periodically), then Async scraping is the recommended way to scrape pages. Give it a try and see the difference it can make in the success rate of your scraping requests.
By following these steps, you can improve your success rate and get the data you need from your API requests. If you continue to experience issues, don't hesitate to reach out to our support team for assistance.