Getting Distil, Cloudflare, etc. bans.
Last updated
Was this helpful?
Last updated
Was this helpful?
Along with constantly fine tuning our proxy and header pools, within the API we’ve built in numerous anti-bot bypasses that enable the API to bypass most challenges thrown by anti-bots. Generally, your success rates will be a small bit lower on sites that make heavy use of anti-bots, however, you should be able to scrape the site reliably at scale with the API.
In the case that the API is completely blocked by a site or you are experiencing a very low success rate (under 70%) then .
This is generally due to the site using either a combination of 2 or more anti-bots in tandem or using a customised version of the anti-bot with higher security settings that stop the general bypass from working. In cases like these, one of our engineers will put in place a custom bypass for you if you .