LogoLogo
Release NotesDataPipelineFAQ
FAQ
FAQ
  • General
    • How to use the API
    • What is an API credit?
    • Concurrent Requests
    • Increasing scraping speed
    • Getting Failed Requests From The API
    • Unused Requests
  • Plans & Billing
    • Free Plan & 7-Day Free Trial
    • What happens if I run out of credits before the end of my current subscription?
    • Cancelling a Plan
    • Getting a refund
    • Change Credit Card Details
    • Pay-As-You-Go Option
    • Bandwidth Based Pricing
    • Buying Individual IPs
  • JS Rendering
    • Enable JavaScript Rendering
    • Extra JavaScript Rendering Costs
    • Clicking Page Elements
    • Page Elements Still Missing
    • JS Rendering Concurrency
  • Geolocation & Residential IPs
    • Geolocation & Residential IPs
    • City-Level Geotargeting
    • Residential IPs
    • Mobile IPs
  • Anti-bots & CAPTCHAs
    • Getting Distil, Cloudflare, etc. bans.
    • How CAPTCHA solving works
    • Getting a CAPTCHA as successful response
    • Solving embedded CAPTCHAs
  • Low Success Rates
    • Check if it's a general issue
    • Check what type of protection your target domain is using
    • Check if the URL is allowed
    • Check your timeout setting
    • Try geo-targeting
    • Use your own custom headers (Advanced)
    • Try our Async scraper
Powered by GitBook

Quick Links

  • Homepage
  • Dashboard
  • Pricing
  • Contact Sales

Resources

  • Developer Guides
  • Blog
  • Contact Support
  • Learning Hub
On this page

Was this helpful?

Export as PDF
  1. Low Success Rates

Use your own custom headers (Advanced)

PreviousTry geo-targetingNextTry our Async scraper

Last updated 1 year ago

Was this helpful?

Some websites may block requests from known scraper user agents. Using your own custom headers can help disguise your requests as normal web traffic and avoid getting blocked. By setting keep_headers=true and sending the API the headers you want to use, you can customize your requests and potentially increase your success rates.

For more information on how to use custom headers and our other features, please see .

If you're new to custom headers, we don't recommend this approach. Custom headers are typically employed in particular situations, such as making subsequent requests along with a session_number or when you need to send custom headers to obtain specific results from a website. Please note that when you use your own custom headers, our header system will be overridden, which may occasionally lead to lower success rates. Nonetheless, if you need help setting this up let us know, we're always here to help and support you through the process!

our documentation