Proxy Port Method

To simplify implementation for users with existing proxy pools, we offer a proxy front-end to the API. The proxy takes your requests, forwards them to ScraperAPI and handles proxy rotation, CAPTCHAs and retries automatically. It provides the same features and performance as the API endpoint.

The username for the proxy is scraperapi and the password is your API key.

curl --proxy 'http://scraperapi:[email protected]:8001' \
  -k \
  'https://www.example.com'

import requests

#Replace the value for api_key with your actual API Key
proxies = {
"http": "http://scraperapi:[email protected]:8001"
}

r = requests.get('https://www.example.com', proxies=proxies, verify=False)
print(r.text)

import axios from 'axios';

axios.get('https://www.example.com', {
  method: 'GET',
  proxy: {
    host: 'proxy-server.scraperapi.com',
    port: 8001,
    auth: {
      username: 'scraperapi',
      //Replace the value for password with your actual API Key
      password: 'API_KEY'  
    },
    protocol: 'http'
  }
})
  .then(response => {
    console.log(response)
  })
  .catch(error => {
    console.log(error)
  });

<?php

$ch = curl_init();  curl_setopt($ch, CURLOPT_URL,
"https://www.example.com");  curl_setopt($ch, CURLOPT_PROXY,
//Replace the value for api_key with your actual API Key
"http://scraperapi:[email protected]:8001");  

curl_setopt($ch, CURLOPT_RETURNTRANSFER,TRUE);
curl_setopt($ch, CURLOPT_HEADER,FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST,0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER,0);

$response = curl_exec($ch);  curl_close($ch);  var_dump($response);

require 'httparty'

HTTParty::Basement.default_options.update(verify: false)

response = HTTParty.get('https://www.example.com', {
http_proxyaddr: "proxy-server.scraperapi.com",
http_proxyport: "8001",
http_proxyuser: "scraperapi",
#Replace the value for api_key with your actual API Key
http_proxypass: "API_KEY"
})

results = response.body
puts results

import java.io.*;
import java.net.*;
import javax.net.ssl.*;

public class Main {
    public static void main(String[] args) {
        try {
            SSLContext sc = SSLContext.getInstance("TLS");
            sc.init(null, new TrustManager[]{(X509TrustManager) new X509TrustManager() {
                public java.security.cert.X509Certificate[] getAcceptedIssuers() { return null; }
                public void checkClientTrusted(java.security.cert.X509Certificate[] c, String a) {}
                public void checkServerTrusted(java.security.cert.X509Certificate[] c, String a) {}
            }}, new java.security.SecureRandom());
            HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
            HttpsURLConnection.setDefaultHostnameVerifier((h, s) -> true);

            String apiKey = "API_KEY"; //Replace the value for api_key with your actual API Key
            Authenticator.setDefault(new Authenticator() {
                protected PasswordAuthentication getPasswordAuthentication() {
                    return new PasswordAuthentication("scraperapi", apiKey.toCharArray());
                }
            });

            System.setProperty("jdk.http.auth.tunneling.disabledSchemes", "");
            System.setProperty("https.proxyHost", "proxy-server.scraperapi.com");
            System.setProperty("https.proxyPort", "8001");

            BufferedReader in = new BufferedReader(new InputStreamReader(
                new URL("https://example.com").openConnection().getInputStream()
            ));
            StringBuilder resp = new StringBuilder();
            for (String line; (line = in.readLine()) != null; ) resp.append(line).append("\n");
            in.close();
            System.out.println(resp);
        } catch (Exception e) { e.printStackTrace(); }
    }
}

Note: So that we can properly direct your requests through the API, your code must be configured to not verify SSL certificates.

To enable extra functionality whilst using the API in proxy mode, you can pass parameters to the API by adding them to the username, separated by dots. For example, if you want to enable Javascript Rendering to a request, the username would be scraperapi.render=true. You can add multiple parameters at a time.

curl --proxy 'http://scraperapi.render=true.country_code=us:[email protected]:8001' \
  -k \
  'https://www.example.com'

import requests

#Replace the value for api_key with your actual API Key
proxies = {
"http": "http://scraperapi.render=true.country_code=us:[email protected]:8001"
}

r = requests.get('https://www.example.com', proxies=proxies, verify=False)
print(r.text)

import axios from 'axios';

axios.get('https://www.example.com', {
  method: 'GET',
  proxy: {
    host: 'proxy-server.scraperapi.com',
    port: 8001,
    auth: {
      username: 'scraperapi.render=true.country_code=us',
      //Replace the value for password with your actual API Key
      password: 'API_KEY'  
    },
    protocol: 'http'
  }
})
  .then(response => {
    console.log(response)
  })
  .catch(error => {
    console.log(error)
  });

<?php

$ch = curl_init();  curl_setopt($ch, CURLOPT_URL,
"https://www.example.com");  curl_setopt($ch, CURLOPT_PROXY,
//Replace the value for api_key with your actual API Key
"http://scraperapi.render=true.country_code=us:[email protected]:8001");  

curl_setopt($ch, CURLOPT_RETURNTRANSFER,TRUE);
curl_setopt($ch, CURLOPT_HEADER,FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST,0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER,0);

$response = curl_exec($ch);  curl_close($ch);  var_dump($response);

require 'httparty'

HTTParty::Basement.default_options.update(verify: false)

response = HTTParty.get('https://www.example.com', {
http_proxyaddr: "proxy-server.scraperapi.com",
http_proxyport: "8001",
http_proxyuser: "scraperapi.render=true.country_code=us",
#Replace the value for api_key with your actual API Key
http_proxypass: "API_KEY"
})

results = response.body
puts results

import java.io.*;
import java.net.*;
import javax.net.ssl.*;

public class Main {
    public static void main(String[] args) {
        try {
            SSLContext sc = SSLContext.getInstance("TLS");
            sc.init(null, new TrustManager[]{(X509TrustManager) new X509TrustManager() {
                public java.security.cert.X509Certificate[] getAcceptedIssuers() { return null; }
                public void checkClientTrusted(java.security.cert.X509Certificate[] c, String a) {}
                public void checkServerTrusted(java.security.cert.X509Certificate[] c, String a) {}
            }}, new java.security.SecureRandom());
            HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
            HttpsURLConnection.setDefaultHostnameVerifier((h, s) -> true);

            String apiKey = "API_KEY"; //Replace the value for api_key with your actual API Ke
            Authenticator.setDefault(new Authenticator() {
                protected PasswordAuthentication getPasswordAuthentication() {
                    return new PasswordAuthentication("scraperapi.render=true.country_code=us", apiKey.toCharArray());
                }
            });

            System.setProperty("jdk.http.auth.tunneling.disabledSchemes", "");
            System.setProperty("https.proxyHost", "proxy-server.scraperapi.com");
            System.setProperty("https.proxyPort", "8001");

            BufferedReader in = new BufferedReader(new InputStreamReader(
                new URL("https://example.com").openConnection().getInputStream()
            ));
            StringBuilder resp = new StringBuilder();
            for (String line; (line = in.readLine()) != null; ) resp.append(line).append("\n");
            in.close();
            System.out.println(resp);
        } catch (Exception e) { e.printStackTrace(); }
    }
}

Proxy Mode with SSL Verification

If you would like to send SSL verified requests to our Proxy API, you can manually trust our certificate by following these steps:

Download Our Proxy CA Certificate

Follow this link to download our proxy CA certificate.

Manually trust it on your device

Once you've downloaded the certificate, manually trust it in your scraping tool or library settings. This step may vary depending on the tool or library you're using, but typically involves importing the certificate into your trusted root store or by configuring SSL/TLS settings. Depending on your operating system, follow the instructions below to install the ScraperAPI CA Certificate.

Windows 10/11

Press the Win key + R hotkey and input mmc in Run to open the Microsoft Management Console window.
Click File and select Add/Remove Snap-ins.
In the opened window select Certificates and press the Add > button.
In the Certificates Snap-in window select Computer account > Local Account, and press the Finish button to close the window.
Press the OK button in the Add or Remove Snap-in window.
Back in the Microsoft Management Console window, select Certificates under Console Root and right-click Trusted Root Certification Authorities
From the context menu select All Tasks > Import to open the Certificate Import Wizard window from which you can add the Scraper API certificate.

More details can be found here.

If you encounter any Certificate Revocation List (CRL) Distribution Points (DPs) related errors, please configure your SSL context to ignore certificate revocation checks.

macOS

Open Keychain Access window (Launchpad > Other > Keychain Access).
Select System tab under Keychains, drag and drop the downloaded certificate file (or select File > Import Items... and navigate to the file).
Enter the administrator password to modify the keychain.
Double-click the ScraperAPI CA certificate entry, expand Trust, next to When using this certificate: select Always Trust.
Close the window and enter the administrator password again to update the settings.

Linux

Install the downloaded ScraperAPI proxyca.pem file:

sudo cp proxyca.pem /usr/local/share/ca-certificates/proxyca.pem

Update stored Certificate Authority files:

sudo update-ca-certificates

If you have any questions or need further assistance regarding web scraping or certificate management, don't hesitate to reach out to the support team.

PreviousUsing the API Endpoint NextMaking POST/PUT Requests

Last updated 2 days ago

hashtagProxy Mode with SSL Verification

hashtagWindows 10/11

hashtagmacOS

hashtagLinux

Proxy Mode with SSL Verification

Windows 10/11

macOS

Linux