Customize Header Parameter with ScraperAPI in NodeJS

Learn how to scrape websites using header-based parameters with ScraperAPI in NodeJS. Pass api_key, render, and instruction_set via headers to simplify requests.

Along with the “traditional” means of passing parameters, we also support passing parameters as headers. Passing parameters such as api_key, render, ultra_premium and instruction_set is very straightforward.

• API REQUEST

Instead of including the parameters in the URL

import fetch from 'node-fetch';

fetch('https://api.scraperapi.com/?api_key=APIKEY&url=http://httpbin.org/ip&render=true')
.then(response => {
console.log(response)
})
.catch(error => {
console.log(error)
})

you can just pass them as headers

import fetch from 'node-fetch';

const headers = {
  'Content-Type': 'application/json',
  'x-sapi-api_key': '<YOUR_API_KEY>',
  'x-sapi-render': 'true'
};

fetch('https://api.scraperapi.com?url=http://httpbin.org/ip', {
  method: 'GET',
  headers: headers
})
.then(response => {
        console.log(response);
    })
    .catch(error => {
        console.error('Error fetching data:', error);
    });

Please note that the 'x-sapi-' prefix is used on each header to avoid collisions with headers used by target sites. We support all standard parameters available with the API. The instruction_set parameter is specifically supported only through headers.

• PROXY MODE

const axios = require('axios');
const headers = {
  'x-sapi-render': 'true'
};
axios.get('http://httpbin.org/ip', {
  method: 'GET',
  headers: headers,
  proxy: {
    host: 'proxy-server.scraperapi.com',
    port: 8001,
    auth: {
      user: 'scraperapi',
      password: '<YOUR_API_KEY>'  
    },
    protocol: 'http'
  }
})
  .then(response => {
    console.log(response)
  })
  .catch(error => {
    console.log(error)
  });

Note that credentials must still be passed to the proxy in the manner shown above, not as headers.