Along with the “traditional” means of passing parameters, we also support passing parameters as headers. Passing parameters such as api_key, render, ultra_premium and instruction_set is very straightforward.
API REQUEST
Instead of including the parameters in the URL
require 'net/http'
require 'uri'
require 'json'
# Define parameters including headers
params = {
api_key: "<YOUR_API_KEY>",
url: "http://httpbin.org/ip",
render: true
}
# URI for the API endpoint (note: use HTTPS)
uri = URI('https://api.scraperapi.com/')
uri.query = URI.encode_www_form(params)
# Create a GET request
req = Net::HTTP::Get.new(uri)
req['Accept'] = 'application/json'
req['X-MyHeader'] = '123'
# Create an HTTPS connection
http = Net::HTTP.new(uri.hostname, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
# Perform the HTTPS request
website_content = http.request(req)
# Output the response body
puts website_content.body
you can just pass them as headers
require 'net/http'
require 'json'
# Define parameters including the target URL
params = {
url: "https://httpbin.org/ip" # Specify the URL you want to scrape
}
# Set the API endpoint URI (HTTPS)
uri = URI('https://api.scraperapi.com/')
uri.query = URI.encode_www_form(params) # Encode parameters into the query string
# Create a new GET request
req = Net::HTTP::Get.new(uri)
# Set Scraper API headers
req['x-sapi-render'] = 'true' # Enable rendering
req['x-sapi-api_key'] = '<YOUR_API_KEY>' # Replace with your actual Scraper API key
# Create an HTTPS connection
http = Net::HTTP.new(uri.hostname, uri.port)
http.use_ssl = true # Enable SSL/TLS encryption
http.verify_mode = OpenSSL::SSL::VERIFY_PEER # Verify the server's certificate
# Perform the HTTPS request and store the response
website_content = http.request(req)
# Output the response body (the scraped content)
puts website_content.body
Please note that the 'x-sapi-' prefix is used on each header to avoid collisions with headers used by target sites. We support all standard parameters available with the API. The instruction_set parameter is specifically supported only through headers.
PROXY MODE
require 'httparty'
# Define headers with the required parameters
headers = {
"x-sapi-render" => "true"
}
# Set default options for HTTParty, disabling SSL verification
HTTParty::Basement.default_options.update(verify: false)
# Make the HTTP request with HTTParty
response = HTTParty.get('http://httpbin.org/ip', {
http_proxyaddr: "proxy-server.scraperapi.com",
http_proxyport: "8001",
http_proxyuser: "scraperapi",
http_proxypass: "<YOUR_API_KEY>",
headers: headers
})
# Capture the response body
results = response.body
# Output the response body
puts results
Note that credentials must still be passed to the proxy in the manner shown above, not as headers.