Rendering Javascript

If you are crawling a page that requires you to render the javascript on the page to scrape the data you need, then we can fetch these pages using a headless browser. To render javascript, simply set render=true and we will use a headless Google Chrome instance to fetch the page. This feature is available on all plans.

• API REQUEST

require 'net/http'
require 'json'
params = {
:api_key => "APIKEY",
:url => "http://httpbin.org/anything",
:keep_headers => true
}
uri = URI('http://api.scraperapi.com/')
uri.query = URI.encode_www_form(params)
req = Net::HTTP::Get.new(uri)
req['Accept'] = 'application/json'
req['X-MyHeader'] = '123'
website_content = Net::HTTP.start(uri.hostname, uri.port) {|http|
http.request(req)
}
print(website_content.body)

• PROXY MODE

require 'httparty'
HTTParty::Basement.default_options.update(verify: false)
response = HTTParty.get('http://httpbin.org/ip', {
  http_proxyaddr: "proxy-server.scraperapi.com",
  http_proxyport: "8001",
  http_proxyuser: "scraperapi.render=true",
  http_proxypass: "APIKEY"
})
results = response.body
puts results

• SDK Method

# remember to install the library: gem install scraperapi
require "scraper_api"
client = ScraperAPI::Client.new("APIKEY")
result = client.get("http://httpbin.org/ip", render: true).raw_body
puts result