Making POST/PUT Requests

Some advanced users may want to send POST/PUT requests in order to scrape forms and API endpoints directly. You can do this by sending a POST/PUT request through ScraperAPI. The return value will be stringified. So if you want to use it as JSON, you will need to parse it into a JSON object.

• API ENDPOINT REQUEST

import requests
payload = {'api_key': 'APIKEY', 'url': 'https://httpbin.org/anything'}

r = requests.post('http://api.scraperapi.com', params=payload, data={'foo': 'bar'})
print(r.text)

• PROXY MODE

import requests
proxies = {
  "http": "http://scraperapi:[e-mail]:8001"
}

r = requests.post('http://httpbin.org/anything', proxies=proxies, data={'foo': 'bar'}, verify=False)
print(r.text)

• SDK Method

# remember to install the library: pip install scraperapi-sdk
  from scraperapi_sdk import ScraperAPIClient
  client = ScraperAPIClient('APIKEY')
  
  postResult = client.post(url = 'http://httpbin.org/anything', body = {'foo': 'bar'}}).text
  putResult = client.put(url = 'http://httpbin.org/anything', body = {'foo': 'bar'}}).text
  
  print(postResult)
  print(putResult)

• ASYNC Method

import requests

body = {'apiKey': 'APIKEY', 'url': 'https://httpbin.org/anything', 'method': 'POST', 'headers': {"content-type": "application/x-www-form-urlencoded"},  'body': 'foo=bar'}

r = requests.post('https://async.scraperapi.com/jobs', headers={"Content-Type": "application/json"},  json=body)
print(r.text)