Links

Making POST/PUT Requests

Some advanced users may want to send POST/PUT requests in order to scrape forms and API endpoints directly. You can do this by sending a POST/PUT request through ScraperAPI. The return value will be stringified. So if you want to use it as JSON, you will need to parse it into a JSON object.
  • API ENDPOINT REQUEST
import requests
payload = {'api_key': 'APIKEY', 'url': 'https://httpbin.org/anything'}
r = requests.post('http://api.scraperapi.com', params=payload, data={'foo': 'bar'})
print(r.text)
  • PROXY MODE
import requests
proxies = {
"http": "http://scraperapi:[email protected]:8001"
}
r = requests.post('http://httpbin.org/anything', proxies=proxies, data={'foo': 'bar'}, verify=False)
print(r.text)
  • SDK Method
# remember to install the library: pip install scraperapi-sdk
from scraper_api import ScraperAPIClient
client = ScraperAPIClient('APIKEY')
postResult = client.post(url = 'http://httpbin.org/anything', body = {'foo': 'bar'}}).text
putResult = client.put(url = 'http://httpbin.org/anything', body = {'foo': 'bar'}}).text
print(postResult)
print(putResult)