If your use case requires you to exclusively use either desktop or mobile user agents in the headers it sends to the website then you can use the device_type parameter.
Set device_type=desktop to have the API set a desktop (e.g. iOS, Windows, or Linux) user agent. Note: This is the default behavior. Not setting the parameter will have the same effect.
Set device_type=mobile to have the API set a mobile (e.g. iPhone or Android) user agent.
Note: The device type you set will be overridden if you use keep_headers=true and send your own user agent in the requests header.
API REQUEST
import requests
payload = {'api_key': 'APIKEY', 'url':'https://httpbin.org/ip', 'device_type': 'mobile'}
r = requests.get('http://api.scraperapi.com', params=payload)
print(r.text)
# Scrapy users can simply replace the urls in their start_urls and parse function
# ...other scrapy setup code
start_urls = ['http://api.scraperapi.com?api_key=APIKEY&url=' + url + 'device_type=mobile']
def parse(self, response):
# ...your parsing logic here
yield scrapy.Request('http://api.scraperapi.com/?api_key=APIKEY&url=' + url + 'device_type=mobile', self.parse)
PROXY MODE
import requests
proxies = {
"http": "http://scraperapi.device_type=mobile:APIKEY@proxy-server.scraperapi.com:8001"
}
r = requests.get('http://httpbin.org/ip', proxies=proxies, verify=False)
print(r.text)
# Scrapy users can likewise simply pass their API key in headers.
# NB: Scrapy skips SSL verification by default.
# ...other scrapy setup code
start_urls = ['http://httpbin.org/ip']
meta = {
"proxy": "http://scraperapi.device_type=mobile:APIKEY@proxy-server.scraperapi.com:8001"
}
def parse(self, response):
# ...your parsing logic here
yield scrapy.Request(url, callback=self.parse, headers=headers, meta=meta)
SDK Method
# remember to install the library: pip install scraperapi-sdk
from scraperapi_sdk import ScraperAPIClient
client = ScraperAPIClient('APIKEY')
result = client.get(url = 'http://httpbin.org/anything', device_type = 'mobile').text
print(result)
# Scrapy users can simply replace the urls in their start_urls and parse function
# Note for Scrapy, you should not use DOWNLOAD_DELAY and
# RANDOMIZE_DOWNLOAD_DELAY, these will lower your concurrency and are not
# needed with our API
# ...other scrapy setup code
start_urls =[client.scrapyGet(url = 'http://httpbin.org/anything', device_type = 'mobile')]
def parse(self, response):
# ...your parsing logic here
yield scrapy.Request(client.scrapyGet(url = 'http://httpbin.org/anything', device_type = 'mobile'), self.parse)