Google News API (Async)

This endpoint will retrieve news data from an Google news result page and transform it into usable JSON.

Single Query Request:

import requests

url = "https://async.scraperapi.com/structured/google/news"
headers = {
    "Content-Type": "application/json"
}
data = {
    "apiKey": APIKEY,
    "query": QUERY,
    "tld": TLD,
    "uule": UULE,
    "num": NUM,
    "hl": HOSTLANGUAGE,
    "gl": GUESTLANGUAGE,
    "ie": QUERYENCODING,
    "oe": RESULTENCODING,
    "start": START,
    "callback": {
        "type": "webhook",
        "url": "YYYY"
    }
}

response = requests.post(url, json=data, headers=headers)
print(response.text)

Multiple Query Request:

import requests

url = "https://async.scraperapi.com/structured/google/news"
headers = {
    "Content-Type": "application/json"
}
data = {
    "apiKey": APIKEY,
    "queries": [QUERY1, QUERY2, QUERY3],
    "tld": TLD,
    "uule": UULE,
    "num": NUM,
    "hl": HOSTLANGUAGE,
    "gl": GUESTLANGUAGE,
    "ie": QUERYENCODING,
    "oe": RESULTENCODING,
    "start": START,
    "callback": {
        "type": "webhook",
        "url": "YYYY"
    }
}

response = requests.post(url, json=data, headers=headers)
print(response.text)

Multiple Parameters can be used with this method:

REQUIRED

APIKEY

User account’s normal API key.

QUERY

Google Search Query.

OPTIONAL

TLD

Set this value to scrape the respective Google domain. Valid values include tlds for those countries or regions where Google has a search engine:

ae: ()

ca: ()

cn: ()

co.jp: ()

co.uk: ()

com: ()

com.au: ()

com.be: ()

com.br: ()

com.mx: ()

com.tr: ()

de: ()

eg: ()

es: ()

fr: ()

in: ()

it: ()

nl: ()

pl: ()

sa: ()

se: ()

sg: ()

COUNTRY

Valid values are two letter country codes for which we offer Geo Targeting (e.g. “au”, “es”, “it”, etc.).

Where a Google domain needs to be scraped from another country (e.g. scraping Google.com from Canada, both TLD and COUNTRY parameters must be specified.

We also support Google Search parameters for this endpoint.

UULE: Set a region for a search. For example: w+CAIQICINUGFyaXMsIEZyYW5jZQ.

You can find an online UULE generator here: NUM: Number of results HL: Host Language. For example: DE GL: Boosts matches whose country of origin matches the parameter value.

For example: DE IE: Character encoding how the engine interpret the query string. For example: UTF8 OE: Character encoding used for the results. For example: UTF8 START: Set the starting offset in the result list. When start=10 set the first element in the result list will be the 10th search result (meaning it starts with page 2 of results if the "num" is 10)

Sample Response

Single Query Request:

{
	"id": "89b388bf-0cea-49c1-8db6-9e00042c8c3a",
	"status": "running",
	"statusUrl": "http://async.scraperapi.com/structured/google/news/89b388bf-0cea-49c1-8db6-9e00042c8c3a",
	"query": "US economy"
}

Multiple Query Request:

[
	{
		"id": "2955ad23-c812-475c-bc52-572576815d78",
		"status": "running",
		"statusUrl": "http://async.scraperapi.com/structured/google/news/2955ad23-c812-475c-bc52-572576815d78",
		"query": "US Economy"
	},
	{
		"id": "120a7344-7832-4fd0-a64f-ce9cd39726f3",
		"status": "running",
		"statusUrl": "http://async.scraperapi.com/structured/google/news/120a7344-7832-4fd0-a64f-ce9cd39726f3",
		"query": "Crowdfunding"
	}
]

After the job(s) finish, you will find the result under the response key in the response JSON object. The structure is the same as in the corresponding SYNC data endpoint.

Last updated 8 months ago