> For the complete documentation index, see [llms.txt](https://docs.scraperapi.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.scraperapi.com/getting-started/tutorials/api-tutorials/any-url-api.md).

# Scrape Any URL

API Playground is ScraperAPI's request builder for testing and configuring scraping requests. In this tutorial, you will use the API Playground to scrape any URL. By the end, you will have working request code, with proxies and anti-bot challenges handled, so you can integrate web data into your application without building scraping infrastructure.

***

## Step 1: Build the request

Open the [**API Playground**](https://dashboard.scraperapi.com/apiplayground) from the left sidebar.

1. Select scraping method: **API**.
2. In the URL field, enter `https://quotes.toscrape.com/`.
3. Review the Summary panel on the right to understand the cost of the job before you run it.
4. Scroll down to the generated code and click **Try it**.

The Playground runs the request and shows the raw HTML response inline.

{% hint style="info" %}
Use the language dropdown to switch to Python, Node.js, PHP, Ruby, or Java. Click **Copy to clipboard** to run the request in your own terminal instead.
{% endhint %}

## Step 2: Additional options and filters

Use **Additional options & filters** to fine-tune your request. The Playground adds the relevant parameters to the generated code automatically.

* **Premium residential and mobile IPs** adds `premium=true`. Higher success rate on difficult sites, at a higher credit cost.
* **Advanced bypass mechanism** adds `ultra_premium=true`. For the most aggressively protected sites.
* **Javascript rendering** adds `render=true`. Use for pages that load content dynamically.
* **Country code** adds `country_code`. Targets a specific country's proxies so the response reflects what a local user would see.
* **Device type** adds `device_type=mobile` or `device_type=desktop`. Use mobile to receive the mobile version of a page.
* **Max Cost** sets a credit ceiling per request. The request fails rather than exceeding the limit you set.

{% hint style="info" %}
Some parameters increase the cost per request. Check the Summary panel before committing to a configuration at scale.
{% endhint %}

## Step 3: Submit a batch of URLs

For large URL lists, switch the scraping method to **Async** in the Playground. Instead of a single URL field, you get a text area that accepts multiple URLs, one per line. The generated request changes from a synchronous GET to an async batch job submission. Each URL in the batch gets its own job, and the response returns a `statusUrl` per job that you use to check progress.

Paste these URLs to try it:

```
https://quotes.toscrape.com/page/1/
https://quotes.toscrape.com/page/2/
https://quotes.toscrape.com/page/3/
```

## Step 4: Poll for results

Async jobs run in the background. Use the `statusUrl` from the batch response to check progress. When `status` is `finished`, the `response.body` field contains your scraped data.

***

{% hint style="info" %}
Prefer a no-code setup? [Scrape any URL with DataPipeline instead](/getting-started/tutorials/no-code-tutorials/any-url-no-code.md).
{% endhint %}

## More API tutorials

* [Track prices across Amazon and Walmart](/getting-started/tutorials/api-tutorials/track-prices-api.md)
* [Monitor unauthorized sellers and MAP violations](/getting-started/tutorials/api-tutorials/monitor-sellers-api.md)
* [Track daily keyword rankings](/getting-started/tutorials/api-tutorials/keyword-rankings-api.md)
* [Monitor Google Shopping prices](/getting-started/tutorials/api-tutorials/google-shopping-api.md)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.scraperapi.com/getting-started/tutorials/api-tutorials/any-url-api.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.