curl -X POST 'https://api.firecrawl.dev/v1/crawl' \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
  "url": "https://example.com",
  "excludePaths": [
    "example-excludePaths"
  ],
  "includePaths": [
    "example-includePaths"
  ],
  "maxDepth": 10,
  "maxDiscoveryDepth": 0,
  "ignoreSitemap": false,
  "ignoreQueryParameters": false,
  "limit": 10000,
  "allowBackwardLinks": false,
  "allowExternalLinks": false,
  "delay": 0,
  "webhook": {
    "url": "example-url",
    "headers": {},
    "metadata": {},
    "events": [
      "completed"
    ]
  },
  "scrapeOptions": {
    "formats": [
      "markdown"
    ],
    "onlyMainContent": true,
    "includeTags": [
      "example-includeTags"
    ],
    "excludeTags": [
      "example-excludeTags"
    ],
    "maxAge": 0,
    "headers": {},
    "waitFor": 0,
    "mobile": false,
    "skipTlsVerification": false,
    "timeout": 30000,
    "parsePDF": true,
    "jsonOptions": {
      "schema": {},
      "systemPrompt": "example-systemPrompt",
      "prompt": "example-prompt"
    },
    "actions": [
      {
        "type": "wait",
        "milliseconds": 0,
        "selector": "#my-element"
      }
    ],
    "location": {
      "country": "US",
      "languages": [
        "en-US"
      ]
    },
    "removeBase64Images": true,
    "blockAds": true,
    "proxy": "basic",
    "changeTrackingOptions": {
      "modes": [
        "git-diff"
      ],
      "schema": {},
      "prompt": "example-prompt",
      "tag": null
    },
    "storeInCache": true
  }
}'

# POST /crawl (Firecrawl API)
Base URL: https://api.firecrawl.dev/v1
Auth: Authorization: Bearer <FIRECRAWL_API_KEY>
Last verified: 2026-10-20

> Crawl multiple URLs based on options

Start a crawl job. See the [crawl operation](https://docs.firecrawl.dev/api-reference/v1-endpoint/crawl-post).

## Critical Gotchas
* The `formats` field lives under `scrapeOptions.formats`, not at the top level, and defaults to `["markdown"]` when `scrapeOptions` is omitted. Set `scrapeOptions.formats` explicitly (e.g. `["markdown","html"]`) to get HTML in results.
* `POST /crawl` is asynchronous and returns a job `id` immediately. Poll [GET /crawl/{id}](/firecrawl/get-crawl-status) until `status` is `completed` (or `failed`); `data` holds the pages scraped so far and `next` links to the next page of results while the crawl keeps running.

## cURL
curl -X POST 'https://api.firecrawl.dev/v1/crawl' \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://example.com","excludePaths":["example-excludePaths"],"includePaths":["example-includePaths"],"maxDepth":10,"maxDiscoveryDepth":0,"ignoreSitemap":false,"ignoreQueryParameters":false,"limit":10000,"allowBackwardLinks":false,"allowExternalLinks":false,"delay":0,"webhook":{"url":"example-url","headers":{},"metadata":{},"events":["completed"]},"scrapeOptions":{"formats":["markdown"],"onlyMainContent":true,"includeTags":["example-includeTags"],"excludeTags":["example-excludeTags"],"maxAge":0,"headers":{},"waitFor":0,"mobile":false,"skipTlsVerification":false,"timeout":30000,"parsePDF":true,"jsonOptions":{"schema":{},"systemPrompt":"example-systemPrompt","prompt":"example-prompt"},"actions":[{"type":"wait","milliseconds":0,"selector":"#my-element"}],"location":{"country":"US","languages":["en-US"]},"removeBase64Images":true,"blockAds":true,"proxy":"basic","changeTrackingOptions":{"modes":["git-diff"],"schema":{},"prompt":"example-prompt","tag":null},"storeInCache":true}}'

## Parameters
| Name | In | Type | Required | Description |
| :--- | :--- | :--- | :--- | :--- |
| url | body | string | Yes | The base URL to s... |
+12 more params

## Response (200 OK)
{"success":true,"id":"string","url":"string"}

---
Need an API key? Get one at https://firecrawl.link/ligabazi

POST

POST /crawl — Firecrawl API

Firecrawl API

Crawl multiple URLs based on options

Start a crawl job. See the crawl operation.

Base URL: https://api.firecrawl.dev/v1
Auth: Authorization: Bearer <FIRECRAWL_API_KEY>
Last verified: 2026-10-20 · upstream hash matched

Actions Try in browser→

Agents: curl -H "Accept: text/markdown" this URL
→ 232 tokens · Vary: Accept

Critical gotchas

The formats field lives under scrapeOptions.formats, not at the top level, and defaults to ["markdown"] when scrapeOptions is omitted. Set scrapeOptions.formats explicitly (e.g. ["markdown","html"]) to get HTML in results.

POST /crawl is asynchronous and returns a job id immediately. Poll GET /crawl/{id} until status is completed (or failed); data holds the pages scraped so far and next links to the next page of results while the crawl keeps running.

cURL

curl -X POST 'https://api.firecrawl.dev/v1/crawl' \  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \  -H 'Content-Type: application/json' \  -d '{  "url": "https://example.com",  "excludePaths": [    "example-excludePaths"  ],  "includePaths": [    "example-includePaths"  ],  "maxDepth": 10,  "maxDiscoveryDepth": 0,  "ignoreSitemap": false,  "ignoreQueryParameters": false,  "limit": 10000,  "allowBackwardLinks": false,  "allowExternalLinks": false,  "delay": 0,  "webhook": {    "url": "example-url",    "headers": {},    "metadata": {},    "events": [      "completed"    ]  },  "scrapeOptions": {    "formats": [      "markdown"    ],    "onlyMainContent": true,    "includeTags": [      "example-includeTags"    ],    "excludeTags": [      "example-excludeTags"    ],    "maxAge": 0,    "headers": {},    "waitFor": 0,    "mobile": false,    "skipTlsVerification": false,    "timeout": 30000,    "parsePDF": true,    "jsonOptions": {      "schema": {},      "systemPrompt": "example-systemPrompt",      "prompt": "example-prompt"    },    "actions": [      {        "type": "wait",        "milliseconds": 0,        "selector": "#my-element"      }    ],    "location": {      "country": "US",      "languages": [        "en-US"      ]    },    "removeBase64Images": true,    "blockAds": true,    "proxy": "basic",    "changeTrackingOptions": {      "modes": [        "git-diff"      ],      "schema": {},      "prompt": "example-prompt",      "tag": null    },    "storeInCache": true  }}'

Parameters

Name	In	Type	Required	Description
url	body	string	Yes	The base URL to start crawling from
excludePaths	body	string[]	No	URL pathname regex patterns that exclude matching URLs from the crawl. For example, if you set "excludePaths": ["blog/.*"] for the base URL firecrawl.dev, any results matching that pattern will be excluded, such as https://www.firecrawl.dev/blog/firecrawl-launch-week-1-recap.
includePaths	body	string[]	No	URL pathname regex patterns that include matching URLs in the crawl. Only the paths that match the specified patterns will be included in the response. For example, if you set "includePaths": ["blog/.*"] for the base URL firecrawl.dev, only results matching that pattern will be included, such as https://www.firecrawl.dev/blog/firecrawl-launch-week-1-recap.
maxDepth	body	integer	No	Maximum depth to crawl relative to the base URL. Basically, the max number of slashes the pathname of a scraped URL may contain.
maxDiscoveryDepth	body	integer	No	Maximum depth to crawl based on discovery order. The root site and sitemapped pages has a discovery depth of 0. For example, if you set it to 1, and you set ignoreSitemap, you will only crawl the entered URL and all URLs that are linked on that page.
ignoreSitemap	body	boolean	No	Ignore the website sitemap when crawling
ignoreQueryParameters	body	boolean	No	Do not re-scrape the same path with different (or none) query parameters
limit	body	integer	No	Maximum number of pages to crawl. Default limit is 10000.
allowBackwardLinks	body	boolean	No	Allows the crawler to follow internal links to sibling or parent URLs, not just child paths. false: Only crawls deeper (child) URLs. → e.g. /features/feature-1 → /features/feature-1/tips ✅ → Won't follow /pricing or / ❌ true: Crawls any internal links, including siblings and parents. → e.g. /features/feature-1 → /pricing, /, etc. ✅ Use true for broader internal coverage beyond nested paths.
allowExternalLinks	body	boolean	No	Allows the crawler to follow links to external websites.
delay	body	number	No	Delay in seconds between scrapes. This helps respect website rate limits.
webhook	body	object	No	A webhook specification object.
scrapeOptions	body	object	No	Request body field scrapeOptions.

Response 200 OK

{
  "success": true,
  "id": "string",
  "url": "string"
}

Get a free Firecrawl API key → sponsored

Affiliate link - we may earn a commission, at no extra cost to you.