Extraction & formats

Return the whole page as HTML, text, or markdown, or pull out just the fields you name with CSS and XPath selectors.

Output format

The format parameter controls how result.content is shaped:

raw (the default) returns the full HTML exactly as fetched.
text returns the visible text with markup stripped.
markdown returns the page converted to markdown, which is compact and well suited to feeding a language model.

Extraction rules

Add an extractionRules object to pull named fields out of the page instead of (or alongside) the full content. Each key becomes a field in the output; each value is a selector. A rule can be written two ways:

A plain string is treated as a CSS selector, and the field is the matched element's text content, for example "title": "h1".
An object for finer control: { css | xpath, attr?, all? }. Use css or xpath to locate the element, attr to read an attribute rather than text, and all: true to return every match as a list.

extract-request.json

{
  "clientKey": "capzy_YOUR_KEY",
  "url": "https://example.com/product/42",
  "renderJs": true,
  "extractionRules": {
    "title": "h1.product-title",
    "price": { "css": ".price", "attr": "data-amount" },
    "sku": { "xpath": "//span[@id='sku']" },
    "images": { "css": "img.gallery", "attr": "src", "all": true },
    "missing": ".does-not-exist"
  }
}

The extracted fields come back under result.extracted, which has a mode of rules and a data object keyed by your rule names.

extract-result.json

{
  "errorId": 0,
  "status": "ready",
  "result": {
    "format": "raw",
    "engine": "browser",
    "extracted": {
      "mode": "rules",
      "data": {
        "title": "Wireless Headphones",
        "price": "129.00",
        "sku": "WH-42",
        "images": [
          "https://cdn.example.com/a.jpg",
          "https://cdn.example.com/b.jpg"
        ],
        "missing": null
      }
    },
    "elapsedMs": 1830
  },
  "cost": 0.0112
}

A failed rule yields null, never a failed scrape

If a selector matches nothing, that field is set to null and the rest of the extraction proceeds. One bad rule never fails the whole request, so you can add speculative fields safely. In the example above, missing came back null while every other field resolved.

Lists with all

By default a rule returns the first match. Set all: true on the rule object to return every matching element as an array, as shown by images above. Combine it with attr to collect an attribute across all matches, for example every image src in a gallery.

Pair extraction with rendering

Extraction runs against whatever the page looks like at capture time. For client-rendered content, set renderJs and a wait condition so the elements your rules target actually exist before extraction runs.

Extraction adds a small flat fee per request. See Pricing for the exact cost.

Rendering & anti-bot

Pricing