Rendering & anti-bot

Turn on a real browser for client-rendered pages, drive it with a step scenario, and let asp clear anti-bot protection automatically.

Rendering JavaScript

By default a scrape is a plain HTTP fetch, which is fast and cheap and works for server-rendered pages. For sites that build their content in the browser, set renderJs to true. The result.engine field then reports browser instead of http.

Waiting for content

Rendered pages often need a moment before the data you want exists in the DOM. Control that with:

waitUntil picks the navigation signal to wait on: load, domcontentloaded, or networkidle. Use networkidle for pages that fetch data after first paint.
waitForSelector waits until a specific element appears, which is the most reliable signal that the content you care about has rendered.
waitMs adds a fixed extra pause after load. It is capped at roughly 15 seconds, so prefer a selector when you can.
autoScroll scrolls to the bottom to trigger lazy-loaded images and infinite lists.

Scripted interaction

jsScenario is an ordered array of steps run inside the browser before the page is captured. Each step is an object with one action: { fill, value }, { click }, { waitForNavigation }, { waitMs }, { scrollY }, { eval }, or { select }. The per-step outcomes come back in result.scenarioResults.

scenario.json

{
  "clientKey": "capzy_YOUR_KEY",
  "url": "https://example.com/search",
  "renderJs": true,
  "waitUntil": "networkidle",
  "jsScenario": [
    { "fill": "input[name=q]", "value": "wireless headphones" },
    { "click": "button[type=submit]" },
    { "waitForNavigation": true },
    { "waitForSelector": ".result-card" },
    { "scrollY": 2000 },
    { "waitMs": 500 }
  ]
}

Screenshots

Set screenshot to capture the viewport as a base64 PNG in result.screenshot, or screenshotFullPage to capture the entire scrollable page. Both require renderJs.

Browser persona

Shape how the rendered browser presents itself with os (linux, windows, or macos), browserBrand (chrome or edge), a custom userAgent, and geolocation coordinates. These keep the persona coherent with your chosen egress.

Anti-bot bypass with asp

Set asp to true to enable Anti-Scraping Protection. Capzy auto-detects the protection in front of the target and works through it. Supported vendors include Cloudflare, DataDome, PerimeterX, Akamai, Imperva, and AWS WAF, among others. The outcome is reported in result.asp with detected, servicesUsed, solved, and blocked.

asp implies residential egress and higher cost

Turning on asp forces the request onto the residential pool, overriding proxyPool. It also adds the anti-bot bypass fee on top of residential per-megabyte pricing. Enable it only for targets that actually block you. See Pricing for the exact numbers.

Parameter	Description
`renderJs`	bool, default false. Run a real headless browser instead of a plain HTTP fetch. Required for JS-heavy sites.
`waitForSelector`	string. Wait until this CSS selector appears before capturing the page.
`waitUntil`	string, default "load". Navigation completion signal: "load", "domcontentloaded", or "networkidle".
`waitMs`	int. Extra fixed wait after load, render only, capped around 15 seconds.
`jsScenario`	array. Ordered browser steps (fill, click, waitForNavigation, waitMs, scrollY, eval, select) run before capture.
`autoScroll`	bool. Scroll to the bottom of the page to trigger lazy-loaded content.
`screenshot`	bool, default false. Return a base64 PNG of the viewport in result.screenshot.
`screenshotFullPage`	bool, default false. Capture the full scrollable page rather than just the viewport.
`os`	"linux" \| "windows" \| "macos". OS persona for the rendered browser.
`browserBrand`	"chrome" \| "edge". Browser brand reported to the site.
`userAgent`	string. Override the User-Agent string outright.
`geolocation`	string "lat,long". Coordinates reported to the Geolocation API.
`asp`	bool, default false. Auto-detect and bypass anti-bot protection. Forces residential egress.

With the page rendered and unblocked, turn it into markdown or structured fields.

Making a request

Extraction & formats