seoder

Browser automation APIs and hosted browsers

Eighteen ways to drive a real browser, from the protocol underneath to hosted fleets you rent by the minute. Memory is the recurring constraint, so each row names its cost.

  • Playwright

    Open source · Apache-2.0 · TypeScript

    A browser automation library from Microsoft driving Chromium, Firefox and WebKit through one API, with bindings for Node, Python, Java and .NET.

    Suits
    Auto-waiting removes most of the sleep calls that make automation flaky, and the trace viewer records a timeline you can step through after a failure instead of guessing at it.
    Does not suit
    The three engines are not equal in practice, and a script that passes on Chromium can behave differently on WebKit. Each language binding also ships its own browser downloads, so a container image with browsers installed starts around a gigabyte.
  • Puppeteer

    Open source · Apache-2.0 · TypeScript

    A Node library from the Chrome team that drives Chrome and Firefox over the DevTools Protocol.

    Suits
    A thin, predictable layer over the protocol, which makes it easy to drop to raw CDP calls for anything the API does not wrap. The largest body of public examples of any browser library.
    Does not suit
    Node only, and the API leaves timing to you, so scripts accumulate waits and retries that Playwright would have handled. Cross-browser support arrived late and remains narrower than the alternatives.
  • Selenium WebDriver

    Open source · Apache-2.0 · Java

    The long-standing browser automation project, implementing the W3C WebDriver standard across every major browser and many languages.

    Suits
    It is a standard rather than one vendor's library, so the same test can run against browsers and grid providers that have never heard of your code. Two decades of ecosystem, including Grid for distributing runs.
    Does not suit
    The WebDriver protocol works through a separate driver process, which adds latency and puts fine-grained control such as request interception out of reach unless you use CDP alongside it. Scripts need explicit waits and go flaky without them.
  • Chrome DevTools Protocol

    Free

    The WebSocket protocol Chrome exposes for instrumentation, and the layer that Puppeteer and most Chrome tooling is built on.

    Suits
    Complete access to what the browser knows: the network log, the accessibility tree, coverage, heap snapshots and CPU profiles, none of which every wrapper surfaces.
    Does not suit
    It is a protocol, not a library, so you own connection management, target lifecycle and every race between a command and a navigation. Domains change between Chrome versions with no compatibility promise, and a wrong domain order leaves a tab wedged rather than erroring.
  • Lightpanda

    Open source · AGPL-3.0 · Zig

    A headless browser engine written from scratch in Zig, with no rendering at all, built to run JavaScript and answer CDP for crawlers and agents.

    Suits
    Opening a page costs a fraction of a Chrome tab in memory and startup time, because nothing lays out or paints. It speaks the DevTools Protocol, so an existing Puppeteer or Playwright script can point at it and find out in an afternoon whether it works.
    Does not suit
    Web platform coverage is partial and the project publishes daily Web Platform Tests results rather than claiming parity, so a page that reaches for an unimplemented API fails in a way Chrome never would. No rendering also means no screenshots and nothing that depends on computed layout. AGPL-3.0, and Windows needs WSL2.
  • Browserless

    Paid

    A hosted Chrome fleet you connect existing Puppeteer or Playwright code to over a WebSocket, also runnable from a container image.

    Suits
    An existing script moves over by changing one connection string, and the service handles the browser lifecycle problems that kill long-running fleets, including zombie processes and leaked sessions.
    Does not suit
    The code is dual licensed under the SSPL or a commercial license, which is not an OSI open source license and will not pass some legal reviews for self-hosting. Sessions are also billed by duration, so a script that idles waiting on a slow page is billed for the waiting.
  • Browserbase

    Paid

    A hosted browser platform aimed at AI agents, with session recording, live view and stealth options.

    Suits
    Session replay and a live view of a running browser make an agent's behavior observable, which is the hardest part of debugging a run that took forty steps and ended somewhere wrong.
    Does not suit
    Built around the agent use case, so long-running crawls of millions of pages are not the shape it is priced or tuned for. Per-session billing rewards short tasks and punishes a browser held open for an hour.
  • Steel

    Free tier, then paid · Apache-2.0 · TypeScript

    An open source browser API for agents, available as a hosted service or self-hosted from the same codebase.

    Suits
    Self-hosting and the managed service run the same code, so a prototype on the hosted tier can move in-house later without a rewrite. Session state, proxies and captcha handling sit behind one interface.
    Does not suit
    A young project with a smaller community than the established libraries, so an unusual failure is more likely to need reading the source than searching for it. Self-hosting hands you back the browser fleet operations the service existed to absorb.
  • Hyperbrowser

    Free tier, then paid

    A hosted browser infrastructure service for agents and scraping, with session management, proxies and scaling built in.

    Suits
    Scales concurrent browser sessions without you sizing any machines, which removes the memory ceiling that ends most self-hosted fleets.
    Does not suit
    Newer than the alternatives and thinner on public documentation of edge cases, so evaluation means running your own load rather than reading about someone else's. Everything runs on their infrastructure, which is a data residency question worth asking early.
  • Kernel

    Free tier, then paid

    Hosted Chrome for agents, reached over CDP, with saved profiles, a live view and a replay of the session, billed by memory and time rather than per session.

    Suits
    The replay is at browser level rather than log level, so a run that ended somewhere wrong can be watched back instead of reconstructed. Profiles carry a signed-in state between sessions, and idle time is not billed, which suits an agent that spends most of a session waiting on a model.
    Does not suit
    Billing in GB-seconds is honest about what a browser costs and much harder to forecast from a URL list than a per-request price. Concurrency is a plan limit rather than something you scale on demand, and the free tier's ceiling is low enough that a load test means paying for one.
  • Cloudflare Browser Run

    Free tier, then paid

    Headless Chrome on Cloudflare's network, driven with Puppeteer, Playwright or CDP, from inside a Worker through a binding or from anywhere else over a CDP endpoint. Cloudflare renamed it from Browser Rendering in 2026.

    Suits
    If the crawl already runs on Workers, the browser is a binding rather than a second vendor: no extra account, no egress between the two, one bill. The free plan includes browser time, so a small job can run without a card.
    Does not suit
    The limits are the product's shape rather than a tier detail. A browser closes after sixty seconds of inactivity and can be held open for at most ten minutes, so nothing long-lived fits, and the free plan allows three concurrent browsers and ten minutes of browser time a day. Pricing and quotas follow Workers plans, not this product.
  • Anchor Browser

    Paid

    A hosted browser built for AI agents, with authenticated session storage and a live view of the running browser.

    Suits
    Storing and reusing authenticated sessions is a first-class feature rather than a cookie jar you maintain, which matters for agents working inside logged-in applications.
    Does not suit
    Holding authenticated sessions on a vendor's infrastructure is a real security decision, not a feature checkbox, and some environments will refuse it outright. Aimed at agent workflows rather than bulk crawling.
  • spider.cloud Browser API

    Free tier, then paid

    A cloud Chrome that Puppeteer and Playwright reach over CDP, on the same balance as the crawl API and proxy pools.

    Suits
    Metered pricing: $1 per GB transferred and $0.0001 per minute a session is open, with no monthly minimum.
    Does not suit
    Your model provider bills the model calls behind act(), extract() and agent() separately.
  • BrowserStack Automate

    Paid

    A hosted grid of real browsers and real devices for running Selenium, Playwright and Cypress suites.

    Suits
    Real devices and old browser versions you cannot install locally, with video, console logs and network logs captured per session.
    Does not suit
    This is test infrastructure, and its pricing assumes test-shaped usage: bounded parallel sessions, short runs. Pointing it at a crawl is both expensive and outside what the terms are written for.
  • LambdaTest

    Free tier, then paid

    A cross-browser testing cloud with a large browser and OS matrix, plus parallel test execution.

    Suits
    A wide version matrix at a lower price point than the incumbents, with a free tier big enough to evaluate on a real suite.
    Does not suit
    Same category constraint: it is built for test runs, so session startup time dominates anything short, and parallelism is a plan limit rather than something you scale on demand.
  • Sauce Labs

    Paid

    An enterprise testing cloud for browsers and mobile devices, with detailed analytics across test runs.

    Suits
    Test analytics across many runs surface flakiness patterns that a single run cannot show, which is the difference between fixing a test and fixing a suite.
    Does not suit
    Enterprise pricing and enterprise onboarding, and like the rest of this group it is built for test traffic. It is the wrong tool and the wrong contract for data collection.
  • chromedp

    Open source · MIT · Go

    A Go library that drives Chrome over the DevTools Protocol with no external dependencies.

    Suits
    Idiomatic Go with context-based cancellation and timeouts, compiled into a single binary, which suits crawlers deployed as one static artifact.
    Does not suit
    Chrome only, and the API is closer to the protocol than to a convenience layer, so you write more code for the same result. A smaller community means fewer worked examples when a page defeats you.
  • rod

    Open source · MIT · Go

    A Go browser automation library over the DevTools Protocol, with chainable helpers and built-in retry behavior.

    Suits
    Friendlier than working the protocol directly, with auto-waiting on element lookups and a debug mode that shows the browser doing what the code asked for.
    Does not suit
    Chrome only, and the fluent API hides where an operation blocks, which makes timeouts harder to reason about in a concurrent crawler. Documentation is thinner than the JavaScript libraries it mirrors.

The question to answer before picking one

Does the page need a browser at all? Fetch it with curl and search the response for a sentence you can see on screen. If the sentence is there, the server rendered it, and a browser costs you a CPU core and a few hundred megabytes for a result you already had.

Plenty of sites sit in between: the article body is in the HTML and the price is not, or the first page of results is server rendered and page two is a fetch. Those are worth finding, because rendering only the pages that need it is usually a large saving rather than a small one. The rendering comparison covers how to read the difference between the two responses and what it does not prove. Hydration and prerendering cover why the same framework can produce either answer depending on how it was configured.

Memory is the ceiling, not CPU

A Chrome tab under load holds a few hundred megabytes, and the page's own JavaScript can double it. Multiply by concurrency and you reach the machine's limit long before the cores are busy. This is why every self-hosted browser fleet eventually grows a supervisor that kills and restarts browsers on a schedule, and why the hosted services in this list exist.

Two habits help more than any tuning. Block images, fonts, media and analytics at the request level, which cuts both bytes and layout work. And close pages rather than browsers, reusing a browser process across tasks, because process startup is the expensive part.

On detection

Headless Chrome is identifiable through the automation flag it sets, the shape of its TLS handshake, its HTTP/2 frame ordering and a handful of browser surfaces that differ from a normal profile. Vendors patch these, detection vendors find new ones, and any statement about what is detectable today has an expiry date.

The more useful distinction is between automation that identifies itself and automation built to look like a person. A crawler sending a documented user agent can be allowed or denied on purpose, which is the whole arrangement robots.txt rests on. ClaudeBot and PerplexityBot are examples of that side of the line.

Crawlers covers the frameworks that call these libraries. Scraping APIs covers hosted services that run the browser for you. Browser agents covers the layer above these: tools that take a goal and work out the clicks at run time, where everything on this page runs the clicks you wrote. Open source projects lists the libraries here with their licenses alongside the extraction tools.

Questions

Do I need a headless browser to crawl a page?

Only if the content is not in the HTML the server sent. Fetch the page plainly and search the response for a string you can see on screen. If it is there, a browser adds a few hundred milliseconds and several hundred megabytes for nothing.

What does a headless browser actually cost to run?

Roughly a CPU core and several hundred megabytes of memory per concurrent page, plus whatever the page itself allocates. That is why hosted browser services exist and why a crawl that renders everything costs far more than one that renders selectively.

Is headless Chrome detectable?

Yes, by several independent signals: the navigator.webdriver flag, the TLS and HTTP/2 handshake shape, timing patterns and missing browser surfaces. Detection is an ongoing contest rather than a fixed check, so any claim that a tool is undetectable has a date on it whether or not it is written down.

Should I block images and stylesheets while crawling?

Usually yes. Blocking images, fonts, media and analytics at the request level cuts bandwidth and page load time sharply and rarely changes the extracted text. Keep stylesheets only if you depend on computed layout or visibility.