Automate a cloud browser with chromiumoxide
Use Steel with chromiumoxide to connect over CDP, drive the handler task, extract page content, and capture a screenshot.
Scaffolds a starter project locally. Requires the Steel CLI.
chromiumoxide speaks the Chrome DevTools Protocol over a websocket, which is exactly what a Steel session exposes. Browser::connect takes the session's websocket URL and hands back a connected browser plus a Handler. From there you get plain async chromiumoxide: new_page, content, get_title, find_elements, evaluate, screenshot. No local Chrome, no chromedriver, no display.
The connection is one line, but it returns a tuple, and the second half is the part that trips everyone up:
let (browser, mut handler) = Browser::connect(cdp_url).await?;let handle = tokio::spawn(async move { while let Some(_) = handler.next().await {} });
chromiumoxide splits the API surface (browser, page) from the connection's event loop (handler). The browser handle only queues CDP commands. Nothing is sent, and no response ever comes back, until something polls handler to completion. If you skip the spawn, browser.new_page(...) does not error: it hangs forever, because the future that would resolve it is never driven. This is the single most common chromiumoxide mistake. Spawn the drain loop right after connect, keep the JoinHandle, and abort it on the way out. run does exactly that.
One build-time gotcha that follows from the same design. chromiumoxide is runtime-agnostic and defaults to the async-std runtime, so a tokio program must opt in explicitly. The dependency in Cargo.toml is:
chromiumoxide = { version = "0.7", default-features = false, features = ["tokio-runtime"] }
Leave default-features on and the spawned handler silently runs on the wrong reactor, which surfaces as the same hang. Turn them off and name tokio-runtime.
Everything after the spawn is ordinary scraping. run opens Hacker News, waits for navigation, reads the title and full HTML, then pulls the top five stories with one page.evaluate call. The browser returns JSON, and chromiumoxide's into_value deserializes it straight into a Vec<Story>, so the extraction stays typed rather than a pile of per-element awaits:
let stories: Vec<Story> = page.evaluate(EXTRACT_STORIES).await?.into_value()?;
The screenshot uses page.screenshot, which returns the PNG as Vec<u8> directly from CDP. This example writes those bytes to screenshot.png, but the same bytes go just as easily into an upload, a vision model prompt, or a diff against a baseline.
Run it
cd examples/chromiumoxidecp .env.example .env # set STEEL_API_KEYcargo run
Grab a key at app.steel.dev/settings/api-keys. The first build pulls chromiumoxide and tokio and takes a minute or two; later runs are quick. As the program starts it prints a session viewer URL. Open it in a second tab to watch the remote browser load the page live.
Your output varies. Structure looks like this:
Creating Steel session...Session live at https://app.steel.dev/sessions/ab12cd34Connected over CDP, opening page...Title: Hacker NewsHTML length: 38214 bytesTop 5 Hacker News stories:1. Writing a Chrome DevTools Protocol client in Rusthttps://example.com/cdp-rust312 points2. Show HN: I built a headless browser farm on a Raspberry Pihttps://github.com/user/project188 points...Saved screenshot.png (245118 bytes)Releasing session...Session released
A run costs a few cents of browser time. Steel bills per session-minute, so the client.sessions().release() call after run returns is not optional: main captures the result, releases the session, and only then propagates any error, so a failed scrape still tears the session down instead of leaving it to idle until the default 5-minute timeout.
Make it yours
- Swap the target. Replace the URL in
new_pageand theEXTRACT_STORIESexpression with your own selectors. The JS runs in the page and returns any JSON-serializable shape; widen theStorystruct to match. Session setup and teardown stay the same. - Prefer typed element queries. If you would rather not write JS,
page.find_elements("tr.athing")returns chromiumoxideElementhandles withinner_textandattribute("href"). It is more Rust, more awaits, and easier to debug one node at a time. - Harden for anti-bot.
SessionCreateParamscarries the same knobs as the other SDKs. Setblock_ads,solve_captcha,use_proxy, or a customdimensionson the struct you pass tosessions().create()for sites that fingerprint or challenge headless traffic. - Keep the page bytes in memory. Drop the
std::fs::writeand feed theVec<u8>frompage.screenshotstraight to whatever consumes it.
Related
- scrape-rs reaches the same page without a browser library, through Steel's
scrapeandscreenshotendpoints. Start there if you only need content or an image and never touch the DOM. - Selenium drives Steel over WebDriver instead of CDP.
- playwright-py is the same connect-over-CDP shape in Python, useful for comparing the handler model against Playwright's.
- chromiumoxide docs cover the
Page,Element, andScreenshotParamsAPIs in full.
Related recipes
Run a Steel browser job with Trigger.dev
Queue a Trigger.dev task that creates a Steel session, drives Playwright over CDP, saves artifacts, and releases the browser in cleanup.
Run a durable browser workflow with Temporal
Build a Temporal TypeScript Workflow that schedules retryable Steel browser Activities to capture page summaries, screenshots, and Markdown artifacts.
Automate a cloud browser with headless_chrome
Use Steel with headless_chrome, the synchronous Rust equivalent of Puppeteer, to connect over CDP and scrape quotes with element handles.