OpenAI Computer Use

Wire OpenAI's computer-use tool into a Steel browser session.

The OpenAI Computer Use integration runs OpenAI's vision-based agent loop on a Steel browser session. The model takes screenshots through Steel, decides the next action (click, type, scroll), and Steel executes it, so you can automate complex web tasks without writing custom selectors.

It pairs well with Steel's anti-bot capabilities, proxy support, and sandboxed environments.

Requirements

  • OpenAI API Key: An OpenAI model with computer use
  • Steel API Key: Active Steel subscription
  • Runtime: Python 3.10+ or Node.js 20+

Connect Steel to OpenAI

Steel's sessions.computer API takes screenshots and executes actions; pair it with OpenAI's Responses API computer-use tool:

Typescript
import { Steel } from "steel-sdk";



const steel = new Steel({ steelAPIKey: STEEL_API_KEY });

const session = await steel.sessions.create({

  dimensions: { width: 1024, height: 768 },

});



// Take a screenshot via Steel

const { base64_image } = await steel.sessions.computer(session.id, {

  action: "take_screenshot",

});



// Send to OpenAI's Responses API with computer-use, then route returned

// actions back through `steel.sessions.computer({ action: ... })`.

Full runnable starter: Steel + OpenAI Computer Use recipe →

Resources

Contents

RequirementsConnect Steel to OpenAIResources