Path to more reliable agents: Migrating to OpenAI Agents API
There are two common ways to run agents with a sandbox: run the harness and tool execution together inside it, or run the harness separately and use the sandbox as a code interpreter tool. OpenAI Agents API gives us a managed harness while keeping tool execution in a sandbox we control. Over the past month, @eshwarhs and I have been working with OpenAI to migrate Hypha to this architecture.
Both the Agents API and our integration are still in beta. This article covers our initial architectural decisions and what we’ve learned so far.
Chaos Monkey testing
Before starting the agent rewrite, we had to assess what the benefits would be for Hypha. The main one is reliability. Building agents in finance is about earning customers’ trust. Making sure agents don’t fail is a big part of that trust.
Separating the agent harness from the sandbox with the Agents API cut failed agent responses by 86%. A customer who used to hit an error every month now sees one every seven.
You can see a visualization of a real trace from the Chaos Monkey testing we performed. We killed a sandbox in the middle of an agent turn to see how the system would handle it. Previously, the user would see an error and have to retry the turn from scratch, with the harness and sandbox restarting from the last completed turn.
You can get your own Chaos Monkey Codex pet.
Old Agent architecture
Hypha's old architecture is relatable to many startups.
- NextJS Frontend hosted on Vercel
- Supabase for DB and Object storage
- E2B as a sandbox provider
We run Codex SDK / Claude Agent SDK in the sandbox with HTTP Server to connect to the client and with some app orchestration around the SDKs for persistence, tracing and lifecycle management.
Such tight bundling has one benefit: lower latency and a multitude of issues:
- Agent overloading the sandbox so that it drops killing the harness too.
- Harness fails to boot inside the sandbox.
- HTTP Server is down and harness cannot be reached.
- Secrets for app orchestration in the same environment as tool execution.
- Sandbox crash looses the turn.
We could fix these issues by adding more orchestration code, but putting band-aids on suboptimal architecture is a wrong solution. Instead we decided to separate harness and tool execution.
Decoupling harness and execution
Separating the harness from the sandbox introduced two responsibilities we now handle through dedicated services: browser connections and conversation persistence in the Gateway, and sandbox lifecycle management in the Pool Manager.
Gateway
Previously, the browser connected directly to the HTTP server inside our E2B sandbox. Moving the harness to OpenAI meant we needed a new way for the browser to communicate with the agent.
That connection had three requirements:
- Keep our OpenAI API key on the server.
- Support streams beyond Vercel’s function execution limit.
- Persist conversation state to Supabase.
Our gateway handles all three through a small API for browser authentication, session management, streaming, and persistence:
| Gateway route | OpenAI Agents SDK | Gateway responsibilities |
|---|---|---|
POST /sessions | sessions.create() | Create session; persist in our db. |
POST /sessions/{id}/message | session.stream() | Stream events; persist messages and results. |
POST /sessions/{id}/cancel | session.cancel() | Cancel the turn. |
POST /auth/browser-token | — | Issue a short-lived browser login token to a trusted backend. |
POST /auth/login | — | Exchange login token for an HTTP-only cookie. |
POST /auth/logout | — | Clear the browser cookie. |
Browser authentication starts with our application backend requesting a short-lived login token from the gateway. The browser exchanges that token at /auth/login for a signed HttpOnly cookie. The browser sends this cookie with subsequent gateway requests, but JavaScript cannot read it. In production, the Secure flag also ensures it is only sent over HTTPS. Our OpenAI API key stays on the server.
For streaming, Microsoft’s @microsoft/fetch-event-source lets us submit a message and receive server-sent events in the same HTTP request. The native browser EventSource API only supports GET, while our message endpoint uses POST with a JSON body. We set credentials: 'include' so the browser sends the authentication cookie with the request.
The gateway is still in beta. We’ve designed it to be extensible so other applications can bring their own persistence. We’ll open-source it once it matures. Reach out if you’d like early access.
Pool manager
In our old architecture, sandbox lifecycle management was split between the sandbox itself and a separate monitoring function. After each agent turn, orchestration inside the sandbox started an idle timer and paused the container when it expired. The monitoring function later disposed of unused containers.
With the harness now running outside the sandbox, we moved lifecycle management into the Pool Manager. It handles webhooks from OpenAI and E2B to attach, resume, pause, and replace sandboxes:
| Source | Webhook | Action |
|---|---|---|
| OpenAI | agent.session.created | Attach sandbox to session. |
| OpenAI | agent.session.action_required with environment_connection | Resume or replace sandbox. |
| OpenAI | agent.session.idle | Schedule pause (default: 5 min). |
| OpenAI | agent.session.in_progress | Cancel scheduled pause. |
| OpenAI | agent.session.failed | Pause sandbox. |
| E2B | sandbox.lifecycle.killed | Replace sandbox. |
| E2B | sandbox.lifecycle.paused | Mark paused; schedule cleanup (default: 24 h). |
| E2B | sandbox.lifecycle.resumed | Mark runtime active. |
A scheduled sweep enforces pause and cleanup deadlines and reconciles sandbox state when webhooks are missed.
The integration is still in beta, but the goal is simple: agents that recover from infrastructure failures and keep working. We’ll share more as the architecture evolves.
Thanks to @eshwarhs for building this with me, and to the OpenAI team: @alfozan for working closely with us through the integration, Julien Reiman for coordinating the collaboration, and @stevendcoffey and @_rohanmehta for supporting the migration.