AI TrackerGet the app
← All news

OpenAI describes the Performance Factory, a Codex-powered system that investigates ChatGPT performance regressions

In an engineering post, OpenAI's Okay Zed and Cyrus Forbes describe the Performance Factory, a cloud system in which Codex agents find and help fix performance problems in ChatGPT. Engineers define workflows as Markdown prompts and a configuration file listing a task's states and transitions.

  • Parts: a task service with configurable workflows, a dispatch loop that starts a Codex turn when a task's next step is ready, an internal API that runs Codex in the cloud under a service identity, and subscriptions that wake a task when something changes, such as a pull request comment or a finished deployment.
  • Regression Workflow: a performance alert opens a task. An agent investigates, a separate Codex session tries to disprove the diagnosis, and an agent proposes a fix as a pull request for an engineer to review. After deployment an agent checks whether performance has returned to baseline, and reopens the work if it has not.
  • Numbers, as OpenAI reports them: in just over three months the workflow tracked more than 1,700 investigations and root-caused over 600, about 17 a day. In a manual review of 100 root-cause suggestions, OpenAI graded 70% accurate; the rest needed refinement or did not fully establish causality.
  • Lessons it lists: protect engineers' attention, evaluate changes to agents (its perf-forensics skill was refined with evals built from past regressions), and turn recurring failures into self-repair.

Get news like this as a push.

AI Tracker alerts you when something changes for the tools you use.

Get the app