Skip to content
TilloTech Docs

OpenAI: skills in Agents SDK maintenance

Source: Using skills to accelerate OSS maintenance

Published: 9 March 2026.

Evidence type: detailed case study from the OpenAI Agents SDK Python and TypeScript repositories, with operational recommendations. Reported throughput increased from 316 to 457 merged PRs across compared three-month periods, but the article does not establish skills as the sole cause.

The comparison is December 2025–February 2026 against September–November 2025: Python increased from 182 to 226 merged pull requests and TypeScript from 134 to 231. At the time, OpenAI also reported roughly 14.7 million 30-day PyPI downloads and 1.5 million npm downloads. That scale makes release compatibility and consumer-path testing central to the case study.

Repository operating system

The repositories combine a short AGENTS.md, local .agents/skills/, optional scripts/references/assets, and the Codex GitHub Action. Metadata is available for routing; the body and support files load only after selection.

The skill catalogue is deliberately workflow-oriented:

  • ordered code-change verification;
  • documentation drift analysis;
  • automated example execution;
  • implementation strategy and compatibility boundaries;
  • current official product documentation lookup;
  • test-coverage prioritisation;
  • release-readiness review;
  • changeset and version-bump validation;
  • published-package integration testing;
  • coordinated toolchain upgrades;
  • rigid PR handoff generation.

The transferable pattern is a narrow contract, clear trigger, and concrete output. Some are gates; others are report-first and require approval before edits. docs-sync, for example, audits and prioritises before mutation and treats source docstrings/comments—not generated docs—as authoritative.

Mandatory trigger design

High-value conditional triggers appear near the top of AGENTS.md:

  • use implementation strategy before compatibility-sensitive runtime/API work;
  • run verification after changes to runtime, tests, examples, or build/test behaviour and do not claim completion until it passes;
  • validate changesets for affected packages;
  • consult official current docs for OpenAI platform work;
  • prepare the PR handoff when substantive work is ready.

The root file also holds release-critical compatibility invariants. One example preserves positional meaning for exported constructors and data fields, appending optional parameters where possible and requiring compatibility tests if reordering cannot be avoided.

Verification is conditional, avoiding the full stack for docs-only changes, but completion-blocking when applicable. The actual ordered commands live in the skill, giving the repository an explicit definition of “verified.”

Judgement versus scripts

The exact published verification order is:

# Python
make format
make lint
make typecheck
make tests


# TypeScript
pnpm i
pnpm build
pnpm -r build-check
pnpm -r -F "@openai/*" dist:check
pnpm lint
pnpm test

Order is part of the contract because earlier steps prepare artifacts required by later checks.

OpenAI’s split is explicit:

  • the model reads source to infer intended behaviour, compares logs, judges compatibility risk, and explains actionable findings;
  • scripts execute fixed command sequences, manage processes, collect logs, locate release tags, and expose repeatable start/stop/status/log/rerun commands.

Scripts are treated like small CLIs: deterministic stdout, loud usage and error failures, and known artifact paths. Rediscovery of the same shell recipe is a signal that a script is missing.

Validation beyond exit codes

The teams first made examples non-interactive: answering common prompts, supporting approvals where safe, maintaining an explicit skip list, creating structured per-example logs, and writing rerun files. Only then did they wrap the runner in a skill.

The runner preserves evidence. Codex reads every example’s source and comments, infers its intended flow, opens its matching log, and compares intent with actual output. A zero exit status is insufficient for examples involving real APIs, tools, or structured output.

The TypeScript repository adds consumer-path testing: publish packages to a local registry, then install and execute them under Node, Bun, Deno, Cloudflare Workers, and a Vite React application. This distinguishes in-repository correctness from published-package correctness.

The TypeScript changeset workflow also compares release metadata with the actual package diff. It reuses an existing branch changeset, requires a one-line Conventional Commit-style summary, checks the bump level, and encodes pre-1.0 and preview-package policy. Merely finding a .changeset file is insufficient.

Release and handoff workflows

Release review diffs the previous release tag against current main and checks public compatibility, behavioural regression, and missing migration or release notes. It starts from “safe to release” and blocks only on concrete evidence; every block needs a specific unblock checklist.

PR handoff collects branch, status, changed files, diff statistics, and recent commits, then produces a rigid branch/title/description artifact. It is scoped to substantive completed work rather than every turn.

CI and review boundaries

OpenAI recommends stabilising a skill locally before running it in CI. Public repository triggers require trusted actors/events or explicit approval, sanitised untrusted input, protected API credentials, an unprivileged runtime, and careful job ordering.

The case study uses automated review for routine program bugs, regressions, and missing tests. Humans remain decision-makers for competing API/architecture options, behaviour and compatibility promises, naming/migration/release communication, rollout policy, and cross-team sequencing.

Standards implications

  • Put mandatory conditional workflow triggers near the top of agent instructions.
  • Define verification by change class and ordered executable evidence.
  • Make examples and packages testable through consumer-realistic, non-interactive runners.
  • Require evidence and unblocking actions from model-based gates.
  • Separate automated correctness review from consequential human choice.