Google Agent Skills: The 6 CI Gates Behind 184 Skills
Google opened up the CI behind its Agent Skills repo: six gates, a YAML version trap, the L1/L2/L3 layout, and the deterministic trap its AI reviewer blocks.
The short version: Google ships skills like products, not documents
On October 29, 2026, Remigiusz Samborski, a Lead Developer Relations Engineer at Google, published a follow-up to his earlier behind-the-scenes post on the Google Agent Skills repository. The first post covered how the project started and how governance works. This one goes inside the machinery: the six automated gates every change must clear before it reaches GitHub, and the AI review agent that inspects skills for design quality.
The numbers are checkable. As of November 4, 2026, github.com/google/skills shows 21,830 stars and an Apache-2.0 license. The index.json registry at the repository root lists 184 skills. Google's post says "over 170." Both are accurate about different moments, but if you intend to cite a figure, hit the registry file — it is the source of truth and it moves daily.
The core claim is simple and slightly uncomfortable: a skill is not documentation. It steers an agent's reasoning loop, its tool selection, and its execution path. A misplaced phrase can push an agent into invalid command flags, an infinite loop, or a long stretch of wasted tokens. Under that framing, editing one sentence of an instruction is a behavior change — and behavior changes get versioned, tested, and evaluated.
Three requirements follow: strict semantic versioning (every behavioral change needs an explicit bump), deterministic automated validation (linters reject bad changes before any reviewer looks), and mandatory evaluation evidence (every change ships with a report comparing the skill against its previous version, or against an agent without it).
Six automated gates
When a pull request lands, a CI/CD pipeline triggers immediately. Before a skill is exported to GitHub, it must clear six automated gates plus one human approval:
- Structural conformance — directory layout, allowed files, taxonomy
- Skill linters — frontmatter, folder naming, Markdown rules
- Semantic versioning — a bump for every behavioral change
- Evaluation suite — valid suite, fresh eval report
- Link validation — links resolve
- AI review agent — quality and design standards
- Human approval — a subject matter expert signs off last
The structural rules are granular. Folder name and metadata name must match. SKILL.md and description length limits are enforced. Required frontmatter fields must exist. Only an allowlisted set of files may live inside a skill directory. And critically, README.md and the machine-readable index.json must be current.
The version string trap nobody expects
This is the detail most coverage skips, and the one most likely to bite on day one. The validator checks two separate things. First, the version must be a quoted string. YAML parses an unquoted 2.0 as a number, producing parsing errors in tooling that expects a string. Second, the transition must be valid. New skills start at 1.0.0; any change to instructions, scripts, or references requires a bump or the PR is blocked.
Why enforce versioning if the runtime model only reads the Markdown? Because the number serves developers and packaging tooling, not the model. Skills do not auto-update, so there is no network check or added latency per invocation. Explicit versions instead let developers diff local skills against the upstream catalog and decide when to pull — for example with npx skills update — or lock a critical skill to a known-good version for reproducible behavior.
I checked the CLI rather than taking the command on faith: the skills package is real on npm, version 1.7.6 at the time of writing, exposing skills and add-skill binaries.
Why eval suite validation is strict
Continuous evaluations compare agent performance with and without each skill. Those results are only as trustworthy as the suites behind them, so the validator enforces hard invariants: file-based prompts must resolve and not be empty; expectations must be atomic, checking exactly one criterion; duplicate prompts and conflicting expectations are flagged; a linked evaluation report is mandatory; and the report must have run against the latest commit. That last check earns its keep — authors iterate on a PR as reviewers comment, and it is easy to leave a report attached that describes a commit three revisions old.
Links: before merge, and again every Monday
More than 170 skills point at documentation for products that change fast. Pull request checks verify relative links such as files under references/. Then a weekly GitHub Actions workflow runs lychee across every external link and reports breakage to skill owners.
What the AI review agent blocks
Linters catch syntax. They cannot tell you an instruction is confusing, a reference is bloated, or a task should have been a bash script. So Google added an AI reviewer that inspects each change using a dedicated review skill — a skill that reviews skills.
The deterministic trap. The first question is whether the task belongs in an instruction at all. The recurring failure is authors writing rigid numbered CLI sequences directly into Markdown. LLMs are probabilistic engines; embedding a strict sequence where any deviation is an error invites hallucination. Rigid CLI steps, JSON parsers, and deployment steps belong in executable scripts underscripts/. Markdown is reserved for judgment work: choosing an architecture, diagnosing error logs, synthesizing code.
The L1/L2/L3 hierarchy. Level 1 is frontmatter metadata for discovery and routing. Level 2 is the SKILL.md body, which must stay under 500 lines. Level 3 is reference material in a references/ directory, subject to two rules: a flat structure and single-hop indirection. Chained references — SKILL.md pointing to a file pointing to another file — are blocked. Each hop costs a tool call, and Google's evaluations found agents frequently stop before following a second level. That is a measured decision, not a stylistic one.
Non-interactive scripts. Helpers are expected to run in terminal sandboxes, so anything that blocks on an interactive prompt is flagged. Scripts must take input via flags or environment variables, provide --help, include --dry-run or a confirmation flag for destructive operations, and write JSON to stdout and actionable errors to stderr.
Four lessons from six months of commit history
Scope CI checks to file operations. Early checks used broad directory patterns. One required an approved bug ticket on every change, so engineers fixing a one-line typo were blocked, while new skills in deeper subdirectories went uninspected. Rules should distinguish added from modified files and verify path depth. Make expectations atomic. Compound expectations produced inconsistent ratings from the LLM-as-a-judge scorer when the agent did two of three things. One criterion per expectation fixed the noise. Documentation drift happens within days. Never rely on contributors to hand-update an index. Write a generator and enforce it with a CI unit test.index.json literally opens with "This file is generated. Do not edit it by hand." That line is a scar.
Gate expensive LLM checks behind cheap deterministic ones. Full evals and an AI reviewer on every change is costly, especially with frequent small commits. Structural conformance, linters, and versioning block first; if any fails, neither the reviewer nor the evals run. Further guardrails worth adding: a manual /run-evals PR comment trigger, or automatic re-runs only above a diff threshold.
A build order that actually pays off
You do not need Google's scale, and you should not copy the apparatus at once.
index.json by hand — generator plus CI test.2.0 problem is a matter of when.SKILL.md — 500 is a reasonable start.What this means beyond Google
The point is not that Google has nice CI. It is that quality management for agents has moved outside prompt engineering and into developer infrastructure — versioning, eval atomicity, link freshness, cost control. None of it is glamorous, and all of it is unavoidable once a repository passes a hundred skills maintained by many teams.
The same pressure arrives at smaller scale the moment a team distributes skills internally. Within weeks someone asks which version is current, whether they can roll back, and why a link is dead. The answer at that point is not a better prompt. It is a generator script for the registry and a CI job that refuses to merge an unbumped version.