AGENTS.md: Add CLAUDE.md symlinks, architecture decisions, and common pitfalls

[youknowriad]

What?

Improves the AGENTS.md setup for coding agents working on Gutenberg.

Why?

Claude Code only reads CLAUDE.md, not AGENTS.md, so it was getting zero project context. The existing AGENTS.md also lacked sections on architectural decisions and common pitfalls, which are high-value for reducing agent mistakes.

How?

1. CLAUDE.md symlinks — Created at root, packages/components/, and packages/ui/ pointing to their respective AGENTS.md files.
2. Architectural decisions section — Covers package layering, block data model, data layer, styles system, and modularity. Links to full architecture docs.
3. Common pitfalls section — Covers lib/compat/ targeting, private APIs in bundled packages, block-editor WP-agnosticism, and @wordpress/scripts genericity.

Testing Instructions

1. Verify symlinks resolve correctly: ls -la CLAUDE.md should show -> AGENTS.md
2. Start a fresh Claude Code session in the repo and confirm it picks up the context from CLAUDE.md
3. Review the new sections in AGENTS.md for accuracy

🤖 Generated with Claude Code

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: youknowriad <youknowriad@git.wordpress.org>
Co-authored-by: aduth <aduth@git.wordpress.org>
Co-authored-by: mtias <matveb@git.wordpress.org>
Co-authored-by: t-hamano <wildworks@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[aduth]

Makes sense; I've seen similar setups in other projects 👍

[t-hamano]

Symlinks are a bit tricky on Windows host OS. Git's default configuration doesn't work properly with symlinks.

Instead, how about adding @AGENTS.md to CLAUDE.md? This seems to be the official way to do it. Many people have reported this format working correctly in this issue. anthropics/claude-code#6235 (comment)

[t-hamano]

LGTM 👍

[justlevine]

Did anyone here eval either the CLAUDE.md file or the ai-generated additions to the AGENTS.md file before merging this?

[tellthemachines]

I think the contents of that AGENTS.md would be helpful to humans, not only to AI. Nothing in there is AI-specific, and I for one would have benefited enormously from reading these clear cut guidelines when I started contributing to the project. Some of that stuff took me quite a while to figure out ���

So what I'd like to propose is adding these to the README.md, either instead of AGENTS, because agents already do read readmes (and we could reserve AGENTS for anything AI-specific that might need adding later) or duplicating it across both files.

I know the README already has links to extensive docs, but those are very fluffy in style so it helps a lot to have concise guidelines too.

[youknowriad]

@tellthemachines I agree and for me it's really unclear what should be the difference between README.md and AGENTS.md I feel like it could be the exact same file.

@justlevine not sure exactly how to do that personally, can you help?

[ciampo]

In our experience we found that human-facing and AI-facing documentation have different requirements, both in terms of information discovery (an agent and a human may "scan" and read the codebase in different ways) and in terms of structure/wording (documentation for agents needs to be a lot more explicit about what has to be done and especially what should not be done, in a writing style that is very different from documenation aimed at humans).

@aduth and @mirka can probably share more details.

[justlevine]

@youknowriad an eval is basically just a WordPress test report. You give an llm+harness a task and see how it behaves, then apply your context hack and give it the same task and see if the results are improved (accuracy, token count, time, whatever you were trying to "address" in the first place).

I think these changes were autogenerated by AI, which is itself problematic, but assuming they had been written with intent:

1. You notice that when asked to update a block pattern wtvr, it mangles the direct html instead of the updating the block-data.
2. Craft the prompt for eval e.g. "Change the buttons on {X} to links. The blocks should still be styled as buttons, and have a large enough target to pass WCAG. Update {X location} to change {x button} and {y buttons} blocks to links"
3. Turn off your memory run the prompt, and record the results.
4. Reset the diff, add your context hack that you're hypothesizing e.g from the diff: **Block data model**: Blocks are in-memory tree structures during editing, serialized as HTML with comment delimiters (). Work with the block tree via APIs, not the serialized HTML. then rerun the test. Did it do a better/faster,cheaper job?
5. Unless you're up-to-date on various agent cards, usually good to test against a few other harness/models too. E.g the ALL CAPS in the slopfile are particularly damaging to GPT-5 models (like the gratis copilot one given to FOSS contributors and powering Copilot PR review ).

Fun litmus: if the friction you're experiencing doesn't feel like it's worth the effort of a basic eval, it's probably not worth addressing via AGENTS.md - especially not in a public repo.

---

In our experience

@ciampo I don't want to be a jerk, but is this experience based on using Claude Code on this repo (where the AGENTS.md had no effect other than polluting discovery when the model greps for context on its own for the last 4 months)?

In terms of structure/wording (documentation for agents needs to be a lot more explicit about what has to be done and especially what should not be done, in a writing style that is very different from documenation aimed at humans).

To my knowledge, almost everything you wrote in this sentence is demonstratable incorrect. At best we can say that the authors have different goals when writing docs that conflict with what's most helpful for users/llms, resulting in lower signal to noise. It's also true that most folks suck at writing docs < inline docs + doc blocks < self-documenting code, which is why:

• the recommendations that get hyped for greenfield projects don't really have the same effect on established FOSS projects with good hygiene like wordpress-develop (and gutenberg though obviously to a much lesser intent)
• Claude Code even has instructions to evaluate and ignore CLAUDE.md, since most likely the contents being fed are slop.

That's what leads us to findings like (preprint:

Across multiple coding agents and LLMs, we find that context files tend to reduce task success rates compared to providing no repository context, while also increasing inference cost by over 20%.

LLM-generated context files increase cost and reduce performance LLM-generated context files cause performance drops in 5 out of 8 settings across SWE-BENCH LITE and AGENTBENCH

while

Human context files increase cost and performance We observe that the developer-provided context files outperform the LLM-generated ones for all four agents, despite not being agent-specific, and improve the performance compared to no context files for all agents but CLAUDE CODE

I've shared this (and other research etc previously) in https://core.trac.wordpress.org/ticket/63901 where I recommend we .gitignore all AGENTS.md files until we can:

1. show demonstratable proof that the specific contents do not actively hurt user+AI contributions
2. have code reviewers with enough "experience" to review, evaluate, and maintain proposed changes to those files once the initial ones have been created.

Would love for you to take a skim and share your thoughts there, so we can push the conversation forward 🙇

( Again I'm trying really hard and likely failing really hard at not sounding like a jerk.
This isn't meant to be a specific attack or condemnation. Sure llm usage can be derived from "first principles", but at the speed of iterations the implementation of them doesn't always resemble what one would assume, or what folks are hyping on socials. Even the folks may of the folks whose primary focus on bringing AI to WordPress core can't always keep up. And even well established committers and contributors test before commiting changes. AGENTS.md is a tool, not a doc, it needs to be triaged as such)

[mirka]

is this experience based on using Claude Code on this repo (where the AGENTS.md had no effect other than polluting discovery when the model greps for context on its own for the last 4 months)?

At least for the AGENTS files we added to @wordpress/components and @wordpress/ui, the exact process was:

1. Someone put together several realistic prompts that a designer or developer could use on an LLM ("Confirm before permanently deleting a post. What WordPress component should I use?", "Build a login form", etc).
2. These were our test prompts to first check how they work with the current repo setup. Most of the time, at least two different people tested the same prompt in their own environments, to reduce the chance of accidental success/failure. Sometimes they revealed a gap, sometimes they already worked fine. When there were gaps, we tried making improvements to the repo setup.
3. Finally, we used the same prompts to confirm that those improvements were actually effective. (Then repeat until we found an improvement that actually works.)

In terms of structure/wording (documentation for agents needs to be a lot more explicit about what has to be done and especially what should not be done, in a writing style that is very different from documenation aimed at humans).

This is specifically referring to this section, which was revealed as a gap in the iterative process explained above. I think this was the third iteration of this wording, until the agent actually stopped the user for confirmation instead of silently proceeding to build what the user asked for.

The test prompt in this case was "Write a login form component (WP compatible) that uses components and form validation primitives from the @wordpress/ui package in the packages/ui folder". This was a real prompt where someone mistakenly believed they should already start using this new components package. Tested with Cursor, Opus 4.5.

[justlevine]

Thanks for sharing @mirka , that sounds like a great process, and indeed that resulting AGENTS.md seems to resemble best practices much more closely than the one in this PR 🙇

(I'm assuming you confirmed that the testers where using harnesses that supported nested AGENTS.md, and then reevaulated claude code's behavior after this PR added the CLAUDE.md shim? )

I guess my follow up questions are:

1. Where is that information shared or recorded? I don't see anything related on Components: Add usage guidance for agents and Storybook #74815 . Even the testing instructions are just to see storybook...
2. How can we ensure that same (albeit more transparently communicated) process can be used across other new AGENTS.md files or updates to existing once that were tested - and prevent a repeat of this PR slipping past y'all?

[youknowriad]

This PR was indeed pushed by Claude Code but I have suggested all the additions myself. If you have suggestions and improvements please open PRs

[mirka]

Acknowledging that it's probably not an exact science, I do agree that in hindsight it does make sense to document in the PRs what the actual prompts were that the changes improved against. It at least documents what the intended use case was, and that there was some attempt to verify that an improvement was observable. Perhaps similar to the "measure first, optimize second" rule of performance improvements.

[youknowriad]

The main difference is that AI is a tool to help developers, performance is something that impact users. What's important is the actual user benefits, and that we ship good features.

I'd love to be able to automate these evals, but we shouldn't forget that all of this is "busy work" that has no immediate value for the users.

[tellthemachines]

In terms of structure/wording (documentation for agents needs to be a lot more explicit about what has to be done and especially what should not be done, in a writing style that is very different from documenation aimed at humans).

This is specifically referring to this section, which was revealed as a gap in the iterative process explained above.

That section is the only part of any of the AGENTS.md files in gutenberg that reads like it's aimed at AI specifically, and that's just due to the way it is worded: "When a human asks...". The actual information contained in the paragraph is still just as useful for humans as AI. I can't find anything in the README of the UI package that explicitly says that these components are not to be used yet and we should use the ones from the components package instead.

I might not be the sharpest knife in the drawer, but the way our AI docs in AGENTS.md files are written is much easier for me to parse than the corresponding README.md files. Maybe some of us need that extra level of explicitness. This is an open source project that should be welcoming of contributions from many different people with different skill level, and having explicit instructions about the most basic things in the docs can only be beneficial for everyone.

[justlevine]

The main difference is that AI is a tool to help developers

Exactly... just because AGENTS.md is coded in natural language, it doesn't make it any less of a tool configuration for your LLM harness.

We didn't start migrating to typescript until committers had grown comfortable committing with it, nor would we change our phpcs/eslint configs without seeing what the value or impact of the new smells would be. I'd like to believe we wouldn't blindly merge in a new build process because it "looks like what we've seen elsewhere", and if we didn't have an automated way to test it, contributors and reviewers would submit a Test Report.

And unlike failures for those tests, because inference isnt idempotent, contributors don't have an easy way to tell it's the config that's broken (or written exclusively for those with company sponsored CC Max $200 plans), they're just left paying significantly more money to produce significantly worse results.

This PR was indeed pushed by Claude Code but I have suggested all the additions myself. If you have suggestions and improvements please open PRs

As previously moted my suggestion is to immediately revert this, and ideally all AGENTS.md files from WordPress/* repos until committers have the know how to review and test them (same as any other tool)

I can open up a Gutenberg-specific issue+pr if you insist, but considering the research, studies, and best practices for FOSS arent gutenberg-specific, I'd much rather keep the conversation centralized on the Trac ticket I linked. LMK and I'll can try to do it next week (after the b1 hecticness) 🙇

[aduth]

I'd like to believe we wouldn't blindly merge in a new build process because it "looks like what we've seen elsewhere"

If this is referring to my review comment in #75507 (review), I'll clarify that the text of that comment was directed specifically at the change to link agent files (i.e. with symlinks and later @ referencing), and not a commentary about AGENTS.md broadly and blindly adopting trends. The agent files themselves had already existed. Though I'm interested to follow the Trac discussion on how we can measure the effectiveness of the content and tooling itself.