Skip to content

Say plainly when an ability is not there, and name the closest ones - #385

Merged
galatanovidiu merged 6 commits into
WordPress:trunkfrom
webmyc:fix/ability-not-found-answers
Oct 15, 2026
Merged

galatanovidiu merged 6 commits into
WordPress:trunkfrom
webmyc:fix/ability-not-found-answers

Conversation

@webmyc

@webmyc webmyc commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

What?

Closes #331

When an MCP client asks for an ability that is not there, say so plainly and name the closest abilities it can call. An ability that is registered but not exposed gets the same answer, since an MCP client cannot call it either.

Why?

Clients guess ability names. Today an unknown name gets "Ability 'x' not found", so the client guesses again. In apeatling's build benchmark one model made 100 to 150 such calls per run, and in 14 requests agents sent the site owner looking for a switch that does not exist: mcp.public!=true reads like a setting.

I see the same on sites running Respira: for one redirect task agents try rank-math/create-redirection, rankmath/create-redirection and yoast-seo/create-redirect. Answering with the closest real names stopped most of that there.

How?

  • AbilityNameSuggester::suggest() ranks registered, MCP-exposed abilities against the requested name: the action part weighs most, separators are ignored (rank-math = rankmath), and word order is ignored (redirection-create = create-redirection). Up to three, above a 0.6 floor. The action itself must also score 0.6 or more, so a shared namespace alone never makes a suggestion, and a read ability is never suggested for a write (get-site-info for set-site-info), as apeatling's benchmark runs showed. Names over 200 characters get no suggestions.
  • Unknown name: Ability 'x' not found on this WordPress site. Closest abilities you can call: a, b. With nothing close, it points to the mcp-adapter-discover-abilities tool. Names are in the message because tool results drop WP_Error data today (ToolsHandler::create_error_result() discards WP_Error's error_code/error_data, leaving clients no machine-readable way to distinguish failure types #298); they are also in the error data as suggestions.
  • Registered but not exposed: the same ability_not_found answer, as galatanovidiu suggested. Naming the difference is what sent agents looking for a switch, and an unexposed ability is not in discovery either. A plain "disabled, enable it here" answer can come once there is a way to enable abilities.
  • Case and whitespace slips (RankMath/Create-Redirection, rankmath/create-redirection) still get the real name as the first suggestion: the name is normalized for scoring only.
  • check_ability_mcp_exposure() triggers _doing_it_wrong for unknown ability names #331: wp_has_ability() before wp_get_ability(), as gziolo suggested, so an unknown name no longer writes a _doing_it_wrong notice. The four tests that expected that notice no longer do.

Error codes: an unexposed ability now returns ability_not_found instead of ability_not_public_mcp, and the docs say so.

Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Claude Opus 5.5
Used for: implementation and tests, from my description of the behaviour; I reviewed the code and the messages.

Testing Instructions

  1. Register a public ability rankmath/create-redirection.
  2. Call mcp-adapter/execute-ability with ability_name: rank-math/create-redirection. The error names rankmath/create-redirection, and the debug log has no _doing_it_wrong entry.
  3. Call it with core/create-page. The error points to the mcp-adapter-discover-abilities tool.
  4. Register test/hidden with meta.mcp.public = false and call it. The answer is the same ability_not_found, and test/hidden is not among the suggestions.

npm run test:php: 523 tests pass (new AbilityNameSuggesterTest). phpcs and phpstan clean.

Changelog Entry

Changed - An unknown ability name gets a plain answer naming the closest abilities the client can call, and an ability that is not exposed through MCP gets the same answer instead of one that reads like a site setting.
Fixed - An unknown ability name from an MCP client no longer triggers _doing_it_wrong (#331).

Open WordPress Playground Preview

An MCP client that asks for an ability that is not registered usually
guessed the name. "Ability 'x' not found" leaves it to guess again, and
some models keep going for a hundred calls or more. An ability that is
registered but not exposed answered "mcp.public!=true", which agents read
as a setting and send the site owner looking for.

- Unknown name: the answer says the ability is not on this WordPress
  site and names up to three close abilities the client can call
  (another spelling of the namespace, the same words in another order).
  With nothing close, it points to mcp-adapter/discover-abilities. The
  names are in the message, since tool results do not carry error data
  yet (WordPress#298); they are also in the WP_Error data.
- Registered but not exposed: the answer says the plugin decides this in
  its code and the site has no setting for it.
- Fixes WordPress#331: an unknown name is checked with wp_has_ability() first, so
  it no longer writes a _doing_it_wrong notice.
Copilot AI balanced review requested due to automatic review settings October 14, 2026 22:48
@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: webmyc <urbankidro@git.wordpress.org>
Co-authored-by: galatanovidiu <ovidiu-galatan@git.wordpress.org>
Co-authored-by: apeatling <apeatling@git.wordpress.org>
Co-authored-by: justlevine <justlevine@git.wordpress.org>
Co-authored-by: gziolo <gziolo@git.wordpress.org>
Co-authored-by: lhero-org <lheroorg@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@codecov

codecov Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.43590% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.12%. Comparing base (81e02be) to head (64eb585).

Files with missing lines Patch % Lines
includes/Abilities/AbilityNameSuggester.php 96.55% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##              trunk     #385      +/-   ##
============================================
+ Coverage     93.05%   93.12%   +0.06%     
- Complexity     1347     1382      +35     
============================================
  Files            58       59       +1     
  Lines          3948     4016      +68     
============================================
+ Hits           3674     3740      +66     
- Misses          274      276       +2     
Flag Coverage Δ
unit 93.12% <97.43%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Suggestions exclude case and whitespace corrections, and the discovery fallback names the wrong MCP tool.

2 open findings
What changed in this PR

Improves unknown-ability errors with nearby callable names and suppresses unnecessary registry notices, addressing #331.

Changes:

  • Adds similarity-ranked suggestions for MCP-exposed abilities.
  • Clarifies missing and unexposed ability messages.
  • Adds suggestion tests and removes expected registry notices.
File Description
tests/​phpunit/​Unit/​Abilities/​GetAbilityInfoAbilityTest.php Removes expected unknown-name notice.
tests/​phpunit/​Unit/​Abilities/​ExecuteAbilityAbilityTest.php Removes expected unknown-name notices.
tests/​phpunit/​Unit/​Abilities/​AbilityNameSuggesterTest.php Tests suggestions and error messages.
includes/​Abilities/​McpAbilityHelperTrait.php Adds recovery messages and guarded lookup.
includes/​Abilities/​GetAbilityInfoAbility.php Guards lookup and reuses missing-name message.
includes/​Abilities/​ExecuteAbilityAbility.php Guards lookup and reuses missing-name message.
includes/​Abilities/​AbilityNameSuggester.php Ranks similar public ability names.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread includes/Abilities/AbilityNameSuggester.php Outdated
Comment thread includes/Abilities/McpAbilityHelperTrait.php Outdated
@webmyc

webmyc commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

Props note for whoever merges: this came out of @apeatling's benchmark finding in #core-ai (agents guessing ability names and sending owners after a missing switch), which the props bot can't see from Slack. Please add:

Co-authored-by: apeatling <apeatling@git.wordpress.org>

Comment thread includes/Abilities/McpAbilityHelperTrait.php Outdated
@apeatling

Copy link
Copy Markdown

Tested with Grok 4.7, the model that made 100+ calls a run in the Build benchmark, on a request no exposed ability can do (add an opening hours page). Runs capped at 60 ability calls.

This works: with the PR, 4 of 5 runs told the user the site can't do it. Without it, 0 of 1.

Ended on its own Ability calls Model calls
0.7.0 0 of 1 129 (capped) ~72
PR @ a135f8b 4 of 5 8–17 (one capped at 81) ~25–53

One change: only suggest a name when the action matches. A shared core/ namespace lifts unrelated abilities over MIN_SCORE, and in the run that didn't stop, Grok used these suggestions to keep guessing (a 42-name loop):

Requested Suggested Score Action score
core/get-opening-hours core/get-user-info 0.63 0.47
core/set-site-info core/get-site-info 0.94 0.91

A minimum action score (~0.6) drops the first. Not suggesting a read ability for a write request drops the second.

Small sample (one request, 6 runs), so a strong signal rather than proof.

Comment thread includes/Abilities/AbilityNameSuggester.php
Comment thread includes/Abilities/McpAbilityHelperTrait.php Outdated
@webmyc

webmyc commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @apeatling, that's the test I hoped for. Done in a545fbf: a suggestion now needs the action itself to match (score 0.6 or more), so a shared core/ no longer lifts core/get-user-info for core/get-opening-hours, and a read ability is never suggested for a write (core/get-site-info for core/set-site-info). Both cases are tests. Ovidiu's two notes are in the same commit.

@galatanovidiu galatanovidiu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both earlier comments are addressed. The oversized request now returns the normal error under the same memory limit, and suggestions are computed once. The action filters cover Andy's examples. The full suite completed with 543 tests and one skip; PHPCS and PHPStan pass. Looks ready to merge.

@galatanovidiu
galatanovidiu merged commit b05137e into WordPress:trunk Oct 15, 2026
39 checks passed
galatanovidiu added a commit that referenced this pull request Oct 16, 2026
## What?

Prepare the repository for the 0.7.1 release, covering #366, #372, #373,
#375, #385, #386, and #387.

- Set the plugin header, `McpAdapter::VERSION`, and WordPress.org stable
tag to `0.7.1`.
- Replace all 31 development-version annotations in eight source and
test files with `0.7.1`.
- Date the changelog section `2026-10-15` and add the missing entries
for ability errors, plugin loading, the site-owner guide, release
deployment, and development dependency updates.
- Add the WordPress.org changelog and a 217-character upgrade notice,
including the reconnect required when oversized session data is cleared.

## Why?

The tag must package a plugin that reports 0.7.1 consistently through
WordPress, the public version constant, and WordPress.org metadata. The
release workflow builds the tagged source; it does not update these
fields or finalize development annotations.

## How?

This changes release metadata and documentation only. Composer and npm
manifests have no explicit package version to bump; Composer takes the
package version from the Git tag, and the release build supplies that
tag through `COMPOSER_ROOT_VERSION` for Jetpack Autoloader arbitration.
Historical version annotations, migration links, and dependency
requirements stay as they are.

The changelog documents the new `ability_not_found` response for
unexposed abilities, the session storage limit, and the omission of
retained client icons. These behaviors are already merged; this PR
records them for the release.

## Testing Instructions

- `npm run test:php`: 543 tests, 4,731 assertions, one skip, no
failures.
- `npm run lint:php:stan`: no errors.
- `npm run lint:php`: zero errors and the existing `WP_CLI_Command`
inheritance warning in `includes/Cli/McpCommand.php`; exits 1 because of
that warning. PHPCS passed separately on all ten changed PHP files.
- Built a disposable production copy with tests removed before
`COMPOSER_ROOT_VERSION=0.7.1 composer install --no-dev
--no-interaction`, then ran `npm run plugin-zip` using the installed
repository tooling.
- Extracted the ZIP and ran `php bin/verify-release-manifests.php`:
every referenced file exists. Verified the plugin header, stable tag,
and public version constant report 0.7.1, and the Jetpack class manifest
uses normalized version `0.7.1.0`. The ZIP contains `composer.json` and
excludes tests.
- Checked that no development-version placeholders remain in source or
tests, the upgrade notice is within 300 characters, and `git diff
--check` passes.

## Changelog Entry

> Developer - Prepare version metadata, release annotations, and
documentation for 0.7.1.

<!-- wp-playground-preview:start -->
<a
href="https://playground.wordpress.net?blueprint-url=data:application/json,%7B%22%24schema%22%3A%22https%3A%2F%2Fplayground.wordpress.net%2Fblueprint-schema.json%22%2C%22login%22%3Atrue%2C%22landingPage%22%3A%22%2Fwp-admin%2Fplugins.php%22%2C%22preferredVersions%22%3A%7B%22php%22%3A%228.5%22%2C%22wp%22%3A%22latest%22%7D%2C%22features%22%3A%7B%22networking%22%3Atrue%7D%2C%22steps%22%3A%5B%7B%22step%22%3A%22installPlugin%22%2C%22pluginData%22%3A%7B%22resource%22%3A%22url%22%2C%22url%22%3A%22https%3A%2F%2Fgithub.com%2FWordPress%2Fmcp-adapter%2Freleases%2Fdownload%2Fci-artifacts%2Fpr-389-6e667f592840cd788188e807d4640f5ac5fb9b0e-mcp-adapter.zip%22%7D%2C%22options%22%3A%7B%22activate%22%3Atrue%7D%7D%5D%7D"
target="_blank" rel="noopener noreferrer">
<img
src="https://raw.githubusercontent.com/adamziel/playground-preview/refs/heads/trunk/assets/playground-preview-button.svg"
alt="Open WordPress Playground Preview" width="220" height="57" />
</a>
<!-- wp-playground-preview:end -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

check_ability_mcp_exposure() triggers _doing_it_wrong for unknown ability names

4 participants