Skip to content

AI: Use Core's tool names, schema preparation and ability queries - #1073

Open
AllTerrainDeveloper wants to merge 4 commits into
trunkfrom
update/ai-core-compliance
Open

AI: Use Core's tool names, schema preparation and ability queries#1073
AllTerrainDeveloper wants to merge 4 commits into
trunkfrom
update/ai-core-compliance

Conversation

@AllTerrainDeveloper

@AllTerrainDeveloper AllTerrainDeveloper commented Nov 6, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Collaborator

What it does

Makes OpenStation's AI layer use WordPress core's APIs wherever core has an equivalent:

  • Tool names: agents and the Copilot offer abilities to the model under core's function names (wpab__desktop-mode__search-posts).
  • Tool schemas: cleanup goes through wp_prepare_json_schema_for_client().
  • Read-only abilities: the Copilot asks wp_get_abilities() for them.
  • Agent model: an agent's stored model now reaches using_model_preference().

It also fixes two agent bugs: agents could never call an ability with no input schema, and OpenStation's own ability could be dropped when a third-party ability with the same short name was listed first.

Behavior changes to review:

  • AI features now need WordPress 7.1, and switch off on 7.0. The rest of the plugin still runs from 6.0. The Drafts widget doesn't need the 7.1 helper, so it keeps its own check.
  • Agents with a stored model start using it. It's a preference, so the AI Client falls back when no provider offers that model.
  • Each request's system prompt gets a little longer, because of the tool name guide described below.

Rationale

OpenStation already relies on core for providers, keys and the ability registry. What it still duplicated was tool naming, schema cleanup and ability filtering, and it stored a model setting while waiting for a core API that now exists.

Core has no agent registry, skills API, MCP server, conversation store or tool loop, so the agent runtime and both loops stay.

Core's higher-level helpers aren't adopted, for these reasons:

  • using_abilities(): it keeps top-level oneOf/anyOf/allOf, and calling it alongside using_function_declarations() appears to replace the earlier declarations. The Copilot sends abilities and browser commands in one request.
  • WP_AI_Client_Ability_Function_Resolver::execute_ability(): on 7.1.4, empty arguments reach an object-schema ability as null and fail validation. It also sends WP_Error data to the model, and returns errors in a form callers can't reliably tell from a successful result.

Implementation

Short names stay wherever they're a contract or stored data:

Surface Name it uses
Sent to the provider wpab__desktop-mode__search-posts
Stable Copilot hooks, openstation_ai_tools, openstation_ai_search_generate, resumeTool search_posts, translated in openstation_ai_search_generate()
openstation_agent_runner_generate (Experimental) $tool_defs carry core names; a call may use either name
Agent traces, openstation_agent_tool_result Full ability name, unchanged

Default agent instructions are copied into user meta when an agent is created, so they say things like "get_post the source". A guide travels with the prompt so those and any custom instructions keep working, and agents still accept a short name if a model uses one:

These instructions may call a tool by a short name. Use the tool's full name when you call it:
- get_post: wpab__desktop-mode__get-post

Two guards stay on purpose:

  • Read-only check: openstation_ai_search_ability_names() rechecks the annotation after core's query. It's the only thing keeping write abilities away from a model reading untrusted text, and wp_get_abilities_result can reshape the query's result.
  • Name length: core has no limit, so a function name over 64 characters keeps its short name instead of risking a failed request.

Docs: docs/hooks-reference.md.

Testing instructions

npm run env:start:tests
npm run test:php:install
npm run build:apps
npm run test:php

Results on WordPress 7.1.4:

  • PHPUnit: 3,360 tests passing, 14 skipped.
  • Multisite subset: 136 tests passing, 2 skipped.
  • JavaScript: Vitest and typecheck pass.

I also ran the real AI Client and ai-provider-for-anthropic with HTTP intercepted and a scripted model, for one Copilot search and one agent run:

  • All 19 Copilot tools reached the request under core names, with the guide in the system prompt.
  • The tool result paired with the model's call.
  • openstation_ai_tool_called still reported search_posts.
  • The agent's stored model appeared in the request.

Not tested:

  • A real model: the scripted run shows what's sent, but not whether a model picks tools as well under the longer names. Before merging, please try a few Copilot queries and an agent run against a real provider.
  • Other providers: only the Anthropic provider was exercised, so dashes in tool names are unconfirmed elsewhere.
  • The 7.1 floor on an older WordPress: the check was only read and run on 7.1.4.

🤖 Generated with Claude Code

Open WordPress Playground Preview

AllTerrainDeveloper and others added 4 commits November 6, 2026 21:08
@AllTerrainDeveloper @claude
AI: Require WordPress 7.1 and use Core's tool schema preparation
fae134d 
The assistant and agents kept their own recursive walk to strip WordPress-only
keys from tool schemas. wp_prepare_json_schema_for_client(), new in 7.1, does
that and also lifts draft-03 per-property `required` flags into a list.

AI features now switch off below 7.1 through openstation_ai_is_available().
The rest of the plugin still runs from 6.0. The top-level reshaping stays,
because Core's helper keeps top-level combinators and providers reject them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AllTerrainDeveloper @claude
Agents: Run abilities through one dispatcher and keep colliding tools
5f3763f 
The runner passed an empty array to abilities with no input schema, and Core
refuses any non-null input to those, so an agent could never call one. The
Copilot already handled this; both now share openstation_ai_execute_ability().

The runner also dropped a tool when two abilities shared a short name, which
could lose OpenStation's own ability if a third-party one was listed first. It
now uses the Copilot's openstation_ai_unique_ability_tool_names(), so both stay
callable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AllTerrainDeveloper @claude
Agents: Use the stored model override
3b007d8 
An agent's model was saved, exposed over REST and exported for privacy, but the
runner never read it, because its docblock was waiting for Core to offer model
selection. Core does now, so pass it through as a preference. The AI Client
falls back when no provider offers that model, and openstation_ai_model_config
still has the last word.

Agents that already have a stored model will start using it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AllTerrainDeveloper @claude
AI: Offer abilities to the model under Core's function names
e53744d 
Agents and the Copilot invented their own tool names by stripping the namespace.
They now use WP_AI_Client_Ability_Function_Resolver's `wpab__` names, and the
Copilot asks Core's wp_get_abilities() for read-only abilities.

Short names stay where they are a contract or stored data. Copilot hooks and
its generation filter keep them, translated at the model boundary. Saved agent
instructions name tools by short name, so a guide mapping them to full names
travels with the prompt, and agents still accept a short name if a model uses it.

The read-only check stays after Core's query, because it is what keeps write
abilities from a model reading untrusted text and a filter can reshape the
result. A name over 64 characters keeps its short name, since one over-long
name would fail the whole request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AllTerrainDeveloper

AllTerrainDeveloper commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator Author

@claude can you review?

@claude

claude Bot commented Nov 6, 2026 •
edited
Loading

Copy link
Copy Markdown

Claude finished @AllTerrainDeveloper's task in 34s —— View job

Review of #1073: Core tool names, schema preparation and ability queries

I read the full diff of includes/, apps/, the docs and the readmes, and followed the call paths in runner.php and search.php. I didn't run the test suite. I found no blocking issues. The design holds together, and the places where it keeps its own logic are the right ones to keep.

What checks out

  • Tool-name round trip (Copilot): openstation_ai_search_generate() renames ability tools on the way out and maps calls back on the way in. The model's own name is kept in function_name, and the tool result echoes it ($fc['function_name'] ?? $tool_name). That pairing is what providers require. The stable hooks (openstation_ai_tools, openstation_ai_search_generate, resumeTool, openstation_ai_tool_called) keep seeing short names.
  • Tool-name round trip (agents): slug_by_name registers both the short and the full name, so saved instructions and a model that falls back to a short name both resolve. Traces still report the slug.
  • Read-only boundary: the annotation is re-checked after wp_get_abilities(), so nothing relaxes. The comment explains why.
  • Name collisions: the agent runner now goes through openstation_ai_unique_ability_tool_names(), so OpenStation's own ability is no longer dropped when a third-party ability with the same short name comes first. The 64-character fallback to the short name is sensible, given that core doesn't limit length.
  • openstation_ai_execute_ability(): it sends null only when there's no input schema and otherwise sends the decoded array. That fixes the no-schema agent bug without hitting the empty-args null problem you describe in core's resolver.
  • Version gate: openstation_ai_is_available() now requires wp_prepare_json_schema_for_client. The agent runner and draft.php both go through it. The docs, readmes and UI string are updated to match, and there are no version-history annotations.

Nits and things to consider

  1. wp_prepare_json_schema_for_client guard (search.php:~934). Below 7.1 (the openstation_agent_runner_generate seam only), openstation_ai_normalize_tool_schema() no longer strips sanitize_callback, validate_callback or arg_options. That's harmless in production, because availability already requires the function. A seam-only run that feeds real REST-derived schemas to a custom generator would now see those keys. The comment states the seam reason, so I'd leave it.
  2. Method existence (abilities.php, openstation_ai_ability_function_name). It checks class_exists( 'WP_AI_Client_Ability_Function_Resolver' ) but not method_exists( …, 'ability_name_to_function_name' ). If a 7.0.x build ships the class without that static method, a seam-only agent run would fatal. The guard is cheap to tighten.
  3. Name guide scope. The agent runner appends the guide once in openstation_agent_invoke(). Since it's added to $instructions before the loop, the wrap-up turn (no tools offered) should carry it too. I didn't trace that path, and the guide may be harmless noise there.
  4. Prompt growth. The guide lists one line per ability (19 for the Copilot). It's small, but it's paid on every turn, which is exactly the "does a real model pick tools as well" question you flagged as untested.
  5. Tests. The added cases cover the collision fix, the no-schema ability, the stored model and the schema normalization. I'd like to see one more: a Copilot call whose name is already a function name the PR didn't generate, such as a model replying with a short name in the Copilot path, to pin the "unknown tool" branch that now echoes function_name. It's optional.

Before merging
As your PR notes, the remaining risk isn't in the code. Try a few Copilot queries and one agent run against a real provider. Dashes in function names are only confirmed on Anthropic, so an OpenAI or Google run is the one I'd prioritize.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@AllTerrainDeveloper