WordPress / WordPress/php-ai-client
Non-generative capability for typed judgement models: classification and scoring with probabilities and confidence
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 308
- Forks
- 84
- Avg merge
- 7d 21h
- Merged PRs (30d)
- 2
Description
The client models seven capabilities, six generative plus embeddings. There is a class of model it does not cover: one that takes state plus typed questions and returns a probability, a choice over a defined set, or a position on a defined scale, each with a confidence figure, and generates nothing. TypeSafe's Jev (System One) is the first I have worked with; the shape is general.
Today the only way in is TEXT_GENERATION with a JSON response schema, which means asking a generative model to emit a number, then parsing prose. A judgement model answers the typed question directly and its answer carries a calibrated confidence, which the parsed number does not. Forcing it through the text path throws that away. It also means a WordPress connector for such a model cannot register as ai_provider, because the provider's key is validated against generative operations it does not have.
Embeddings are the precedent: a non-generative capability with its own enum value, model interface, builder and result DTO (CapabilityEnum::EMBEDDING_GENERATION, EmbeddingGenerationModelInterface, EmbeddingBuilder, EmbeddingResult). A judgement capability would follow the same pattern:
CapabilityEnum::CLASSIFICATION(orDECISION; naming is yours)- a model interface taking
stateplus a map of typed questions:noul(probability that a statement is true),choiceover named options,scoreover ordered levels - a result DTO: per question, the value, a probability distribution where one applies, and a confidence figure
- a builder entry point alongside
generateEmbedding()
Request and response shape, as the live API has it:
// request
{ "state": { "comment": "..." }, "model": "jev-latest",
"questions": {
"spam": { "type": "noul", "instructions": "Is this comment spam?" },
"route": { "type": "choice", "instructions": "How should a moderator handle it?",
"criteria": { "approve": "...", "hold": "...", "trash": "..." } } } }
// response
{ "model": "jev-latest",
"answers": {
"spam": { "noul": 0.03 },
"route": { "choice": "approve", "confidence": 0.91,
"probabilities": { "approve": 0.91, "hold": 0.07, "trash": 0.02 } } },
"usage": { "input_tokens": 312, "output_tokens": 9 } }
I maintain a WordPress connector for Jev (https://github.com/juanlentino/jev-connector) that has this shape working end to end against the live API, plus a consumer that backs the AI plugin's Comment Moderation through its wpai_comment_analysis_result filter (https://github.com/juanlentino/jev-comment-analysis). Happy to draft the interfaces and DTO as a PR if the shape is welcome; I would rather get the naming and placement from you first.
A companion Trac ticket about the Connectors API side (a documented type for AI services the client does not drive) will follow. Jeff Paul suggested filing here after a conversation about where Jev-like integrations should live.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing EmbeddingGenerationModelInterface, EmbeddingBuilder, EmbeddingResult, and CapabilityEnum::EMBEDDING_GENERATION as the stated precedent. Clarify the capability naming, typed-question model interface, result DTO, builder entry point, and provider registration expectations with maintainers before implementation. Done means the agreed design preserves typed answers, distributions where applicable, and confidence without routing through TEXT_GENERATION.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- ai, api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100