PolicyEngine / PolicyEngine/policyengine-uk-chat
Add variable lookup tool after compiled variable metadata exists
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- Avg merge
- 16h 46m
- Merged PRs (30d)
- 2
Description
Problem
UK Chat should eventually be able to answer questions such as:
- what does
income_taxcalculate? - which entity does Universal Credit live on?
- what variables are available for a programme?
- what formula/source definition is associated with a variable?
However, the chat backend currently uses policyengine_uk_compiled as its calculation source of truth, and the compiled package does not yet expose a variable metadata/formula catalog. Pulling variable metadata and formulas from the separate policyengine_uk package would risk showing users metadata that diverges from the compiled engine actually used for calculations.
Dependency
Blocked on PolicyEngine/policyengine-uk-v2#106 exposing machine-readable variable metadata from policyengine_uk_compiled, especially:
- canonical variable name
- label/title
- entity
- description/definition
- input/output/intermediate status where available
- formula/source reference where feasible
Suggested scope after the dependency lands
- Add a
lookup_variablemodel-facing tool backed bypolicyengine_uk_compiledmetadata. - Support exact variable names and deterministic natural-language matching.
- Return
needs_confirmationwith ranked options when the string match is ambiguous. - Clearly label formula/source fields according to the compiled package contract.
- Add unit tests for exact, natural-language, ambiguous, and unknown variable queries.
- Add eval coverage across
tool_contract,trajectory, andanswerso model responses do not over-claim source/formula certainty.
Out of scope
Do not source variable formulas from policyengine_uk in UK Chat unless the product explicitly accepts that divergence risk.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by checking PolicyEngine/policyengine-uk-v2#106 and the metadata contract it exposes from policyengine_uk_compiled. Then inspect the chat backend's model-facing tool integration and the named eval areas: tool_contract, trajectory, and answer. Done means exact, natural-language, ambiguous, and unknown variable queries are tested without sourcing formulas from policyengine_uk.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100