PolicyEngine / PolicyEngine/policyengine-uk-chat

Add variable lookup tool after compiled variable metadata exists

Open
#140 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1
Forks
1
Avg merge
16h 46m
Merged PRs (30d)
2

Description

Problem

UK Chat should eventually be able to answer questions such as:

  • what does income_tax calculate?
  • which entity does Universal Credit live on?
  • what variables are available for a programme?
  • what formula/source definition is associated with a variable?

However, the chat backend currently uses policyengine_uk_compiled as its calculation source of truth, and the compiled package does not yet expose a variable metadata/formula catalog. Pulling variable metadata and formulas from the separate policyengine_uk package would risk showing users metadata that diverges from the compiled engine actually used for calculations.

Dependency

Blocked on PolicyEngine/policyengine-uk-v2#106 exposing machine-readable variable metadata from policyengine_uk_compiled, especially:

  • canonical variable name
  • label/title
  • entity
  • description/definition
  • input/output/intermediate status where available
  • formula/source reference where feasible

Suggested scope after the dependency lands

  • Add a lookup_variable model-facing tool backed by policyengine_uk_compiled metadata.
  • Support exact variable names and deterministic natural-language matching.
  • Return needs_confirmation with ranked options when the string match is ambiguous.
  • Clearly label formula/source fields according to the compiled package contract.
  • Add unit tests for exact, natural-language, ambiguous, and unknown variable queries.
  • Add eval coverage across tool_contract, trajectory, and answer so model responses do not over-claim source/formula certainty.

Out of scope

Do not source variable formulas from policyengine_uk in UK Chat unless the product explicitly accepts that divergence risk.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by checking PolicyEngine/policyengine-uk-v2#106 and the metadata contract it exposes from policyengine_uk_compiled. Then inspect the chat backend's model-facing tool integration and the named eval areas: tool_contract, trajectory, and answer. Done means exact, natural-language, ambiguous, and unknown variable queries are tested without sourcing formulas from policyengine_uk.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.