[TS Calls] Build a real-project call census and model roadmap

Open
#367 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
kotlin, typescript

Research direction

Start with the existing native frontend, runner, and observer, then review the inventory in #368 and the related work in #360, #364, and #385. Define the census profile, deterministic aggregation, machine-readable observations, and explicit limits described here. Done means a reviewed P0/P1/P2 model subset is ready for #385, with implementation split into bounded follow-up issues.

Written by the indexing model from the issue text.

Description

Part of #360. Uses #364; contributes to #385.

Goal

Measure unknown calls reached on real TypeScript projects and select a small evidence-based model roadmap.

Scope

  • Freeze development-project revisions, licenses, selection criteria, analyzed methods, input domains, frontend/tool versions, seeds and budgets.
  • Reuse the existing native frontend, runner and observer. Aggregate decision events outside the engine; #373 snapshots are not required for a basic census.
  • Define the observation profile explicitly. A stop-on-first-unknown run cannot reveal later calls on that path. Use a fixed opaque-continuation census and/or a separately reported stop profile to distinguish encountered calls and stopping points.
  • Opaque exploration can include infeasible concrete paths and omit callee effects. Report a census of observed events under the profile, not an exhaustive list of all calls or all runtime blockers in a project.
  • Normalize API families/methods without conflating same-named user methods with proven built-ins.
  • Count projects, distinct call sites, distinct containing functions and dynamic events with explicit denominators. Rank primarily by project prevalence and distinct sites; report loop event counts separately.
  • Classify observations as model candidates, frontend/engine defects, external/user behavior or already supported semantics.
  • Use #368's inventory to avoid reimplementing existing models. Record candidate domains, residual cases and the rationale for source code versus a genuine engine intrinsic.
  • Produce a small P0/P1/P2 list. Select a finite experimental subset before final evaluation; held-out projects never select or tune it.

Definition of Done

  • Machine-readable observations preserve profile, limits, project/source examples and raw counts.
  • Source-site identity and aggregation are deterministic; repeated budget-limited runs report observed variability instead of promising identical explored events.
  • Stop-profile censoring and opaque-profile approximation are explicit.
  • A reviewed model subset is ready for #385; missing implementation work is split into bounded family issues with evaluation dependencies.
  • No catalog expansion is implemented in this issue.

The integration examples in #365/#366 are allowed before this roadmap; they validate the mechanism and must not be retroactively presented as census-selected models. Corpus selection and aggregation work can proceed independently of #383.

Dominant language
Kotlin
Stars
33
Forks
27
Avg merge
4d 3h
Merged PRs (30d)
15

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from UnitTestBot/usvm

All issues in UnitTestBot/usvm

Similar issues

More Kotlin issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.