microsoft / microsoft/TypeAgent

Identify Test Gaps and Add Coverage

Open
#2,170 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
740
Forks
106
Avg merge
1d 6h
Merged PRs (30d)
84

Description

Since we're leaning heavily on AI and moving pretty fast, we should make sure we take time to invest in more robust test coverage. A quick scan of our test infra gives this overview :

Test Infrastructure Overview

Framework: Jest + TypeScript
Convention: .spec.ts = unit tests (offline),
.test.ts = integration tests (require API keys)
Root scripts: pnpm run test:local, test:live,
shell:test
Note: Tests run against compiled output in
dist/test/ — you must build first.


Packages WITH Tests

┌───────────────┬───────────────────────────┐
│ Package │ What's Covered │
├───────────────┼───────────────────────────┤
│ │ NFA/DFA compiler, grammar │
│ actionGrammar │ merging, terminal │
│ │ symbols, schema parsing │
├───────────────┼───────────────────────────┤
│ │ TypeScript → JSON Schema │
│ actionSchema │ conversion, type │
│ │ validation │
├───────────────┼───────────────────────────┤
│ cache │ LLM call caching, │
│ │ hit/miss logic │
├───────────────┼───────────────────────────┤
│ │ Action routing, │
│ dispatcher │ input→agent matching, │
│ │ type validation │
├───────────────┼───────────────────────────┤
│ knowPro │ RAG, conversational │
│ │ memory, semantic search │
├───────────────┼───────────────────────────┤
│ │ Individual agent action │
│ agents/* │ handlers (player, │
│ │ calendar, email, list, │
│ │ browser, weather) │
├───────────────┼───────────────────────────┤
│ cli │ Argument parsing, command │
│ │ execution │
├───────────────┼───────────────────────────┤
│ shell │ Electron GUI (via │
│ │ shell:test) │
├───────────────┼───────────────────────────┤
│ testLib │ Shared test utilities and │
│ │ mock factories │
└───────────────┴───────────────────────────┘


Packages With NO Tests

These have zero test files:

  • agentRpc — RPC communication protocol between
    dispatcher and agents
  • agentServer / api — Server infrastructure
  • chat-ui — Chat UI components
  • codeProcessor / coderWrapper — Code
    generation
  • schemaAuthor / actionSchemaCompiler — Schema
    authoring tools
  • knowledgeVisualizer / kp / textPro —
    Knowledge/text processing
  • uriHandler, interactiveApp, montage,
    cacheExplorer

Notable Gaps

  1. IPC/RPC layer (agentRpc) — No tests for the
    communication protocol between dispatcher and
    agents, which is critical infrastructure
  2. Server layer — agentServer and api have no
    coverage; server startup, routing, auth are
    untested
  3. UI components — chat-ui has no automated
    tests
  4. Schema tooling — Compiler/authoring tools
    are untested despite being part of the
    development workflow
  5. Error/edge case paths — Integration tests
    require live API keys, so CI likely skips them
    entirely, leaving a gap between unit tests and
    real behavior
  6. End-to-end flows — No apparent e2e tests
    that simulate a full user request through
    dispatcher → agent → response

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the listed untested packages, such as agentRpc, agentServer/api, chat-ui, and schema tooling, then build the project because tests run against dist/test/. Use pnpm run test:local and the relevant package entry points to establish current coverage; done means adding focused tests for an agreed gap and having the local suite pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.