get-convex / get-convex/convex-test

args get validated but not function returns

Open
#71 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
22
Forks
12
Avg merge
4d 9h
Merged PRs (30d)
12

Description

convex-test appears to validate function arguments, but not function return values against returns validators.
This causes tests to pass while real Convex runtime rejects the same function result.

Repro scenario

A query with a strict returns validator:

export const queryRoles = query({
  args: { search: v.optional(v.string()) },
  returns: v.array(
    v.object({
      _id: v.id("roles"),
      name: v.string(),
      permissions: v.array(permissionValidator),
    }),
  ),
  handler: async (ctx) => {
    // raw documents include _creationTime
    return await ctx.db.query("roles").collect();
  },
});

In real runtime, this fails return validation because _creationTime is extra.
In convex-test, this passes.

Source findings

In index.ts:

  • queryFromPath validates only exportArgs:
    • validateValidator(JSON.parse((func as any).exportArgs()), args ?? {});
    • around line ~1966
  • mutationFromPath and actionFromPath do the same for args only:
    • around ~1982 and ~1995
  • Query execution then goes through runQueryWithHandler (around ~1895), which wraps the function with:
    • queryGeneric({ handler: ... })
    • but does not apply func.exportReturns() from the original function definition.

Because the original function definition (with its returns) is not passed through, return validation is effectively skipped in tests.

Expected behavior

convex-test should enforce returns validators similarly to runtime, or clearly document that returns are not validated.

Impact

  • False positives in tests
  • Runtime-only failures
  • Contract drift between test and production behavior

Request

  • Validate returned values against exportReturns()

Environment

  • convex-test: 0.0.41
  • convex: 1.32.0
  • Bun + Vitest setup

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in index.ts at runQueryWithHandler and the queryFromPath, mutationFromPath, and actionFromPath entry points identified in the issue. Trace how exportArgs is validated and how exportReturns() could remain associated with the original function definition. Done means convex-test rejects the extra _creationTime in the repro result, matching runtime return-validation behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.