posit-dev / posit-dev/py-shiny

`Jsonifiable`'s `dict`/`list` arms are invariant, so `dict[str, int]` is not assignable to it

Open
#2,497 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ai-triage:done Priority: Medium
Dominant language
Python
Stars
1.8k
Forks
135
Avg merge
2d 18h
Merged PRs (30d)
21

Description

Summary

Jsonifiable spells its container arms as List and Dict. Both are invariant in their element types, so the ordinary values user code produces — a function returning dict[str, int], or a dict[str, str] variable — are not assignable to Jsonifiable, even though every element is itself Jsonifiable.

The result is a type error at the most common call site there is, with no runtime problem behind it.

Reproducer

No third-party code; just the documented custom-renderer extension point.

from shiny.render.renderer import Renderer
from shiny.types import Jsonifiable


class render_json(Renderer[Jsonifiable]):
    async def transform(self, value: Jsonifiable) -> Jsonifiable:
        return value


@render_json
def scores():
    return {"alice": 1, "bob": 2}
error: Argument of type "() -> dict[str, int]" cannot be assigned to parameter "_fn"
  of type "(() -> Jsonifiable) | (() -> Awaitable[Jsonifiable]) | None" in function "__init__"
    Function return type "dict[str, int]" is incompatible with type "Jsonifiable"
      Type "dict[str, int]" is not assignable to type "Jsonifiable"
        "dict[str, int]" is not assignable to "str"
        "dict[str, int]" is not assignable to "int"
        ...

pyright 1.1.x, typeCheckingMode = "basic", shiny from main.

Why it happens
Jsonifiable = Union[str, int, float, bool, None,
                    List["Jsonifiable"], Tuple["Jsonifiable", ...], "JsonifiableDict"]
JsonifiableDict = Dict[str, Jsonifiable]

dict and list are mutable, so they must be invariant — the element type has to match exactly, not merely be compatible. That invariance is correct and load-bearing in general:

counts: dict[str, int] = {"a": 1}
wide: dict[str, Jsonifiable] = counts   # if this were allowed...
wide["b"] = "not an int"                # legal, str IS Jsonifiable
counts["b"] + 1                         # TypeError at runtime

But Jsonifiable is used almost exclusively as an input type — a value Shiny reads and serializes, never writes into. The soundness that invariance buys is not needed there, and the cost is that the annotation rejects what callers actually write.

Why it is easy to miss

It only bites when the value type is inferred independently of the target:

def wants(v: Jsonifiable) -> None: ...

wants({"a": 1})   # OK    -- literal, inferred bidirectionally against the param type
wants(f())        # ERROR -- where `def f() -> dict[str, int]`

A dict literal passed straight as an argument is checked against the parameter and infers dict[str, Jsonifiable]. A function's return type is inferred on its own and only then compared. So APIs usually called with a literal (send_custom_message(..., {"a": 1})) look fine, while anything taking a user's function — every Renderer — breaks at every call site.

Suggested fix

Spell the containers with the covariant read-only protocols:

Jsonifiable = Union[str, int, float, bool, None,
                    Sequence["Jsonifiable"], Mapping[str, "Jsonifiable"]]

Mapping has no __setitem__, so the unsoundness above is structurally impossible and covariance is safe: dict[str, int] is a Mapping[str, Jsonifiable]. Same for list[int]Sequence[Jsonifiable]. Tuple[Jsonifiable, ...] is already covered by Sequence.

Caveat worth weighing: str is itself a Sequence[str], and the internal sites that construct or mutate a Jsonifiable (JsonifiableDict returns in render/_data_frame*.py, rendered_deps_to_jsonifiable, …) would need the concrete dict/list types or a cast. A narrower alternative is to leave Jsonifiable alone and introduce a covariant sibling for the input direction only — keeping Jsonifiable as the type on the way out to the wire.

Context

Found in posit-dev/shinyreact, whose reactive_output is a Renderer[Jsonifiable] publishing raw JSON to a React client. Returning a dict is the single most common thing it does, and it type-errored in every example app in the repo. We worked around it locally with a covariant alias, then reverted in favor of fixing it here — a user's only workaround today is annotating their own render functions -> Jsonifiable, which is exactly the noise the alias exists to avoid.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in the Jsonifiable and JsonifiableDict definitions, then inspect Renderer and the internal construction sites named in render/_data_frame*.py and rendered_deps_to_jsonifiable. Reproduce the reported pyright error with the custom renderer example, evaluate the suggested read-only container types or input-only alias, and confirm that existing mutation sites and the function return type-check correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.