posit-dev / posit-dev/py-shiny
`Jsonifiable`'s `dict`/`list` arms are invariant, so `dict[str, int]` is not assignable to it
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 135
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 21
Description
Summary
Jsonifiable spells its container arms as List and Dict. Both are invariant in their element types, so the ordinary values user code produces — a function returning dict[str, int], or a dict[str, str] variable — are not assignable to Jsonifiable, even though every element is itself Jsonifiable.
The result is a type error at the most common call site there is, with no runtime problem behind it.
Reproducer
No third-party code; just the documented custom-renderer extension point.
from shiny.render.renderer import Renderer
from shiny.types import Jsonifiable
class render_json(Renderer[Jsonifiable]):
async def transform(self, value: Jsonifiable) -> Jsonifiable:
return value
@render_json
def scores():
return {"alice": 1, "bob": 2}
error: Argument of type "() -> dict[str, int]" cannot be assigned to parameter "_fn"
of type "(() -> Jsonifiable) | (() -> Awaitable[Jsonifiable]) | None" in function "__init__"
Function return type "dict[str, int]" is incompatible with type "Jsonifiable"
Type "dict[str, int]" is not assignable to type "Jsonifiable"
"dict[str, int]" is not assignable to "str"
"dict[str, int]" is not assignable to "int"
...
pyright 1.1.x, typeCheckingMode = "basic", shiny from main.
Why it happens
Jsonifiable = Union[str, int, float, bool, None,
List["Jsonifiable"], Tuple["Jsonifiable", ...], "JsonifiableDict"]
JsonifiableDict = Dict[str, Jsonifiable]
dict and list are mutable, so they must be invariant — the element type has to match exactly, not merely be compatible. That invariance is correct and load-bearing in general:
counts: dict[str, int] = {"a": 1}
wide: dict[str, Jsonifiable] = counts # if this were allowed...
wide["b"] = "not an int" # legal, str IS Jsonifiable
counts["b"] + 1 # TypeError at runtime
But Jsonifiable is used almost exclusively as an input type — a value Shiny reads and serializes, never writes into. The soundness that invariance buys is not needed there, and the cost is that the annotation rejects what callers actually write.
Why it is easy to miss
It only bites when the value type is inferred independently of the target:
def wants(v: Jsonifiable) -> None: ...
wants({"a": 1}) # OK -- literal, inferred bidirectionally against the param type
wants(f()) # ERROR -- where `def f() -> dict[str, int]`
A dict literal passed straight as an argument is checked against the parameter and infers dict[str, Jsonifiable]. A function's return type is inferred on its own and only then compared. So APIs usually called with a literal (send_custom_message(..., {"a": 1})) look fine, while anything taking a user's function — every Renderer — breaks at every call site.
Suggested fix
Spell the containers with the covariant read-only protocols:
Jsonifiable = Union[str, int, float, bool, None,
Sequence["Jsonifiable"], Mapping[str, "Jsonifiable"]]
Mapping has no __setitem__, so the unsoundness above is structurally impossible and covariance is safe: dict[str, int] is a Mapping[str, Jsonifiable]. Same for list[int] → Sequence[Jsonifiable]. Tuple[Jsonifiable, ...] is already covered by Sequence.
Caveat worth weighing: str is itself a Sequence[str], and the internal sites that construct or mutate a Jsonifiable (JsonifiableDict returns in render/_data_frame*.py, rendered_deps_to_jsonifiable, …) would need the concrete dict/list types or a cast. A narrower alternative is to leave Jsonifiable alone and introduce a covariant sibling for the input direction only — keeping Jsonifiable as the type on the way out to the wire.
Context
Found in posit-dev/shinyreact, whose reactive_output is a Renderer[Jsonifiable] publishing raw JSON to a React client. Returning a dict is the single most common thing it does, and it type-errored in every example app in the repo. We worked around it locally with a covariant alias, then reverted in favor of fixing it here — a user's only workaround today is annotating their own render functions -> Jsonifiable, which is exactly the noise the alias exists to avoid.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the Jsonifiable and JsonifiableDict definitions, then inspect Renderer and the internal construction sites named in render/_data_frame*.py and rendered_deps_to_jsonifiable. Reproduce the reported pyright error with the custom renderer example, evaluate the suggested read-only container types or input-only alias, and confirm that existing mutation sites and the function return type-check correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100