Within Annotated[T, x], type errors in x are completely ignored

Bug Report

mypy ignores the x expression in Annotated[T, x] without type-checking it. Although x has no effect on the typing interpretation of T, the expression x is still code that’s going to be executed at runtime*, so mypy should type-check x (as a separate object) before ignoring it.

(* Although it might not be executed immediately if from __future__ import annotations is in effect, it will be executed when typing.get_type_hints is used, so it’s still important for mypy to check it.)

An example of a real subtle bug that was missed because of this is zulip/zulip#26710.

To Reproduce

from typing import Annotated

def f(p: Annotated[int, 2 + "oops"]) -> None:
    pass

mypy Playground, Pyright playground

Expected Behavior

I expect a type error at 2 + "oops". (Pyright agrees.)

Actual Behavior

mypy finds no errors.

Your Environment

• Mypy version used: 1.5.1
• Mypy command-line flags: --strict
• Mypy configuration options from mypy.ini (and other config files): none
• Python version used: 3.11.4

Start by reproducing the Annotated[int, 2 + "oops"] example under Python 3.11 with --strict, then trace mypy's handling of Annotated metadata. Done means mypy reports a type error for the metadata expression while preserving the typing interpretation of int.