Conformance tests: Align automated and manual scoring
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 1.8k
- 派生
- 302
- 平均合并
- 23 小时
- 30 天内合并 PR
- 8
描述
In #1678 I added an automated tool for scoring the conformance test suite, and #1682 and #1683 added some features to support edge cases.
Now I am working on going through the remaining files where the automated conformance judgment on a file differs from the manual one (open PRs: #1684, #1685, #1687, #1688, #1689, #1690, #1691).
I'll use this issue to track the remaining discrepancies (from conformance/src/unexpected_fails.py):
- mypy/dataclasses_usage.toml: Pass vs. Fail (#1684)
- mypy/directives_version_platform.toml: Pass vs. Fail (#1685)
- mypy/namedtuples_define_functional.toml: Pass vs. Fail
- mypy/overloads_basic.toml: Pass vs. Fail (#1687)
- mypy/specialtypes_never.toml: Pass vs. Fail (#1688)
- mypy/typeddicts_alt_syntax.toml: Pass vs. Fail (#1689)
- mypy/typeddicts_type_consistency.toml: Pass vs. Fail (#1690)
- pyre/directives_reveal_type.toml: Unsupported vs. Pass
- pyre/namedtuples_define_functional.toml: Pass vs. Fail
- pyright/classes_override.toml: Pass vs. Fail (#1691)
- pyright/dataclasses_frozen.toml: Pass vs. Fail (#1699)
- pyright/dataclasses_slots.toml: Pass vs. Fail (#1699)
- pyright/dataclasses_transform_field.toml: Pass vs. Fail (#1699)
- pyright/dataclasses_transform_func.toml: Pass vs. Fail (#1699)
- pyright/generics_defaults.toml: Pass vs. Fail (#1700)
- pyright/literals_literalstring.toml: Pass vs. Fail (#1701)
- pyright/overloads_basic.toml: Pass vs. Fail (#1687)
- pyright/protocols_variance.toml: Pass vs. Fail (#1702)
- pyright/specialtypes_never.toml: Pass vs. Fail (#1688)
- pyright/tuples_unpacked.toml: Pass vs. Fail (#1703)
- pytype/dataclasses_usage.toml: Pass vs. Fail (#1684)
- pytype/directives_assert_type.toml: Pass vs. Fail (#1706)
- pytype/directives_version_platform.toml: Pass vs. Fail (#1685)
- pytype/generics_scoping.toml: Pass vs. Fail (#1704)
- pytype/typeddicts_inheritance.toml: Pass vs. Fail (#1705)
Any help is appreciated.
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
从 conformance/src/unexpected_fails.py 开始,检查 mypy/namedtuples_define_functional.toml、pyre/directives_reveal_type.toml 和 pyre/namedtuples_define_functional.toml 中剩余的未勾选差异。将每个自动化结果与其手动判断进行比较,并协调剩余案例。完成的标准是列出的三个差异已解决,并且 conformance 评分与手动结果一致。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python
- 领域
- testing-qa
- Issue 类型
- 缺陷
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100