Rewrite .test as .py

Many of us are using editors that handle Python code (syntax highlighting, symbol browsing). Would it be a good idea to restructure the .test data-files as Python files?

I've been musing about it for a while, so hear me out, then let me know if it's a stupid idea :)

Example:

-[case testFoo]
-s1: str = 42  # E: Incompatible types in assignment (expression has type "int", variable has type "str")
+def test_foo():
+  s1: str = 42  # E: Incompatible types in assignment (expression has type "int", variable has type "str")

To clarify, this wouldn't be true Python code, i.e. it won't be fed directly to CPython or Mypy; rather, it'll be a Python-flavored data file — perhaps parsed by AST (if feasible and performant), but where the function bodies are passed to the current machinery as text.

Extra qualifies like # flags: --strict-optional could be represented as decorators, e.g.

@flags('--strict-optional')
def test_foo():
   ...

[case testFoo-xfail] would become:

@pytest.mark.xfail
def test_foo():
   ...

[out] would become:

@expected_output("""
main:1: Incompatible types in assignment (expression has type "int", variable has type "str")
""")
def test_foo():
   ...

[file a.py] would become:

@testcase
def test_foo():
   ...  # main goes here


@test_foo.file('a.py')
def test_foo():
   ...  # a.py goes here

Benefits:

• First-class support in IDEs, e.g.
  
  
• The disarray of testcase options, sections and modifiers could be formalized in a Pythonic syntax and a module that would document them, e.g. from mypy.testcase import expected_output and then

  # mypy/testcase.py
  
  def expected_output(output: str) -> Testcase:
      """Documentation goes here"""
      ... 
  

• Can use some Python tooling like Black
• A more flexible representation than INI, while providing first-class editor experience (unlike, say, converting to YAML and putting the Python test cases into a multiline YAML block).
• More natural for newcomers?

Cons:

• Harder to implement?
• Confusing that it's Python but not really?
• Less natural to newcomers?

既存の .test データファイル形式と、testcase bodies を消費する現在の仕組みから始め、提案されている Python-flavored files を AST でパースする方法と比較してください。実現可能性、オプション・セクション・修飾子の構文、および既存のテスト動作を形式が維持できるかどうかについて判断できれば、有用な成果になります。