JUnit report jenkins compatibility

Bug Report

It looks like the current stable mypy-0.800-cp36-cp36m-manylinux2010_x86_64.whl does not generate a "valid" junit compatible file as jenkins treats it. I know that the junit format is not really standardised, but I think it would be cool if the file generated was jenkins compatible (or may be a little tool was provided that converts the "incorrect" format to a "correct" one).

An example of a file that fails in jenkins is

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="1" name="mypy" skips="0" tests="1" time="3.867">
  <testcase classname="mypy" file="mypy" line="1" name="mypy-py3_6-linux" time="3.867">
    <failure message="mypy produced messages">xxx.py:62: error: Incompatible types in assignment (expression has type "None", variable has type "Union[aaa, bbb]")</failure>
  </testcase>
</testsuite>

and the corresponding stacktrace:

00:01:59.084  Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.51.5.215/10.51.5.215:40124
00:01:59.084  		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
00:01:59.084  		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
00:01:59.084  		at hudson.remoting.Channel.call(Channel.java:1001)
00:01:59.084  		at hudson.FilePath.act(FilePath.java:1157)
00:01:59.084  		at hudson.FilePath.act(FilePath.java:1146)
00:01:59.084  		at org.jenkinsci.plugins.xunit.XUnitProcessor.processTestsReport(XUnitProcessor.java:183)
00:01:59.084  		at org.jenkinsci.plugins.xunit.XUnitProcessor.process(XUnitProcessor.java:157)
00:01:59.084  		at org.jenkinsci.plugins.xunit.pipeline.XUnitResultsStepExecution.run(XUnitResultsStepExecution.java:83)
00:01:59.084  		at org.jenkinsci.plugins.xunit.pipeline.XUnitResultsStepExecution.run(XUnitResultsStepExecution.java:49)
00:01:59.084  		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
00:01:59.084  		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
00:01:59.084  		at java.util.concurrent.FutureTask.run(FutureTask.java:266)
00:01:59.084  		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
00:01:59.084  		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
00:01:59.084  org.jenkinsci.plugins.xunit.service.TransformerException: The result file '/home/jenkins/agent/workspace/project/.ci/logs/mypy.xml' for the metric 'JUnit' is not valid. The result file has been skipped.
00:01:59.084  	at org.jenkinsci.plugins.xunit.service.XUnitTransformerCallable.invoke(XUnitTransformerCallable.java:111)
00:01:59.084  	at org.jenkinsci.plugins.xunit.service.XUnitTransformerCallable.invoke(XUnitTransformerCallable.java:38)
00:01:59.084  	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3313)
00:01:59.084  	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
00:01:59.084  	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
00:01:59.084  	at hudson.remoting.Request$2.run(Request.java:376)
00:01:59.084  	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
00:01:59.084  	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
00:01:59.084  	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
00:01:59.084  	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
00:01:59.084  	at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119)
00:01:59.084  	at java.lang.Thread.run(Thread.java:748)

To Reproduce

(Write your steps here:)

1. Generate a junit report with --junit-xml
2. Feed it to the jenkins junit

Expected Behavior

(Write what you thought would happen.)

Actual Behavior

(Write what happened.)

Your Environment

• Mypy version used: mypy-0.800-cp36-cp36m-manylinux2010_x86_64.whl
• Mypy command-line flags: mypy *.py --junit-xml /path/to/report.xml
• Mypy configuration options from mypy.ini (and other config files):
• Python version used: 3.6.13
• Operating system and version: docker (python:3.6-slim)

Commencez par l’implémentation de l’option de ligne de commande --junit-xml et reproduisez le rapport généré avec le traitement JUnit de Jenkins. Comparez le rapport et l’erreur Jenkins, puis vérifiez que le fichier généré est accepté par Jenkins sans perdre les détails signalés de l’échec de mypy.