JUnit report jenkins compatibility

Bug Report

It looks like the current stable mypy-0.800-cp36-cp36m-manylinux2010_x86_64.whl does not generate a "valid" junit compatible file as jenkins treats it. I know that the junit format is not really standardised, but I think it would be cool if the file generated was jenkins compatible (or may be a little tool was provided that converts the "incorrect" format to a "correct" one).

An example of a file that fails in jenkins is

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="1" name="mypy" skips="0" tests="1" time="3.867">
  <testcase classname="mypy" file="mypy" line="1" name="mypy-py3_6-linux" time="3.867">
    <failure message="mypy produced messages">xxx.py:62: error: Incompatible types in assignment (expression has type "None", variable has type "Union[aaa, bbb]")</failure>
  </testcase>
</testsuite>

and the corresponding stacktrace:

00:01:59.084  Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.51.5.215/10.51.5.215:40124
00:01:59.084  		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
00:01:59.084  		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
00:01:59.084  		at hudson.remoting.Channel.call(Channel.java:1001)
00:01:59.084  		at hudson.FilePath.act(FilePath.java:1157)
00:01:59.084  		at hudson.FilePath.act(FilePath.java:1146)
00:01:59.084  		at org.jenkinsci.plugins.xunit.XUnitProcessor.processTestsReport(XUnitProcessor.java:183)
00:01:59.084  		at org.jenkinsci.plugins.xunit.XUnitProcessor.process(XUnitProcessor.java:157)
00:01:59.084  		at org.jenkinsci.plugins.xunit.pipeline.XUnitResultsStepExecution.run(XUnitResultsStepExecution.java:83)
00:01:59.084  		at org.jenkinsci.plugins.xunit.pipeline.XUnitResultsStepExecution.run(XUnitResultsStepExecution.java:49)
00:01:59.084  		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
00:01:59.084  		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
00:01:59.084  		at java.util.concurrent.FutureTask.run(FutureTask.java:266)
00:01:59.084  		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
00:01:59.084  		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
00:01:59.084  org.jenkinsci.plugins.xunit.service.TransformerException: The result file '/home/jenkins/agent/workspace/project/.ci/logs/mypy.xml' for the metric 'JUnit' is not valid. The result file has been skipped.
00:01:59.084  	at org.jenkinsci.plugins.xunit.service.XUnitTransformerCallable.invoke(XUnitTransformerCallable.java:111)
00:01:59.084  	at org.jenkinsci.plugins.xunit.service.XUnitTransformerCallable.invoke(XUnitTransformerCallable.java:38)
00:01:59.084  	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3313)
00:01:59.084  	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
00:01:59.084  	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
00:01:59.084  	at hudson.remoting.Request$2.run(Request.java:376)
00:01:59.084  	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
00:01:59.084  	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
00:01:59.084  	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
00:01:59.084  	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
00:01:59.084  	at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119)
00:01:59.084  	at java.lang.Thread.run(Thread.java:748)

To Reproduce

(Write your steps here:)

1. Generate a junit report with --junit-xml
2. Feed it to the jenkins junit

Expected Behavior

(Write what you thought would happen.)

Actual Behavior

(Write what happened.)

Your Environment

• Mypy version used: mypy-0.800-cp36-cp36m-manylinux2010_x86_64.whl
• Mypy command-line flags: mypy *.py --junit-xml /path/to/report.xml
• Mypy configuration options from mypy.ini (and other config files):
• Python version used: 3.6.13
• Operating system and version: docker (python:3.6-slim)

Inizia dall’implementazione dell’opzione da riga di comando --junit-xml e riproduci il report generato con l’elaborazione JUnit di Jenkins. Confronta il report e l’errore di Jenkins, quindi verifica che il file generato venga accettato da Jenkins senza perdere i dettagli dell’errore mypy riportato.