python / python/mypy

JUnit report jenkins compatibility

Open
#10,102 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Bug Report

It looks like the current stable mypy-0.800-cp36-cp36m-manylinux2010_x86_64.whl does not generate a "valid" junit compatible file as jenkins treats it. I know that the junit format is not really standardised, but I think it would be cool if the file generated was jenkins compatible (or may be a little tool was provided that converts the "incorrect" format to a "correct" one).

An example of a file that fails in jenkins is

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="1" name="mypy" skips="0" tests="1" time="3.867">
  <testcase classname="mypy" file="mypy" line="1" name="mypy-py3_6-linux" time="3.867">
    <failure message="mypy produced messages">xxx.py:62: error: Incompatible types in assignment (expression has type "None", variable has type "Union[aaa, bbb]")</failure>
  </testcase>
</testsuite>

and the corresponding stacktrace:

00:01:59.084  Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.51.5.215/10.51.5.215:40124
00:01:59.084  		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
00:01:59.084  		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
00:01:59.084  		at hudson.remoting.Channel.call(Channel.java:1001)
00:01:59.084  		at hudson.FilePath.act(FilePath.java:1157)
00:01:59.084  		at hudson.FilePath.act(FilePath.java:1146)
00:01:59.084  		at org.jenkinsci.plugins.xunit.XUnitProcessor.processTestsReport(XUnitProcessor.java:183)
00:01:59.084  		at org.jenkinsci.plugins.xunit.XUnitProcessor.process(XUnitProcessor.java:157)
00:01:59.084  		at org.jenkinsci.plugins.xunit.pipeline.XUnitResultsStepExecution.run(XUnitResultsStepExecution.java:83)
00:01:59.084  		at org.jenkinsci.plugins.xunit.pipeline.XUnitResultsStepExecution.run(XUnitResultsStepExecution.java:49)
00:01:59.084  		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
00:01:59.084  		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
00:01:59.084  		at java.util.concurrent.FutureTask.run(FutureTask.java:266)
00:01:59.084  		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
00:01:59.084  		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
00:01:59.084  org.jenkinsci.plugins.xunit.service.TransformerException: The result file '/home/jenkins/agent/workspace/project/.ci/logs/mypy.xml' for the metric 'JUnit' is not valid. The result file has been skipped.
00:01:59.084  	at org.jenkinsci.plugins.xunit.service.XUnitTransformerCallable.invoke(XUnitTransformerCallable.java:111)
00:01:59.084  	at org.jenkinsci.plugins.xunit.service.XUnitTransformerCallable.invoke(XUnitTransformerCallable.java:38)
00:01:59.084  	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3313)
00:01:59.084  	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
00:01:59.084  	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
00:01:59.084  	at hudson.remoting.Request$2.run(Request.java:376)
00:01:59.084  	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
00:01:59.084  	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
00:01:59.084  	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
00:01:59.084  	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
00:01:59.084  	at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:119)
00:01:59.084  	at java.lang.Thread.run(Thread.java:748)

To Reproduce

(Write your steps here:)

  1. Generate a junit report with --junit-xml
  2. Feed it to the jenkins junit

Expected Behavior

(Write what you thought would happen.)

Actual Behavior

(Write what happened.)

Your Environment

  • Mypy version used: mypy-0.800-cp36-cp36m-manylinux2010_x86_64.whl
  • Mypy command-line flags: mypy *.py --junit-xml /path/to/report.xml
  • Mypy configuration options from mypy.ini (and other config files):
  • Python version used: 3.6.13
  • Operating system and version: docker (python:3.6-slim)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the implementation of the --junit-xml command-line option and reproduce the generated report with Jenkins JUnit processing. Compare the report and Jenkins error, then verify that the generated file is accepted by Jenkins without losing the reported mypy failure details.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.