linkedin / linkedin/pygradle

PyTest and Flake8 outputs not always visible in console

Open
#203 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
603
Forks
142
PR merge metrics
No merged PRs in 30d

Description

**Problem**
Sometimes the `pytest` or `flake8` tasks fail with no output visible in the console, making it very hard to investigate issues.

**Repro Steps**

1. From the example-project provided in this repo, I added a few dummy tests
2. Run `gradle pytest --info`
3. Observe output, it will inconsistently return one or the other:

Expected:

```
> Task :pytest
Task ':pytest' is not up-to-date because:
Task has not declared any outputs.
Starting process 'command 'C:\hack\example-project\build\venv\Scripts\python.exe''. Working directory: C:\hack\example-project Command: C:\hack\example-project\build\venv\Scripts\python.exe C:\hack\example-project\build\venv\Scripts\py.test.exe C:\hack\example-project\test
Successfully started process 'command 'C:\hack\example-project\build\venv\Scripts\python.exe''
============================= test session starts =============================
platform win32 -- Python 2.7.2, pytest-3.1.2, py-1.4.34, pluggy-0.4.0
rootdir: C:\hack\example-project, inifile:
plugins: xdist-1.17.1, cov-2.5.1
collected 2 items

test\test_example.py ..

========================== 2 passed in 0.04 seconds ===========================

:pytest (Thread[Task worker for ':',5,main]) completed. Took 0.745 secs.

```
Actual:

```
Task ':pytest' is not up-to-date because:
Task has not declared any outputs.
Starting process 'command 'C:\hack\example-project\build\venv\Scripts\python.exe''. Working directory: C:\hack\example-project Command: C:\hack\example-project\build\venv\Scripts\python.exe C:\hack\example-project\build\venv\Scripts\py.test.exe C:\hack\example-project\test
Successfully started process 'command 'C:\hack\example-project\build\venv\Scripts\python.exe''
:pytest (Thread[Task worker for ':',5,main]) completed. Took 0.773 secs.
```

**Additionnal Info**
The issue is probably that the buffered output is not flushed before the process ends. I have tried to work around the problem on our side by running the pytest task with PYTHONUNBUFFERED (-u) but didn't see any improvement.

Then, as I was investigating the issue, I've been able to get stdout to print consistently with this sample task:

```
task myTestTask(type:Exec) {
workingDir = project.rootDir
if(os.isWindows()) {
commandLine "${project.buildDir}\\venv\\Scripts\\python.exe", "${project.buildDir}\\venv\\Scripts\\py.test.exe", "${project.rootDir}\\test"
}
else {
commandLine "${project.buildDir}/venv/bin/python", "${project.buildDir}/venv/bin/py.test", "${project.rootDir}/test"
}
standardOutput = new ByteArrayOutputStream();
doLast {
logger.info(standardOutput.toString())
}
}
```
Capturing stdout and logging it explicitly in a doLast block was key to success and I'm hoping you guys could fix the `pytest` and `flake8` tasks similarly within PyGradle.

Note that you would need to fix it for both stdout and stderr.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the PyGradle task definitions for pytest and flake8, then reproduce the issue from the example-project with `gradle pytest --info`. Verify both stdout and stderr remain visible consistently for successful and failing runs, including the Windows command shown in the report.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.