pytest filter drops the duration line, so a 3-second run and a 10-minute run look identical

Open Beginner friendly
#4,071 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python, rust
Domain
cli, testing

Research direction

Start by locating the Rust pytest filter implementation and its inline filter tests, then trace how the pytest summary is reduced to the current pass/fail line. Preserve the elapsed duration for both passing and failing summaries, and verify the output matches the examples in this issue.

Written by the indexing model from the issue text.

Description

area:cli bug good first issue priority:medium

rtk 0.49.0, Homebrew, macOS arm64 (Darwin 27.0.0), Python 3.11.8, pytest 7.4.0.

What happens

rtk pytest drops pytest's summary line, so the elapsed time is gone from the output.

$ cat test_slow.py
import time

def test_fast():
    assert 1 == 1

def test_slow():
    time.sleep(3)
    assert 2 == 2

$ rtk proxy pytest test_slow.py | tail -3
test_slow.py ..                                                          [100%]

============================== 2 passed in 3.04s ===============================

$ rtk pytest test_slow.py
Pytest: 2 passed

Same on the failing path:

$ rtk pytest test_fail.py
Pytest: 0 passed, 1 failed

Failures:
1. [FAIL] test_boom
     test_fail.py:4: in test_boom
     assert 1 == 2
     E   assert 1 == 2

No duration anywhere.

Why this one hurts

I run pytest through the Claude Code hook, so the model only ever sees the filtered output. A tool result carries no timing of its own, the model cannot tell a call that took 0.2s from one that took 10 minutes. Pytest's summary line was the only place that number existed, and the filter removes it.

The practical effect is that a suite which quietly goes from 12s to 10 minutes reads exactly the same as before. Nothing in the agent's context says anything changed, so nobody looks into it.

Other filters already keep it

This looks like an oversight in the pytest filter, not a project-wide policy. From the inline filter tests in the 0.49.0 binary:

  • gradle: expected = "BUILD SUCCESSFUL in 8s\n7 actionable tasks: 7 executed"
  • gradle (failing): expected = "> Task :app:test\n3 tests completed, 1 failed\nBUILD FAILED in 12s"
  • cargo: expected = " Finished release [optimized] target(s) in 45.23s\n Binary: target/release/my-crate (5.2MB)"

and vitest parses Duration\s+([\d.]+)(ms|s), ctest parses Total Test time (real) = ([\d.]+) sec, dotnet captures a Duration: group.

Suggestion

Put the seconds back on the summary line:

Pytest: 2 passed in 3.04s
Pytest: 0 passed, 1 failed in 1.02s

That is roughly 3 tokens, and it is the only timing signal an agent gets for the whole run.

Workaround does not exist today

A project-local .rtk/filters.toml cannot patch this, since pytest is routed to the Rust module. Trusted and confirmed applied, output unchanged:

schema_version = 1

[filters.pytest-keep-duration]
description = "keep pytest duration line"
match_command = "pytest"
strip_ansi = true
keep_lines_matching = ["passed in", "failed", "error"]
max_lines = 20
$ rtk trust -y
    pytest-keep-duration pytest
  Enabled — revoke with `rtk untrust`.
Filters will now be applied.

$ rtk pytest test_slow.py
Pytest: 2 passed

That part is #3904, so I am not asking for it here. Just the duration line.

Dominant language
Rust
Stars
81.1k
Forks
5.1k
Avg merge
4d 11h
Merged PRs (30d)
40

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from rtk-ai/rtk

All issues in rtk-ai/rtk

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.