CircleCI-Public / CircleCI-Public/circleci-cli

Test splitting fails with "invalid timing type filename" despite docs stating it's valid

Open
#1,150 3 comments 2 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
459
Forks
255
Avg merge
1d 4h
Merged PRs (30d)
59

Description

## Meta:

**CircleCI CLI Version:**
```
$ circleci version

Build Agent version: 1.0.282855-5d940a9a
Built: 2025-06-25T14:29:19+0000
```

**CircleCI Tests Plugin CLI Version:**
```
Installing circleci-tests-plugin-cli plugin.
circleci-tests-plugin-cli plugin Installed. Version: 1.0.8767-cce2816
```

**Operating System:**
```
$ uname -a

Linux db1d44f44033 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 GNU/Linux
```
**CircleCI CLI Diagnostic:**
```
$ circleci diagnostic

Error: unknown command "diagnostic" for "circleci"
Run 'circleci --help' for usage.
ERRO[0000] Exit with status 1 due to generic error error="unknown command \"diagnostic\" for \"circleci\""

Exited with code exit status 1
```

---
## Current behavior:
Running the following job step in CircleCI:

```
- run:
name: "Run RSpec Parallel"
command: |
circleci tests glob "./spec/**/*_spec.rb" | circleci tests run --command="xargs bundle exec rspec --format progress -r rspec_junit_formatter --format RspecJunitFormatter -o /tmp/repo/test_results/rspec/junit.xml" --verbose --split-by=timings --timings-type=filename
no_output_timeout: 20m
```

...now results in this error:

```
Installing circleci-tests-plugin-cli plugin.
circleci-tests-plugin-cli plugin Installed. Version: 1.0.8767-cce2816
DEBU[2025-06-25T17:16:49Z] Attempting to read from stdin. This will hang if no input is provided.
DEBU[2025-06-25T17:16:49Z] There was an error when creating the sorter: invalid timing type filename
Error: invalid timing type filename
ERRO[2025-06-25T17:16:49Z] invalid timing type filename
Error: exit status 1

Exited with code exit status 1
```

## Expected behavior:
The command should accept --timings-type=filename as a valid value.
This used to work reliably in our CI pipeline, and the CircleCI documentation still mentions filename as a supported option under test splitting.

## When did this begin / Was this previously working?:
This began failing on or around June 25th, 2025.
Previously, this exact job and configuration had been running successfully for months.

We're using the latest auto-installed version of the `circleci-tests-plugin-cli`:
```
Version: 1.0.8767-cce2816
```

And CircleCI CLI version:
```
circleci version
0.1.27921+abcdef
```

## Additional Information:

Plugin seems to now validate --timings-type strictly and rejects filename.

If filename has been deprecated, could you please:

Confirm the officially supported values (name? classname?).

Update the official documentation to reflect this change.

Consider providing a clearer changelog or migration path for those affected.

Thanks!

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported `circleci tests run` command with `--split-by=timings --timings-type=filename` and the listed plugin version. Trace where the plugin validates timing types, then verify the supported values and documentation agree; done means the documented configuration behaves consistently and the command no longer rejects a valid value.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
ci-cd, cli, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.