allure-framework / allure-framework/allure3

RetrySubstore.upsert appends a duplicate entry when the same result id is read twice

Open Beginner friendly
#896 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
401
Forks
58
Avg merge
2d 20h
Merged PRs (30d)
34

Description

## What happened?

When the store reads two result files that resolve to the same test result id (same `uuid`, so the same `md5(uuid)` store id), the test ends up listed as its own retry.

The store itself handles a re-read of the same id correctly: `DefaultAllureStore.visitTestResult` does `#testResults.set(testResult.id, testResult)` (`packages/core/src/store/store.ts`), and a `Map.set` with the same key replaces the old entry.

`RetrySubstore.upsert` (`packages/core/src/store/retrySubstore.ts`) does not follow the same rule. It keeps a plain array per `retryHash` and always pushes:

```ts
upsert(testResult: TestResult) {
testResult.isRetry = false;

if (!testResult.retryHash) {
return;
}

const results = this.#testResultsByRetryHash.get(testResult.retryHash);

if (!results) {
this.#testResultsByRetryHash.set(testResult.retryHash, [testResult]);
return;
}

results.push(testResult); // <- no check for an existing entry with the same id
results.sort((first, second) => this.#compareResults(first, second));

results.forEach((attempt, index) => {
attempt.isRetry = index !== 0;
});
}
```

If `visitTestResult` runs twice for the same id (two files on disk that share a `uuid`), `upsert` is called twice with two different `TestResult` object instances that happen to carry the same `id`. Both land in the `results` array, so `retriesByTr` reports the test as having one retry — itself, from the earlier read.

This isn't a contrived edge case tied to one tool. The reader walks a directory and calls `visitTestResult` once per **file**, not once per `uuid` — nothing before `RetrySubstore` treats `uuid` as a de-dup key except the `#testResults` Map. Two files that legitimately describe the same attempt (e.g. a directory that gets re-read, or a result file that's rewritten to the same `uuid` as it's produced) hit this.

Because `allure watch` never evicts old entries, in a long-running watch session a file that keeps getting re-read this way grows the `results` array by one stale entry every time — the retries list for that test keeps growing.

## How can we reproduce it?

This is self-contained: `npm install allure`, two JSON files that share a `uuid`, one CLI command. No test framework involved.

```bash
mkdir allure-retry-dup-repro && cd allure-retry-dup-repro
npm init -y
npm install allure
mkdir allure-results
```

`write-results.mjs` — writes two result files for the same `uuid`, as if the same attempt had been read twice:

```js
import { writeFileSync } from "node:fs";

const uuid = "11111111-1111-1111-1111-111111111111";
const now = Date.now();

const base = {
uuid,
name: "a test",
fullName: "a test",
status: "passed",
stage: "finished",
steps: [],
attachments: [],
parameters: [],
labels: [],
links: [],
};

writeFileSync("allure-results/first-result.json", JSON.stringify({ ...base, start: now, stop: now + 1 }, null, 2));
writeFileSync("allure-results/second-result.json", JSON.stringify({ ...base, start: now + 1000, stop: now + 1001 }, null, 2));
```

```bash
node write-results.mjs
npx allure agent inspect --output .agent-output allure-results
cat .agent-output/manifest/tests.jsonl
```

Actual output (`tests.jsonl`, trimmed to the relevant field):

```json
{"full_name":"a test", ... "retries":1, ...}
```

`retries: 1` for a test that was only ever executed once — the two files describe the same attempt, not two attempts.

## What did you expect?

`retries: 0`. Two files that resolve to the same result id should still be treated as one logical result — the same way `#testResults` already treats them — not as one primary result plus a retry of itself.

## Environment

- Allure 3 version: 3.16.0 (also the latest version on npm at the time of writing; confirmed with `npm view allure version`)
- Node.js version: v26.7.0
- Package manager and version: npm 11.x (bundled with the Node.js version above)
- OS: macOS, arm64

## Additional context

I searched for an existing issue describing this and didn't find one that matches. #713 has a similar-sounding title ("Incorrectly Displays Passed Test Executions as Retries"), but its mechanism is different — it's about separate `mvn test` runs across a persisted `historyPath`/`appendHistory` history, each with its own `uuid`, not a single read where two files share one `uuid`. Flagging it here only so it isn't mistaken for a duplicate of this one.

I have a fix ready (`RetrySubstore.upsert` replaces an existing entry with the same `id` instead of pushing a duplicate, matching the `#testResults` Map's existing replace-by-id behavior) — happy to open a PR.

Contributor guide

Open the contributing guide

Research direction

Start in packages/core/src/store/retrySubstore.ts, focusing on RetrySubstore.upsert, and compare its handling of repeated result ids with DefaultAllureStore.visitTestResult in packages/core/src/store/store.ts. Run the two-file reproduction with npx allure agent inspect and verify that the generated tests.jsonl reports retries: 0 for the shared uuid.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
85/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.