Bug Report: stepByStepReport not capturing screenshots in CodeceptJS 3.6+

lib/plugin/stepByStepReport.js から始め、提供された最小限の CodeceptJS 設定をバージョン 3.7.5 で使用して問題を再現します。event.step.after ハンドラーと recorder の使用箇所を調べ、その後、失敗するテストによって通常実行と並列実行の両方で output/record*/ に期待される PNG スクリーンショットが生成されることを確認します。

Hello,

The stepByStepReport plugin does not capture screenshots after each step in CodeceptJS versions 3.6.0 through 3.7.5. The record_*/ directory is created but only contains an empty index.html file with no screenshots (PNG files).

Affected Versions

• Broken: CodeceptJS 3.6.0, 3.6.1, 3.6.2, 3.6.3, 3.6.4, 3.7.0, 3.7.1, 3.7.2, 3.7.3, 3.7.4, 3.7.5
• Working: CodeceptJS 3.5.0 and earlier

Root Cause

The bug was introduced in commit 1040494 (Jun 8, 2024) titled "fix: screenshot error in beforeSuite/AfterSuite (#4385)".

The Change

In lib/plugin/stepByStepReport.js, the event.step.after handler was changed:

Before (working - v3.5.0):

event.dispatcher.on(event.step.after, (step) => {
  recorder.add('screenshot of failed test', async () => persistStep(step), true);
});

After (broken - v3.6+):

event.dispatcher.on(event.step.after, persistStep);

Why This Breaks

The persistStep function is an async function that calls helper.saveScreenshot(). In the original implementation, this was wrapped in recorder.add(), which properly queues the async operation and waits for completion.

In the new implementation, persistStep is called directly as an event handler callback. Since it's an async function, the event dispatcher doesn't wait for it to complete. The screenshots are never captured because:

1. The async function starts executing
2. The event dispatcher immediately moves on to the next event
3. The screenshot operation may be cancelled or never completes

Reproduction Steps

1. Create a minimal CodeceptJS config with only stepByStepReport enabled
2. Run any test that fails (to keep the report)
3. Check the _output/record_*/ directory
4. The directory contains only index.html with no PNG files

Minimal Reproduction

// codecept.minimal.conf.js
const path = require('path');

exports.config = {
  output: path.resolve(__dirname, '_output'),
  helpers: {
    Playwright: {
      browser: 'chromium',
      url: 'https://www.google.com',
      show: true
    }
  },
  plugins: {
    stepByStepReport: {
      enabled: true
    }
  },
  tests: './test_minimal.js',
  name: 'minimal-test'
};

// test_minimal.js
Feature('Minimal Test');

Scenario('Test stepByStepReport', ({ I }) => {
  I.amOnPage('https://www.google.com');
  I.wait(2);
  I.see('Google');
  I.see('TEXT THAT DOES NOT EXIST'); // Force failure to keep report
});

Run with:

npx codeceptjs@3.7.5 run --config codecept.minimal.conf.js

Expected: _output/record_*/ contains 0000.png, 0001.png, etc.
Actual: _output/record_*/ contains only index.html with no PNGs

Suggested Fix

Restore the original behavior by wrapping persistStep in recorder.add():

event.dispatcher.on(event.step.after, step => {
  recorder.add('screenshot of step of test', async () => persistStep(step), true);
});

This ensures the async screenshot operation is properly queued and executed.

Additional Context

• This issue has been reported before as #4856 (Feb 2025) and is a follow-up of #4637
• The original fix was intended to prevent errors in BeforeSuite/AfterSuite hooks, but it inadvertently broke the core screenshot functionality
• The issue affects both single-run and parallel execution modes

Environment

• CodeceptJS: 3.7.5
• Node.js: 20.x
• Playwright: 1.57.0
• OS: Linux (Ubuntu 22.04)

Written with help from Claude 4.5 Opus.