Bug Report: stepByStepReport not capturing screenshots in CodeceptJS 3.6+

从 lib/plugin/stepByStepReport.js 开始，在 3.7.5 版本上使用提供的最小 CodeceptJS 配置重现该问题。检查 event.step.after 处理程序和 recorder 的使用情况，然后验证失败的测试是否会在普通运行和并行运行中都在 output/record*/ 生成预期的 PNG 屏幕截图。

Hello,

The stepByStepReport plugin does not capture screenshots after each step in CodeceptJS versions 3.6.0 through 3.7.5. The record_*/ directory is created but only contains an empty index.html file with no screenshots (PNG files).

Affected Versions

• Broken: CodeceptJS 3.6.0, 3.6.1, 3.6.2, 3.6.3, 3.6.4, 3.7.0, 3.7.1, 3.7.2, 3.7.3, 3.7.4, 3.7.5
• Working: CodeceptJS 3.5.0 and earlier

Root Cause

The bug was introduced in commit 1040494 (Jun 8, 2024) titled "fix: screenshot error in beforeSuite/AfterSuite (#4385)".

The Change

In lib/plugin/stepByStepReport.js, the event.step.after handler was changed:

Before (working - v3.5.0):

event.dispatcher.on(event.step.after, (step) => {
  recorder.add('screenshot of failed test', async () => persistStep(step), true);
});

After (broken - v3.6+):

event.dispatcher.on(event.step.after, persistStep);

Why This Breaks

The persistStep function is an async function that calls helper.saveScreenshot(). In the original implementation, this was wrapped in recorder.add(), which properly queues the async operation and waits for completion.

In the new implementation, persistStep is called directly as an event handler callback. Since it's an async function, the event dispatcher doesn't wait for it to complete. The screenshots are never captured because:

1. The async function starts executing
2. The event dispatcher immediately moves on to the next event
3. The screenshot operation may be cancelled or never completes

Reproduction Steps

1. Create a minimal CodeceptJS config with only stepByStepReport enabled
2. Run any test that fails (to keep the report)
3. Check the _output/record_*/ directory
4. The directory contains only index.html with no PNG files

Minimal Reproduction

// codecept.minimal.conf.js
const path = require('path');

exports.config = {
  output: path.resolve(__dirname, '_output'),
  helpers: {
    Playwright: {
      browser: 'chromium',
      url: 'https://www.google.com',
      show: true
    }
  },
  plugins: {
    stepByStepReport: {
      enabled: true
    }
  },
  tests: './test_minimal.js',
  name: 'minimal-test'
};

// test_minimal.js
Feature('Minimal Test');

Scenario('Test stepByStepReport', ({ I }) => {
  I.amOnPage('https://www.google.com');
  I.wait(2);
  I.see('Google');
  I.see('TEXT THAT DOES NOT EXIST'); // Force failure to keep report
});

Run with:

npx codeceptjs@3.7.5 run --config codecept.minimal.conf.js

Expected: _output/record_*/ contains 0000.png, 0001.png, etc.
Actual: _output/record_*/ contains only index.html with no PNGs

Suggested Fix

Restore the original behavior by wrapping persistStep in recorder.add():

event.dispatcher.on(event.step.after, step => {
  recorder.add('screenshot of step of test', async () => persistStep(step), true);
});

This ensures the async screenshot operation is properly queued and executed.

Additional Context

• This issue has been reported before as #4856 (Feb 2025) and is a follow-up of #4637
• The original fix was intended to prevent errors in BeforeSuite/AfterSuite hooks, but it inadvertently broke the core screenshot functionality
• The issue affects both single-run and parallel execution modes

Environment

• CodeceptJS: 3.7.5
• Node.js: 20.x
• Playwright: 1.57.0
• OS: Linux (Ubuntu 22.04)

Written with help from Claude 4.5 Opus.