web-infra-dev / web-infra-dev/midscene

[Feature]: Add configurable report screenshot retention and image encoding policies

Open
#2,833 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
14.9k
Forks
1.2k
Avg merge
1d 14h
Merged PRs (30d)
96

Description

What problem does this feature solve?

Midscene reports may contain a large number of high-resolution screenshots,
especially during long-running mobile or desktop automation sessions.

The existing html-and-external-assets output format separates screenshots from
the HTML file, but it does not reduce:

  • the number of screenshots retained in a report;
  • the total report asset size;
  • unnecessary screenshots from actions that are not useful for human review.

For some use cases, users only need screenshots for specific actions, such as
Tap. Screenshots from planning, querying, input, scrolling, or other actions
do not always need to be persisted in the final report.

Users may also only need a smaller JPEG asset for human inspection instead of
the original high-resolution PNG/JPEG bytes.

This should be a Core report-generation capability, independent of Studio and
independent of screenshots used for AI perception.

Required report policies:

  1. Screenshot retention:

    • retain all screenshots;
    • retain no screenshots;
    • retain screenshots only for selected action subtypes.
  2. Report image persistence:

    • preserve the original image bytes;
    • encode persisted report images as JPEG;
    • configure JPEG quality.

The default behavior must remain unchanged for backward compatibility.

What does the proposed API look like?

Extend AgentOpt with report-specific options:

export type ReportScreenshotFilter =
  | { mode: 'all' }
  | { mode: 'none' }
  | { mode: 'actions'; actions: string[] };

export type ReportImageFormat = 'original' | 'jpeg';

export interface AgentOpt {
  /**
   * Select which action screenshots are retained in generated reports.
   * Action names match `ExecutionTask.subType`, such as `Tap` or `Input`.
   *
   * @default { mode: 'all' }
   */
  reportScreenshotFilter?: ReportScreenshotFilter;

  /**
   * Encoding used when report screenshots are persisted.
   *
   * @default 'original'
   */
  reportImageFormat?: ReportImageFormat;

  /**
   * JPEG quality used when `reportImageFormat` is `jpeg`.
   * Valid range: 1-100.
   *
   * @default 80
   */
  reportImageQuality?: number;
}



example:

const agent = new AndroidAgent(device, {
  outputFormat: 'html-and-external-assets',
  reportScreenshotFilter: {
    mode: 'actions',
    actions: ['Tap'],
  },
  reportImageFormat: 'jpeg',
  reportImageQuality: 80,
});

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating AgentOpt and the Core report-generation path for the html-and-external-assets output format, then trace how ExecutionTask.subType is associated with screenshots. Implement the three retention modes, original or JPEG persistence, and configurable JPEG quality while preserving defaults. Done means the proposed AndroidAgent options work and report assets follow the selected policy.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.