web-infra-dev / web-infra-dev/midscene

[Feature]: Export AI execution steps as reusable Playwright scripts

Open
#2,240 3 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
14.9k
Forks
1.2k
Avg merge
1d 14h
Merged PRs (30d)
96

Description

English Translation

What problem does this feature solve?
🚀 Feature Request

Allow Midscene to export AI execution steps as reusable Playwright scripts.

💡 Background

Midscene currently provides powerful AI testing capabilities:

  • Execute tests via natural language (aiAct)
  • Automatic page understanding and interaction
  • Execution logs and reports
  • Can generate Playwright-like test case examples
🎯 Problem

I need:

  • Repeatable, deterministic scripts
  • Stable selectors (avoiding flakiness from visual recognition)
  • CI/CD execution (no AI dependency, reducing cost and uncertainty)

Currently missing:

After AI executes once successfully, can the corresponding Playwright script be exported for independent future execution?

✅ Expected Behavior

Provide an export capability that converts successfully executed test cases into actual Playwright scripts.

🧠 Desired behavior:

  • Convert AI execution steps into Playwright code
  • Generate stable selectors where possible, e.g.: getByRole, getByText, with fallback to CSS / XPath when necessary
  • Preserve execution order and semantics
  • Allow users to manually optimize after export
What does the proposed API look like?

N/A


原始中文描述 / Original Chinese

What problem does this feature solve?

🚀 功能请求

希望 Midscene 能支持:将 AI 执行过程导出为可复用的 Playwright 脚本


💡 背景

当前 Midscene 提供了非常强大的 AI 测试能力:

  • 通过自然语言执行测试(aiAct)
  • 自动完成页面理解与操作
  • 提供执行日志和报告
  • 可以生成类似 Playwright 的测试用例示例

🎯 问题

我需要:

  • 可重复执行的确定性脚本(deterministic)
  • 稳定的 selector(避免视觉识别带来的波动)
  • 在 CI/CD 中运行(不依赖 AI,降低成本和不确定性)

目前缺少一种能力:

能否在 AI 执行一次后,导出对应的 Playwright 脚本,并在后续独立执行?

✅ 期望

提供一种导出能力,将本次执行成功的用例导出为实际playwright的脚本

🧠 期望行为
将 AI 执行步骤转换为 Playwright 代码
尽可能生成稳定 selector,例如:
getByRole
getByText
必要时 fallback 到 CSS / XPath
保留执行顺序和语义
允许用户在导出后手动优化

What does the proposed API look like?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by reading the existing execution logs, reports, and Playwright-like test example generation, then determine how a successful AI run can be represented for export. Done means producing reusable Playwright scripts that preserve execution order and semantics, use stable selectors where possible, and remain manually editable.

Written by the indexing model from the issue text.

Assessment

Tech stack
playwright, typescript
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.