scanapi / scanapi/scanapi

Local execution history and trend analysis

Open
#973 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

CLI Performance Stale
Dominant language
Python
Stars
1.6k
Forks
140
Avg merge
1h 5m
Merged PRs (30d)
1

Description

Feature request

Description the feature

Introduce a lightweight, local execution history mechanism to perform trend analysis and warn users about performance degradation or sudden changes in response sizes.

Goals of this feature:

  • Zero-Dependency History Storage: Establish a lightweight, local, and file-based mechanism (e.g., JSON or SQLite) to persist metadata of past runs without requiring external database setups.
  • Performance Regression Detection: Implement a simple comparison algorithm to detect significant increases in latency or response size compared to historical running averages.
  • Actionable Developer Alerts: Warn developers directly in the CLI and HTML reports when an endpoint's performance degrades beyond a configurable threshold.
  • Strict Storage Bounds: Cap the historical log size to prevent infinite disk space consumption on the user's local machine.
Is your feature request related to a problem?

Yes. Currently, every ScanAPI execution is stateless and completely isolated. There is no historical context. If an endpoint becomes 50% slower after a new deployment, ScanAPI will still pass (as long as it doesn't timeout), and the developer might miss a critical performance regression.

Do you have any suggestions on how to add this feature in scanapi ?
  • Storage: Create a HistoryManager class to handle reading/writing to a local, hidden file (e.g., .scanapi_history.json in the project root or user's cache directory). Ensure the file is capped (e.g., keeping only the last 20 runs) to keep it lightweight.
  • Logic: During a run, ScanAPI compares the current average response times and success rates against the moving average of the last 5 or 10 runs stored in the history file.
  • User Alerts:
    • If an endpoint's response time increases by a certain threshold (e.g., >20%), the CLI can print a warning: ⚠️ Warning: GET /users is 25% slower than its 10-run average (250ms vs 200ms).
    • Add a CLI flag to disable this behavior (e.g., --no-history) or to clear history.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the ScanAPI execution flow and the CLI and HTML report entry points mentioned in the issue. Define how local history, bounded retention, moving-average comparisons, warnings, and opt-out or clearing flags should fit together. Done means the feature is implemented across storage, regression detection, and both reporting surfaces with a documented way to verify alerts.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, cli, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.