apache / apache/hop

[Feature Request]: Execution Information Location: retention policy (by count / by days)

Open
#6,497 3 comments 1 reaction 0 assignees View on GitHub
Hop Gui Metadata P3
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
18h 32m
Merged PRs (30d)
216

Description

### What would you like to happen?

Problem
Execution Information Location keeps accumulating execution records (e.g., log_file location) without built-in retention controls. In long-running production usage this increases disk usage and makes later analysis/navigation slower.

Proposal
Add optional retention settings to Execution Information Location metadata:

Retention by count: keep last N executions per pipeline/workflow (or per location), delete older records automatically.

Retention by age: keep executions for last X days, delete older records automatically.

Option to choose scope:

per “location” (global)

per pipeline/workflow name (recommended)

Acceptance criteria

When retention is enabled, older execution records are removed automatically.

Deletion should be safe: remove only records belonging to the configured location.

Works for “local/log_file” execution information location and any other supported backend.

Retention runs:

on write (after storing a new execution), and/or

as a periodic cleanup task (configurable)

Notes / rationale
This supports production hygiene and avoids external scripts for rotation/cleanup.

### Issue Priority

Priority: 3

### Issue Component

Component: Hop Gui, Component: Metadata

Contributor guide

Open the contributing guide

Research direction

Start with the Hop Gui and Metadata components that define Execution Information Location metadata. Clarify whether retention is scoped per location or per pipeline/workflow, and whether cleanup runs on write or periodically. Done means safely removing only matching older records for local/log_file and other supported backends.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.