opensearch-project / opensearch-project/OpenSearch

[Feature Request] Add Version Tracking for Search Pipelines

Open
#17,539 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement Search
Dominant language
Java
Stars
13.7k
Forks
3k
Avg merge
2d 23h
Merged PRs (30d)
108

Description

Is your feature request related to a problem? Please describe
Description:

Currently, when a user updates a search pipeline, there's no way to track or view the previous versions of that pipeline. This makes it difficult to understand the evolution of a pipeline over time or to revert to a previous version if needed.

Describe the solution you'd like

Feature Request:
Implement a version tracking system for search pipelines that allows users to:

  1. View the history of changes made to a search pipeline
  2. Compare different versions of a search pipeline
  3. Revert to a previous version of a search pipeline if needed
Related component

No response

Describe alternatives you've considered
Use Case:

When using a search pipeline like:
GET /my_index/_search?search_pipeline=my_pipeline

Users should be able to:

  • See when and how the pipeline was last modified
  • Access a list of previous versions
  • View the differences between versions
  • Easily roll back to a previous version if the current one has issues
Potential Implementation:
  • Store each version of a search pipeline with a timestamp and version number
  • Provide a new API endpoint to retrieve version history
  • Add a UI component in the OpenSearch Dashboards to visualize and manage pipeline versions
Benefits:
  • Improved tracking and auditing of search pipeline changes
  • Easier troubleshooting and debugging of search behavior
  • Ability to revert problematic changes quickly
  • Better collaboration among team members working on the same pipelines

Additional Considerations:

  • Consider adding user information to track who made each change
  • Implement a retention policy for old versions to manage storage
  • Allow users to add comments or descriptions to each version
Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the search pipeline behavior behind GET /my_index/_search?search_pipeline=my_pipeline. Define the scope for version history, comparison, rollback, timestamps, user attribution, retention, comments, and any OpenSearch Dashboards UI before identifying implementation entry points. Done means the versioning workflow and API are specified for these requested operations.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.