opensearch-project / opensearch-project/flow-framework

[META] Improvements for default use cases

Open
#666 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Java
Stars
62
Forks
66
Avg merge
3d 20h
Merged PRs (30d)
20

Description

Is your feature request related to a problem?

In 2.13 we released a new feature that enables customers to utilize predefined templates to configure their workflows by selecting a given use case. Currently the feature is documented here: https://opensearch.org/docs/latest/automating-configurations/workflow-templates/

For the first release of the feature in order not to cause any later breaking changes we made a few decisions that can be revisited.

  1. We currently host our substitution ready templates in a folder here, and the different defaults for each use case here. The reason these aren’t in the sample is because we didn’t want to make a breaking change to our template and because we felt that in the future their can be more optimal ways to utilize one substitution template with multiple different default files so it wont always be 1:1 match between substitution ready templates and defaults.

  2. The defaults and predefined templates are currently in our repo as part of the resources folder to allow for us to have full ownership and to couple them with the official release of the feature in 2.13. However as we expand the number of use cases we support we might need a better path forward on where to host these templates, whether it be in a different structure within the repo, uploaded to a system index, or in an entirely new location. (related: https://github.com/opensearch-project/flow-framework/issues/584)

Additional Action Items and suggest improvements
  • We need to define criteria on what meets the bar for a new predefined template
  • id field need to be removed from some of the create index steps in various templates
  • Template that use pretrained model can have their dimensions automatically fetched from get model so user doesn’t have to over-write that parameter if they change model
  • Increased variation in defined template https://github.com/opensearch-project/flow-framework/issues/667
  • https://github.com/opensearch-project/flow-framework/issues/584
  • Change substiution file names to align with use case names, with _ instead -
  • Remove the default weights field from hybrid search on template and defaults file
  • Allow capability to change default text and image type for multimodal templates
  • Add more pretrained model templates
  • Add template with reindex step

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the substitutionTemplates and defaults directories, then read the related issues 584 and 667 and the workflow templates documentation. The issue spans template structure, naming, model dimensions, and criteria for new use cases; done would require an agreed scope and resolution for each remaining action item.

Written by the indexing model from the issue text.

Assessment

Domain
ai
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.