opensearch-project / opensearch-project/flow-framework

[FEATURE] Improve robustness of WorkflowNode parsing of User Inputs

Open
#650 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

backlog enhancement
Dominant language
Java
Stars
62
Forks
66
Avg merge
3d 20h
Merged PRs (30d)
20

Description

Is your feature request related to a problem?

Parsing of Workflow Nodes (steps) assumes that JSON objects are key-value maps unless specifically coded otherwise:
https://github.com/opensearch-project/flow-framework/blob/51e229ca0a54a33fef1da42be704b070cc3072b4/src/main/java/org/opensearch/flowframework/model/WorkflowNode.java#L163-L180

This is brittle and can lead to failure if a new feature/API is added which includes a more complex nested object. As more workflow steps are created, the chances of more custom objects increases.

What solution would you like?

Add a processing layer in between encountering the start of an object and reading the map:

  • Start building a new object with a new XContent Builder
  • Process tokens, copying them over into the new builder until the corresponding END_OBJECT is reached
  • Build the object
  • Send the entire (new) XContent object to the existing map parsing method
  • Catch a parsing exception creating the map; on exception, simply place a String value (containing the JSON form of the object) into the map for that key. Alternately, just ignore that value (or place null in the map) since in any case, if the step doesn't include that as a required or optional input, it's not even going to be evaluated. In either case, log the parsing exception.
What alternatives have you considered?

We can also try to parse multiple layers of maps, but would need to protect against arbitrary levels of recursion. This is essentially what's done by other JSON parsers such as GSON, but we decided early on to use XContent parsing.

Do you have any additional context?

In addition to robustness in parsing, we need to set up a way to test against all implemented APIs (steps) using their specifications, to catch API changes early.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with WorkflowNode.java lines 163-180 and review the XContent parsing approach referenced there, along with issue #46. Trace how user inputs are parsed for workflow steps, then examine the implemented API specifications for test coverage. Done means nested objects no longer cause brittle parsing failures and parsing exceptions are handled as described.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.