NVIDIA / NVIDIA/NeMo-Retriever

[FEA]: Enhanced tracing for individual pipeline stages - raw output

Open
#100 1 comment 0 reactions 1 assignee View on GitHub

@edknv is already working on this.

Since Oct 9, 2024.

feature request
Dominant language
Python
Stars
3k
Forks
349
Avg merge
1d 23h
Merged PRs (30d)
116

Description

Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request

Currently preventing usage

Please provide a clear description of problem this feature solves

Several people have requested being able to view the output from each individual stage in a pipeline. Since we now have our opentelemetry global and unique trace-id I think it would make sense to add a configuration that would allow for each stage in the morpheus pipeline to output/write it raw data to a folder that has the same name as the trace-id and then the file named the same as the stage. This would allow for users to view the output for each stage and individually analyze it as well.

While this could possibly be done with zipkin and opentelemetry itself given we mostly deal with images I think the output would be far to large to capture there.

Describe the feature, and optionally a solution or implementation and any alternatives

Capture output from each individual stage in a complete Morpheus pipeline when a yet to be added configuration is enabled.

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.