nushell / nushell/nushell

`PipelineData::into_value` implicitly waits for `ExternalStream` to finish and irreversibly converts it

Open
#11,870 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

category:enhancement status:needs-triage
Dominant language
Rust
Stars
40.5k
Forks
2.3k
Avg merge
1d 19h
Merged PRs (30d)
85

Description

Related problem

For example, you can run this command:

~> run-external --redirect-combine cmd /c "echo test" | complete
╭───────────┬──────╮
│ stdout    │ test │
│           │      │
│ exit_code │ 0    │
╰───────────┴──────╯

But, if you were to have an intermediate pipeline step that does not attempt to change the pipeline data, like inspect or describe:

~> run-external --redirect-combine cmd /c "echo test" | describe | complete
Error:   × Complete only works with external streams
   ╭─[entry #2:1:1]
 1 │ run-external --redirect-combine cmd /c "echo test" | describe | complete
   ·                                                                 ────┬───
   ·                                                                     ╰── complete only works on external streams
   ╰────


Any attempt to convert the pipeline data into a value causes the consuming command to block until it is done, and irreversibly converts the data into a string.

This is caused by this logic:
https://github.com/nushell/nushell/blob/903afda6d946960682dc5234724a09d2bfed288b/crates/nu-protocol/src/pipeline_data.rs#L156-L209

Describe the solution you'd like

It's surprising that interpreting the pipeline data associated with an external command causes it to:

  1. Be transformed
  2. Block until the external command has finished executing

For my use case, I'm trying to run an external command, collect its output and echo its output to nushell's stdout, and wait for the command to finish to get its exit code.
I cannot do all three at the same time.

As for the solution, I'm not sure. Perhaps it would be worth considering a new value type for an external command, or to convert the external stream into a record and back into an external stream for use in a command like collect.

Describe alternatives you've considered

No response

Additional context and details

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the PipelineData::into_value logic in crates/nu-protocol/src/pipeline_data.rs around lines 156-209, then reproduce the behavior with run-external, inspect or describe, and complete. Define and test behavior that preserves the external stream while allowing output collection and exit-code handling; the issue does not specify the final representation or implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.