`PipelineData::into_value` implicitly waits for `ExternalStream` to finish and irreversibly converts it
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 40.5k
- Forks
- 2.3k
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 85
Description
Related problem
For example, you can run this command:
~> run-external --redirect-combine cmd /c "echo test" | complete
╭───────────┬──────╮
│ stdout │ test │
│ │ │
│ exit_code │ 0 │
╰───────────┴──────╯
But, if you were to have an intermediate pipeline step that does not attempt to change the pipeline data, like inspect or describe:
~> run-external --redirect-combine cmd /c "echo test" | describe | complete
Error: × Complete only works with external streams
╭─[entry #2:1:1]
1 │ run-external --redirect-combine cmd /c "echo test" | describe | complete
· ────┬───
· ╰── complete only works on external streams
╰────
Any attempt to convert the pipeline data into a value causes the consuming command to block until it is done, and irreversibly converts the data into a string.
This is caused by this logic:
https://github.com/nushell/nushell/blob/903afda6d946960682dc5234724a09d2bfed288b/crates/nu-protocol/src/pipeline_data.rs#L156-L209
Describe the solution you'd like
It's surprising that interpreting the pipeline data associated with an external command causes it to:
- Be transformed
- Block until the external command has finished executing
For my use case, I'm trying to run an external command, collect its output and echo its output to nushell's stdout, and wait for the command to finish to get its exit code.
I cannot do all three at the same time.
As for the solution, I'm not sure. Perhaps it would be worth considering a new value type for an external command, or to convert the external stream into a record and back into an external stream for use in a command like collect.
Describe alternatives you've considered
No response
Additional context and details
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the PipelineData::into_value logic in crates/nu-protocol/src/pipeline_data.rs around lines 156-209, then reproduce the behavior with run-external, inspect or describe, and complete. Define and test behavior that preserves the external stream while allowing output collection and exit-code handling; the issue does not specify the final representation or implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100