nushell / nushell/nushell.github.io

The cookbook example "Accessing HTTP Response Metadata While Streaming" doesn't stream

Open
#2,232 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
258
Forks
561
Avg merge
3h 20m
Merged PRs (30d)
15

Description

The example in question is:

http get --allow-errors https://api.example.com/events.jsonl
| metadata access {|meta|
    print $"Status: ($meta.http_response.status)"
    print $"Content-Type: ($meta.http_response.headers | where name == content-type | get value.0)"

    if $meta.http_response.status != 200 {
        error make {msg: $"Failed with status ($meta.http_response.status)"}
    } else { }
  }
| lines
| each { from json }
| where event_type == "error"

This won't work because the input to the closure is fed to the first statement, which is print $"Status: ...", while the closure output is taken from else {}. If the http response status is 200, the closure will always output nothing because the input for the closure was consumed by print. We can't test this because api.example.com is a fake subdomain, so let's modify the example to use example.com and not assume the output is json:

http get --allow-errors https://example.com
| metadata access {|meta|
    print $"Status: ($meta.http_response.status)"
    print $"Content-Type: ($meta.http_response.headers | where name == content-type | get value.0)"

    if $meta.http_response.status != 200 {
        error make {msg: $"Failed with status ($meta.http_response.status)"}
    } else { }
  }
| lines

If you run this, you will see that the status and content type are printed, but the http body is not returned from the closure.

One solution is to use the $in variable to collect the input and return it from the else statement:

http get --allow-errors https://example.com
| metadata access {|meta|
    let input = $in
    print $"Status: ($meta.http_response.status)"
    print $"Content-Type: ($meta.http_response.headers | where name == content-type | get value.0)"

    if $meta.http_response.status != 200 {
        error make {msg: $"Failed with status ($meta.http_response.status)"}
    } else { $input }
  }
| lines

This does return the http body from the closure, but the example is called "Accessing HTTP Response Metadata While Streaming", and $in collects the input, so it's no longer a stream.

If we want to make this work while streaming, we can make the if statement into the only top level statement of the closure by moving the print statements into its conditional expression:

http get --allow-errors https://example.com
| metadata access {|meta|
    if (
        print $"Status: ($meta.http_response.status)";
        print $"Content-Type: ($meta.http_response.headers | where name == content-type | get value.0)";

        $meta.http_response.status != 200
    ) {
        error make {msg: $"Failed with status ($meta.http_response.status)"}
    } else {  }
  }
| lines

When run, this streams the http body after printing the status and content-type. However, it is pretty hacky. If we were to put this in the cookbook, it would need an accompanying explanation as to how sticking statements in the conditional expression prevents them from eating the closure input. Perhaps someone with more nushell experience than me can come up with a cleaner way to write this?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the cookbook example titled "Accessing HTTP Response Metadata While Streaming" and reproduce it against https://example.com. Determine a cleaner streaming form than the shown $in or conditional-expression workarounds; done means the status and content type are printed while the HTTP body remains available to the following lines command.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell
Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.