forwardemail / forwardemail/superagent
using with paginated api requests
- Dominant language
- JavaScript
- Stars
- 16.6k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
So I'm basically using node-huntergatherer (https://github.com/erictj/node-huntergatherer) for one of my projects for my api pagination needs but as the library is not updated any more, i'm considering to switch superagent.
But what I couldn't find out yet is how to support paginated api calls with it;
To make it more clear, node-huntergatherer supports them natively. Here is the explanation from the project;
> It's ideally suited to the types of calls you see via APIs. Here's one example from the Etsy API (http://developer.etsy.com/docs) that you need to make if you want to gather all of a users' active listings:
>
> ``` http://openapi.etsy.com/v2/listings/active?limit=50&offset=0
> http://openapi.etsy.com/v2/listings/active?limit=50&offset=50
> http://openapi.etsy.com/v2/listings/active?limit=50&offset=100
> http://openapi.etsy.com/v2/listings/active?limit=50&offset=150
> http://openapi.etsy.com/v2/listings/active?limit=50&offset=200
> http://openapi.etsy.com/v2/listings/active?limit=50&offset=250
> ```
>
> Traditionally, you'd loop through each call, process the results of that call, and then call the next offset, process, ad nauseum.
>
> Not only is this super inefficient, it's slow and a pain in the ass to re-code all the time. Especially when this pattern occurs in many APIs, and all that changes between them for the most part are the URL and how you process the results. So let's abstract out everything else!
>
> And lastly, with HunterGatherer each of these calls happen simultaneously, up to the maximum number of simultaneous connections allowed based on the remote API terms of usage. Non-blocking FTW!
Is this possible with superagent and I'll be really glad if i can see a working sample.
Contributor guide
Assessment
This issue has not been assessed yet.