Feature Request: Add unfold construct to Flowable/Observable

Version 3.1.4

Scala as of 2.13 and Akka (another reactive framework) have added various unfold patterns, and I'm finding a need for this currently. Here's the use case - pagination of responses to a "cursor"-like REST api.

• Akka Source.unfold
• Scala unfold

Essentially, I'd like to make a Flowable that makes requests to get another page - where it starts without a "continuation" parameter, and keeps making requests (honoring backpressure) until the response comes without a "continuation".

Flowable.create doesn't seem to deal with the backpressure well, and Flowable.generate is blocking, which seems disappointing as Flowable.fromFuture is great at dealing with async http clients.

This unfold pattern seems to offer what would be very helpful to the cause - where a new request isn't made until the subscriber asks for more, and the termination state is known from some inspection of the previous element in the feed (the http response, or the json parsing of it).

Here's a stab of making these using RxJava constructs; it would be nice if Flowable.unfold and Flowable.unfoldMaybe (and Observable too) could be added to RxJava itself.

import java.util.Optional;
import java.util.function.Function;

import io.reactivex.rxjava3.core.Flowable;
import io.reactivex.rxjava3.core.Maybe;
import io.reactivex.rxjava3.core.MaybeSource;

public class Unfold {
    public static class Pair<L, R> {
        private final L left;
        private final R right;

        public Pair(L left, R right) {
            this.left = left;
            this.right = right;
        }

        public L getLeft() {
            return left;
        }

        public R getRight() {
            return right;
        }
    }

    public static <S, E> Flowable<E> flowableUnfold(S state, Function<S, Optional<Pair<S, E>>> generator) {
        return Flowable.defer(() -> Flowable.fromOptional(generator.apply(state)))
                .flatMap(pair -> Flowable.just(pair.getRight())
                        .concatWith(flowableUnfold(pair.getLeft(), generator)));
    }

    public static <S, E> Flowable<E> flowableUnfoldMaybe(S state, Function<S, ? extends MaybeSource<Pair<S, E>>> generator) {
        return Flowable.defer(() -> Flowable.fromMaybe(generator.apply(state)))
                .flatMap(pair -> Flowable.just(pair.getRight())
                        .concatWith(flowableUnfoldMaybe(pair.getLeft(), generator)));
    }
}

And some scala to test it

@RunWith(classOf[JUnitRunner])
class UnfoldTest extends AnyFlatSpec with Matchers {
  "flowableUnfold" should "work with range of 0 to 10" in {
    val flowable = Unfold.flowableUnfold[(Int, Int), Int]((0, 10), {
      case (start: Int, end: Int) if start < end =>
        Option(new Pair(start + 1 -> end, start)).asJava
      case _ => None.asJava
    })
    flowable.toList.blockingGet().asScala shouldBe (0 until 10)
  }

  "flowableUnfoldMaybe" should "work with range of 0 to 10" in {
    val flowable = Unfold.flowableUnfoldMaybe[(Int, Int), Int]((0, 10), {
      case (start: Int, end: Int) if start < end =>
        Maybe.fromCallable(() => new Pair(start + 1 -> end, start))
      case _ => Maybe.empty()
    })
    flowable.toList.blockingGet().asScala shouldBe (0 until 10)
  }
}

Start by reading the existing Flowable.create, Flowable.generate, and Flowable.fromFuture entry points and compare their backpressure and asynchronous behavior. Review the proposed Flowable.unfold, Flowable.unfoldMaybe, and Observable counterparts, then use the supplied range examples to define completion, demand, and async behavior before considering implementation and tests.