Conversion from blocking primitives (e.g. Iterable) and deadlocks
- Dominant language
- Java
- Stars
- 1k
- Forks
- 230
- Avg merge
- 23h 23m
- Merged PRs (30d)
- 16
Description
additional context: https://github.com/apple/servicetalk/pull/2204
If `FromIterablePublisher` isn't able to unwrap to get access to the underlying `Publisher` deadlock is possible on consumption for the same reasons described in this PR. There are two options
#### Ensure the subscription and subscriber aren't using the same thread.
We can force a `subscribeOn` however this brings new questions:
- offloading may not always be necessary
- what level is the offloading applied at (e.g. do we require it during the conversion, for http we do this under the covers so we may have to hook into the execution strategy)
- how do we infer/accept an executor (if on the `concurrent.api` package we don't have an executor to use, if we require one as a parameter that starts to overload operator responsibilities)
- are there other areas in the blocking APIs with similar issues (InputStream, ..)?
#### Reduce default demand to `1` from `PublisherAsBlockingIterable`
This isn't ideal because it means demand will be very chatty and no batching is done, but will mean additional offloading isn't required.
Contributor guide
Research direction
Start by reading pull request #2204 and tracing the conversion involving FromIterablePublisher and PublisherAsBlockingIterable. Compare the proposed offloading and demand approaches, investigate whether InputStream and other blocking APIs have similar issues, and define a decision that prevents consumption deadlocks without unnecessary offloading.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend-api-design, networking
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100