Connectable observable sends events multiple times
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5k
- Forks
- 365
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 4
Description
I'm using rxpy for a realtime audio processing tool. The tool receives two event streams, one that contains audio chunks and one that contains small text like annotation snippets. The flow looks similar to this:
audio ----> .publish() ----> rec ----> p1 ---> p2 ---> p 3 ---> p4 ---> merge ---> output
\ / /
---> p5 ---- /
annotations --------------------------------------------------> p6 ---
Here, the p<x> nodes are processing nodes (implemented in numpy/pytorch) and the rec node is a recorder, that writes its input to disk and otherwise passes it on unchanged. I'm using the .publish() call to support the branching that comes after the rec node.
When I open the audio file written by the rec node, every chunk has been written 3 times, which implies that the rec node received every chunk 3 times. Is this intended behaviour? How can I avoid this? I'm worried that downstream nodes (p1-p5) might receive multiple repetitions of the same chunk as well and therefore might not operate as intended. However, the pipeline as a whole seems to work correctly.
I tried a number of variations:
- No
.publish()call: Events in the first pipeline get stuck right beforep3.p4never receives any events. - Introduce a separate step before the
recnode that drops events if they have the same md5 sum as the previous event (either using a combination ofscanandfilteror afilterwith a class). This makes the audio file look ok, but the overall pipeline becomes prohibitively slow and is essentially broken. - Move the
.publish()call to a later stage: The results are essentially the same as 1.
Thank you for your help.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the .publish() call and recorder node described in the report, then reproduce the three-branch flow with the audio and annotation streams. Trace subscriptions and verify whether each chunk is written once, documenting the intended behavior and any relevant regression test entry point found in the repository.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python, pytorch
- Domain
- audio-video-rtc, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100