HarperFast / HarperFast/harper
Follow-up to #1736: streaming results and isJob for component-registered operations
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Context
#1736 (fixed by the cross-thread bridge PR) made `server.registerOperation()` from per-worker component contexts reachable via the main-thread ops API by forwarding the request to one registering worker over ITC, with the result returned via structured clone.
Two capabilities were deliberately scoped out of that fix:
## 1. Streaming results
A forwarded operation that returns a `Readable` currently gets an explicit `501`:
> Operation '' returned a stream; streaming results are not supported for operations registered from a component
Supporting this needs MessagePort transfer plumbing (transfer a port with the request, pipe the stream across it, re-wrap as a `Readable` on the main thread with backpressure and error propagation).
## 2. `isJob`
`OperationDefinition.isJob` was already ignored by `registerOperation()` before #1736 — it only ever wires `execute` as the regular `operation_function`, never a `job_operation_function`. If we want component-registered ops runnable as jobs, job worker threads would also need to load the registering component (they currently load only built-in env components), or the job dispatch path needs the same forwarding treatment.
— KrAIs 🤖 (on Kris's behalf)
Contributor guide
Research direction
Trace server.registerOperation through the main-thread ops API and the worker ITC forwarding path first. For streaming, investigate MessagePort transfer, backpressure, and error propagation; for jobs, inspect the job dispatch path and OperationDefinition.isJob handling. Done means component-registered operations support streamed results and, if included, run correctly as jobs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100