HarperFast / HarperFast/harper

Follow-up to #1736: streaming results and isJob for component-registered operations

Open
#1,742 0 comments 0 reactions 0 assignees View on GitHub
area:operations-api
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

## Context

#1736 (fixed by the cross-thread bridge PR) made `server.registerOperation()` from per-worker component contexts reachable via the main-thread ops API by forwarding the request to one registering worker over ITC, with the result returned via structured clone.

Two capabilities were deliberately scoped out of that fix:

## 1. Streaming results

A forwarded operation that returns a `Readable` currently gets an explicit `501`:

> Operation '' returned a stream; streaming results are not supported for operations registered from a component

Supporting this needs MessagePort transfer plumbing (transfer a port with the request, pipe the stream across it, re-wrap as a `Readable` on the main thread with backpressure and error propagation).

## 2. `isJob`

`OperationDefinition.isJob` was already ignored by `registerOperation()` before #1736 — it only ever wires `execute` as the regular `operation_function`, never a `job_operation_function`. If we want component-registered ops runnable as jobs, job worker threads would also need to load the registering component (they currently load only built-in env components), or the job dispatch path needs the same forwarding treatment.

— KrAIs 🤖 (on Kris's behalf)

Contributor guide

Open the contributing guide

Research direction

Trace server.registerOperation through the main-thread ops API and the worker ITC forwarding path first. For streaming, investigate MessagePort transfer, backpressure, and error propagation; for jobs, inspect the job dispatch path and OperationDefinition.isJob handling. Done means component-registered operations support streamed results and, if included, run correctly as jobs.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.