thefrontside / thefrontside/effectionx

process: join() settles before stdout/stderr pumps finish Stdio middleware

Open
#244 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
12
Forks
4
Avg merge
3h 17m
Merged PRs (30d)
1

Description

Problem

Process.join() can settle after Node's child-process close event but before
the Effection stdout and stderr pump tasks have finished processing the final
chunks through Stdio middleware.

This is distinct from #228. That issue correctly separates direct-child exit
from pipe closure and keeps join() close-settled. The remaining gap is between
the OS streams closing and the asynchronous Effection pumps finishing the data
already delivered to them.

In the POSIX adapter, the close watcher resolves processResult immediately:

let value = yield* once<ProcessResultValue>(childProcess, "close");
processResult.resolve(Ok(value));

join() waits only for processResult. Each output pump independently awaits
public middleware before marking itself done:

yield* Stdio.operations.stdout(next.value);
stdout.send(next.value);
// ...
io.stdoutDone.resolve();

Consequently, close-settled does not currently mean pump-complete on POSIX.
The Windows adapter already waits for stdoutDone and stderrDone before
resolving the process result.

Reproduction

Holding a stdout middleware operation proves the ordering without depending on
the volume of output. On @effectionx/process@0.8.1, join() resolves while
the final chunk is still blocked inside the handler:

import {
  main,
  race,
  scoped,
  sleep,
  spawn,
  withResolvers,
} from "effection";
import { exec, Stdio } from "@effectionx/process";
import process from "node:process";

await main(function* () {
  const entered = withResolvers<void>();
  const release = withResolvers<void>();
  const joined = withResolvers<number | undefined>();

  yield* scoped(function* () {
    yield* Stdio.around({
      *stdout([bytes], next) {
        entered.resolve();
        yield* release.operation;
        yield* next(bytes);
      },
    });

    const task = yield* spawn(function* () {
      const child = yield* exec(process.execPath, {
        arguments: ["-e", 'console.log("TAIL")'],
      });
      const status = yield* child.join();
      joined.resolve(status.code);
    });

    yield* entered.operation;
    const observed = yield* race([
      function* () {
        return `joined:${yield* joined.operation}`;
      }(),
      function* () {
        yield* sleep(100);
        return "pending";
      }(),
    ]);

    console.log(observed); // joined:0 — expected: pending
    release.resolve();
    yield* task;
  });
});

The visible TAIL is forwarded only after the release, even though join()
has already reported completion.

Consequences

A caller that retains or decodes output through Stdio middleware can:

  • finalize its TextDecoder before the final chunk reaches it;
  • persist a result that omits final stdout or stderr;
  • begin later work before all foreground output has been forwarded; or
  • dismantle the middleware scope while a pump still owns pending output.

The same issue affects expect() and the collecting Exec.join() / Exec.expect()
helpers through their use of Process.join() or Process.expect().

Required contract

Process.join() and Process.expect() remain close-settled as decided in #228,
and additionally do not settle until both channel pumps have completed all of
the following:

  1. every source chunk has been passed through its corresponding Stdio
    operation;
  2. the public output Signal has received the forwarded chunk; and
  3. the channel has reached EOF and its pump has completed.

Stdio middleware therefore provides real backpressure: if a handler is still
processing a chunk, join() remains pending.

Acceptance criteria

  • POSIX join() and expect() wait for both stdout and stderr pumps after the
    child-process close event.
  • A blocked stdout or stderr Stdio handler keeps join() pending until it is
    released.
  • The final output chunk is observable before join() settles.
  • Pump or Stdio failures have defined failure behavior and cannot leave
    join() waiting forever.
  • The portable contract is covered for both POSIX and Windows adapters; the
    Windows implementation retains its existing drain-before-result ordering.
  • Exec.join() and Exec.expect() inherit the same output-completeness
    guarantee.
  • Regression tests use a deterministic gate in the adapter/pump seam rather
    than scheduler sleeps.
  • No new settle option or public API is introduced.

Non-goals

  • Adding raw-output observers or changing Stdio middleware ordering.
  • Changing the exited() design in #228.
  • Repairing the halt-during-acquisition orphan window in #236.
  • Changing process-group termination or Windows force-kill policy.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Trace Process.join() and Process.expect() through the POSIX and Windows adapters, then inspect the stdout and stderr pump completion paths around Stdio middleware. Add deterministic gate coverage at the adapter/pump seam for blocked handlers, final chunks, pump failures, and both platforms; done means Process and Exec helpers settle only after output pumps complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
backend, operating-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.