antirez / antirez/h3.c

Bug: Interactive session doesn't emit 'Done ->' line when stdout is a pipe (non-TTY)

Abierto
#25 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
2.7k
Forks
210
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

## Bug: Interactive session doesn't emit `Done ->` line when stdout is a pipe (non-TTY)

### Summary

When driving the h3 interactive session with **piped stdin/stdout** (e.g. from a server process, Python subprocess, or any non-terminal harness), the `Done -> [s]` line is **never flushed to stdout**, even though the video is fully generated and written to disk. This makes automation/API integration of the interactive mode unreliable: a caller blocks forever waiting for completion, while h3 is actually sitting idle at the `h3>` prompt.

### Environment

- **Device:** MacBook Pro, Apple M5 Max, 128 GB unified memory, macOS 26.5.2
- **Build:** latest `main` (tested at `8974cc0`)
- **Model:** original BF16 checkpoint (`MiniMaxAI/MiniMax-H3`, FL2VA + Ref2VA trees)
- **ffmpeg:** 8.1 (homebrew), libx264 + aac present

### Reproduction

Start the interactive session with a pipe on stdin/stdout (as any server wrapper would), then send a prompt:

```python
import subprocess, select, time

p = subprocess.Popen(
["./h3", "-d", "./MiniMax-H3",
"--width", "512", "--height", "512", "--frames", "90",
"--steps", "10", "--layers", "50", "--reuse", "1"],
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
text=True, bufsize=1)

time.sleep(2)
p.stdin.write("A lighthouse at sunset, waves crashing\n")
p.stdin.flush()

while True:
r, _, _ = select.select([p.stdout], [], [], 1)
if r:
line = p.stdout.readline()
print(line.rstrip())
if "Done ->" in line:
break
```

**Expected:** after `FFmpeg 90/90`, the session prints `Done -> /tmp/h3-XXXX/video-0001.mp4 [..s]` and returns to the `h3>` prompt.

**Actual:** output stops at `FFmpeg 0/90` → `FFmpeg 90/90`, then **nothing**. The process keeps running (state `Ss`, waiting on `read(stdin)`), and no `Done` line ever appears. The video file, however, **is complete and valid** on disk.

### Evidence that the generation actually succeeded

- The MP4 exists and is fully playable: `ffprobe` reports `duration=3.750000`, `size=9624108` (a 90-frame 512×512 clip, byte-identical size to a successful CLI run).
- The ffmpeg subprocess exits cleanly (wrapped `H3_FFMPEG` with a diagnostic script: `posix_spawnp` succeeds, args are correct, no errors, no lingering process).
- Attaching lldb to the stuck process shows the **main thread blocked in `__read_nocancel` inside `h3_cli_run`** — i.e. it already returned to the `linenoise("h3> ")` read, waiting for the next command. Generation finished; only the completion line is missing.

```text
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
* frame #0: libsystem_kernel.dylib`__read_nocancel + 8
frame #1: libsystem_c.dylib`__sread
frame #2: libsystem_c.dylib`_sread
...
frame #7: h3`h3_cli_run + 892
```

### Root cause

**stdout line-buffering.** When stdout is not a TTY, the C standard library uses **full buffering** (typically 8 KB). The `Done -> ...` line written by `generate()` in `h3_cli.c` goes to `stdout` and sits in the userspace buffer. It is only flushed when the buffer fills, when the process exits, or on explicit `fflush`. Progress lines go to `stderr` (line-buffered / `\r`-based via `cli_progress`), so they arrive fine — but the completion line never does.

This is why it works in a terminal (line-buffered TTY) and in one-shot `-p` CLI mode (process exits → buffers flushed), and why it intermittently "worked" for short videos in earlier tests (coincidental buffer fill from banner output).

### Fix

Force line-buffered stdout at startup (one line in `main()`):

```c
int main(int argc, char **argv) {
/* Line-buffer stdout so Done/status lines flush immediately
* when stdout is a pipe (non-TTY). */
setvbuf(stdout, NULL, _IOLBF, 0);
...
}
```

### Verified after fix (interactive session, piped stdio)

| Run | Config | Result |
|---|---|---|
| 1st (model load) | 512×512, 90 f, 10 steps | `Done -> ... [79.27s]` ✅ |
| 2nd (resident) | 512×512, 56 f, 10 steps | `Done -> ...` in ~46s ✅ |
| 3rd (resident) | 512×512, 90 f, 10 steps | `Done -> ...` in ~80s ✅ |

Video numbering increments correctly (`video-0001/0002/0003.mp4`), confirming the session returns to a healthy interactive loop after each generation.

### Suggested improvement (optional)

For robust programmatic use, consider also adding a machine-readable completion signal independent of stdout buffering (e.g. print `Done` to stderr like the progress lines, or support an `--output-json` flag). That would make server-side wrappers immune to buffering semantics entirely.

Thanks for this amazing project — the Metal implementation is outstanding. 🙏

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.