allenai / allenai/molmoweb

Enhancements for macOS (Apple Silicon) Compatibility and Playwright Stability

Aperta
#11 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
592
Fork
81
Merge medio
9g 22h
PR unite (30g)
1

Descrizione

# Enhancements for macOS (Apple Silicon) Compatibility and Playwright Stability

### **Summary**
When running MolmoWeb locally on macOS (M-series), users encounter a "float64 on MPS" runtime error and indefinite hangs during Playwright screenshots. I have implemented a set of surgical fixes to address these, along with minor bug fixes for missing imports and client-side validation.

---

### **System Environment**
- **Hardware:** Mac Studio M4 Max (128GB Unified Memory)
- **OS:** macOS Tahoe 26.4
- **Python:** 3.12.x
- **Models Tested:** MolmoWeb-4B and MolmoWeb-8B (HuggingFace versions)

---

### **Issue 1: MPS `float64` Crash**
The standard `torch` backend on macOS (MPS) does not support double precision (`float64`), but the current model/server implementation defaults to `cuda` or `cpu` and occasionally introduces `float64` tensors during inference.

#### **How to Reproduce**
1. Install MolmoWeb on an M-series Mac.
2. Start the server: `bash scripts/start_server.sh ./checkpoints/MolmoWeb-8B-HF 8001` (with `PREDICTOR_TYPE=hf`).
3. Run an inference query via `test_server.py`.

#### **Observed Error**
```text
Predictor error: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
```

---

### **Issue 2: Playwright Screenshot Hangs**
In headless mode on Apple Silicon, `page.screenshot()` frequently hangs indefinitely. This appears to be a known conflict between Chromium’s GPU acceleration and macOS frame synchronization.

#### **How to Reproduce**
1. Run any agent task using `MolmoWeb.run()` in local/headless mode.
2. Observe the process hanging at the first step (taking a screenshot).

#### **Observed Logs**
```text
playwright._impl._errors.TimeoutError: Page.screenshot: Timeout 30000ms exceeded.
Call log: taking page screenshot ... waiting for fonts to load...
```

---

### **Issue 3: Minor Bugs & UX**
1. **Missing Import:** `agent/model_backends.py` is missing `import os`, causing a `NameError`.
2. **Cryptic Error:** Initializing `MolmoWeb` without an endpoint results in:
`AttributeError: 'NoneType' object has no attribute 'reset'`

---

### **Proposed Fixes**
I have verified a set of fixes locally that resolve these issues:
1. **MPS Support:** Added `mps` device auto-detection and forced `float32` dtypes.
2. **Stability Flags:** Added `--disable-gpu` and `--force-cpu-draw` flags to `SimpleEnv`.
3. **Screenshot Robustness:** Set `PW_TEST_SCREENSHOT_NO_FONTS_READY=1` and implemented a Chrome DevTools Protocol (CDP) fallback for screenshots (which bypasses standard API hangs related to frame synchronization).

I have these fixes verified and running in a local branch. Let me know if I can help out by filing a PR or providing the diffs in the comments.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.