allenai / allenai/molmoweb

Enhancements for macOS (Apple Silicon) Compatibility and Playwright Stability

オープン
#11 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Python
スター
592
フォーク
81
平均マージ
9日 22時間
マージ済み PR(30日)
1

説明

# Enhancements for macOS (Apple Silicon) Compatibility and Playwright Stability

### **Summary**
When running MolmoWeb locally on macOS (M-series), users encounter a "float64 on MPS" runtime error and indefinite hangs during Playwright screenshots. I have implemented a set of surgical fixes to address these, along with minor bug fixes for missing imports and client-side validation.

---

### **System Environment**
- **Hardware:** Mac Studio M4 Max (128GB Unified Memory)
- **OS:** macOS Tahoe 26.4
- **Python:** 3.12.x
- **Models Tested:** MolmoWeb-4B and MolmoWeb-8B (HuggingFace versions)

---

### **Issue 1: MPS `float64` Crash**
The standard `torch` backend on macOS (MPS) does not support double precision (`float64`), but the current model/server implementation defaults to `cuda` or `cpu` and occasionally introduces `float64` tensors during inference.

#### **How to Reproduce**
1. Install MolmoWeb on an M-series Mac.
2. Start the server: `bash scripts/start_server.sh ./checkpoints/MolmoWeb-8B-HF 8001` (with `PREDICTOR_TYPE=hf`).
3. Run an inference query via `test_server.py`.

#### **Observed Error**
```text
Predictor error: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
```

---

### **Issue 2: Playwright Screenshot Hangs**
In headless mode on Apple Silicon, `page.screenshot()` frequently hangs indefinitely. This appears to be a known conflict between Chromium’s GPU acceleration and macOS frame synchronization.

#### **How to Reproduce**
1. Run any agent task using `MolmoWeb.run()` in local/headless mode.
2. Observe the process hanging at the first step (taking a screenshot).

#### **Observed Logs**
```text
playwright._impl._errors.TimeoutError: Page.screenshot: Timeout 30000ms exceeded.
Call log: taking page screenshot ... waiting for fonts to load...
```

---

### **Issue 3: Minor Bugs & UX**
1. **Missing Import:** `agent/model_backends.py` is missing `import os`, causing a `NameError`.
2. **Cryptic Error:** Initializing `MolmoWeb` without an endpoint results in:
`AttributeError: 'NoneType' object has no attribute 'reset'`

---

### **Proposed Fixes**
I have verified a set of fixes locally that resolve these issues:
1. **MPS Support:** Added `mps` device auto-detection and forced `float32` dtypes.
2. **Stability Flags:** Added `--disable-gpu` and `--force-cpu-draw` flags to `SimpleEnv`.
3. **Screenshot Robustness:** Set `PW_TEST_SCREENSHOT_NO_FONTS_READY=1` and implemented a Chrome DevTools Protocol (CDP) fallback for screenshots (which bypasses standard API hangs related to frame synchronization).

I have these fixes verified and running in a local branch. Let me know if I can help out by filing a PR or providing the diffs in the comments.

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。