ModelEngine-Group / ModelEngine-Group/DataMate
datamate-frontend 冷启动时可能连不上 datamate-gateway;datamate-runtime 在 Docker 默认共享内存下可能 OOM
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 368
- Forks
- 46
- Avg merge
- 35m
- Merged PRs (30d)
- 4
Description
问题一:服务启动顺序缺失
deployment/docker/datamate/docker-compose.yml 里 datamate-frontend 的
depends_on 列出了 datamate-backend 和 datamate-backend-python,但没有
datamate-gateway。冷启动 docker-compose up 时,frontend 可能在 gateway
就绪之前就先启动了,导致 frontend 请求 gateway 失败,直到手动重启才恢复。
问题二:Ray 在默认 Docker 共享内存下 OOM
datamate-runtime 跑的是基于 Ray 的流水线,但没有设置 shm_size。Docker 默认
的 /dev/shm(64MB)远低于 Ray object store 实际需要的量——这是 Ray 部署里
一个众所周知的、会导致高负载下 worker 崩溃的原因。
建议修复方向
- 给 datamate-frontend 的 depends_on 加上 datamate-gateway。
- 给 datamate-runtime 加 shm_size: "10gb",并调整
RAY_DEFAULT_OBJECT_STORE_MEMORY_PROPORTION。
我已经准备好 compose 文件的修改,会提一个 PR。
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Open deployment/docker/datamate/docker-compose.yml and inspect the datamate-frontend and datamate-runtime service definitions first. Verify the dependency and shared-memory settings, then run docker compose config and a cold-start compose run; done means frontend can start with datamate-gateway available and the runtime has the requested Ray memory configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, docker-compose
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100