bytedance / bytedance/CloudShuffleService
cannot run css cluster
- Dominant language
- Java
- Stars
- 260
- Forks
- 57
- PR merge metrics
- No merged PRs in 30d
Description
1. if I use all default settings, and start as `sbin/start-all.sh`, a Worker and a Master are running
but when I submit a Spark app, it throw `Caused by: java.lang.RuntimeException: replica num must less than worker num`
2. if I run zk mode by changing `conf/css-default.cnf` as:
~~~shell
css.zookeeper.address=MyZkIP:2181
css.worker.registry.type=zookeeper
~~~
and start as `sbin/start-workers.sh` or `sbin/start-worker.sh` or `sbin/start-all.sh`
it throw
~~~shell
com.bytedance.css.service.deploy.worker.Worker --host xxx07v.xxxx.net
failed to launch: nice -n 0 /yy/java8/bin/java -Xmx1024m -XX:MaxDirectMemorySize=4096m -Dcss.log.dir=/home/aa/css/logs -Dcss.log.filename=css-aa-worker-1.out -classpath /yyy/java8/lib:/home/aa/css/lib/* com.bytedance.css.service.deploy.worker.Worker --host xxx07v.xxxx.net
tail: 无法打开"/home/aa/css/logs/css-aa-worker-1.out" 读取数据: 没有那个文件或目录
full log in /home/aa/css/logs/css-aa-worker-1.out
~~~
3. if I deploy as the `README.md` and set 3 workers in `conf/workers` + zk mode and `start-workers.sh`
then entering my password of the 3 workers, it returns `pemission denied`
I am sure my password is correct.
Any suggestion? I think the `README` is ambiguous
Contributor guide
Research direction
Start with README.md, conf/css-default.cnf, conf/workers, and the sbin/start-all.sh, sbin/start-workers.sh, and sbin/start-worker.sh entry points. Reproduce the default and ZooKeeper-mode startup paths and the three reported errors; done means the deployment instructions and observed behavior are consistent, with successful worker startup and Spark application submission.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, shell, spark
- Domain
- devops, distributed-systems, documentation
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100