bytedance / bytedance/CloudShuffleService

cannot run css cluster

Open
#5 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
260
Forks
57
PR merge metrics
No merged PRs in 30d

Description

1. if I use all default settings, and start as `sbin/start-all.sh`, a Worker and a Master are running
but when I submit a Spark app, it throw `Caused by: java.lang.RuntimeException: replica num must less than worker num`

2. if I run zk mode by changing `conf/css-default.cnf` as:
~~~shell
css.zookeeper.address=MyZkIP:2181
css.worker.registry.type=zookeeper
~~~
and start as `sbin/start-workers.sh` or `sbin/start-worker.sh` or `sbin/start-all.sh`
it throw
~~~shell
com.bytedance.css.service.deploy.worker.Worker --host xxx07v.xxxx.net
failed to launch: nice -n 0 /yy/java8/bin/java -Xmx1024m -XX:MaxDirectMemorySize=4096m -Dcss.log.dir=/home/aa/css/logs -Dcss.log.filename=css-aa-worker-1.out -classpath /yyy/java8/lib:/home/aa/css/lib/* com.bytedance.css.service.deploy.worker.Worker --host xxx07v.xxxx.net
tail: 无法打开"/home/aa/css/logs/css-aa-worker-1.out" 读取数据: 没有那个文件或目录
full log in /home/aa/css/logs/css-aa-worker-1.out
~~~

3. if I deploy as the `README.md` and set 3 workers in `conf/workers` + zk mode and `start-workers.sh`
then entering my password of the 3 workers, it returns `pemission denied`
I am sure my password is correct.

Any suggestion? I think the `README` is ambiguous

Contributor guide

Open the contributing guide

Research direction

Start with README.md, conf/css-default.cnf, conf/workers, and the sbin/start-all.sh, sbin/start-workers.sh, and sbin/start-worker.sh entry points. Reproduce the default and ZooKeeper-mode startup paths and the three reported errors; done means the deployment instructions and observed behavior are consistent, with successful worker startup and Spark application submission.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, shell, spark
Domain
devops, distributed-systems, documentation
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.