Angel-ML / Angel-ML/angel

可以提供一些LDA参数的建议么

Open
#766 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.8k
Forks
1.6k
Avg merge
44m
Merged PRs (30d)
1

Description

```
sh ./angel-submit \
-Daction.type=train \
-Dangel.app.submit.class=com.tencent.angel.ml.lda.LDARunner \
-Dml.model.class.name=com.tencent.angel.ml.lda.LDAModel \
-Dangel.train.data.path="hdfs://jr-hdfs//tmp/wangqinghua/lda/angel_test/data/trainData" \
-Dangel.log.path="hdfs://jr-hdfs//tmp/wangqinghua/lda/angel_test/log" \
-Dangel.save.model.path="hdfs://jr-hdfs//tmp/wangqinghua/lda/angel_test/model" \
-Dsave.doc.topic.distribution=true \
-Dsave.topic.word.distribution=true \
-Dsave.doc.topic=true \
-Dsave.word.topic=true \
-Dml.lda.word.num=33404450 \
-Dml.lda.topic.num=100 \
-Dsave.word.topic=true \
-Dml.epoch.num=20 \
-Dml.data.type=dummy \
-Dml.feature.index.range=1024 \
-Dangel.job.name=LDAtest \
-Dangel.am.memory.gb=20 \
-Dangel.worker.memory.gb=2 \
-Dangel.ps.memory.gb=2 \
-Dangel.staging.dir="hdfs://jr-hdfs//tmp/wangqinghua/lda/angel_test/stage" \
--queue datamin.default \
-Dangel.output.path.deleteonexist=true \
-Dangel.workergroup.number=20 \
-Dangel.ps.number=20 \
-Dangel.ps.cpu.vcores=15 \
-Dangel.am.cpu.vcores=28 \
-Dangel.am.java.opts -Xmx8192m \
-Dangel.ps.java.opts -Xm8192m
```
目前参数如下,总训练数据为9000W条,大约1.6T
希望能提供一些建议

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.