alibaba / alibaba/x-deeplearning
TDM严格按照文档,使用官方docker无法跑通
- Dominant language
- PureBasic
- Stars
- 4.3k
- Forks
- 1k
- PR merge metrics
- No merged PRs in 30d
Description
运行至`python train.py --run_mode=local --config=config.train.json`时,提示段错误:

```bash
> python train.py --run_mode=local --config=config.train.json
/usr/local/lib/python2.7/dist-packages/sklearn/utils/__init__.py:12: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
from .murmurhash import murmurhash3_32
config: {u'ps': {u'instance_num': 16, u'memory_m': 64000, u'gpu_cores': 0, u'cpu_cores': 16}, u'dependent_dirs': u'/home/disk4/tdm/tdm_ub_att_ubuntu', u'script': u'train.py', u'worker': {u'instance_num': 20, u'memory_m': 100000, u'gpu_cores': 2, u'cpu_cores': 46}, u'max_local_failover_times': 3, u'auto_rebalance': {u'enable': u'false'}, u'min_finish_worker_rate': 100, u'max_failover_times': 3, u'job_name': u'xdl_tdm', u'docker_image': u'registry.cn-hangzhou.aliyuncs.com/xdl/xdl:ubuntu-cpu-mxnet1.3', u'checkpoint': {u'output_dir': u'/user/paddle/chengmo/tdm/tdm_mock_train/checkpoint'}}
mv data/userbehavoir_tree.pb data/userbehavoir_tree.pb.bak
hadoop fs -get /user/paddle/chengmo/tdm/upload/data/userbehavoir_tree.pb data/userbehavoir_tree.pb
20/02/05 03:33:23 INFO fs.LibdfsLoader: Trying to load the libdfs library...
20/02/05 03:33:23 INFO fs.LibdfsLoader: Loaded the libdfs library
20/02/05 03:33:23 INFO fs.DFileSystem: Loaded the libdfs library
Load successfully, leaf node count:End init
Begin training
Segmentation fault
```
寻找其他issue,也出现了相同的问题且未解决,设置`DataIO(namenode="http://localhost:9000")`后报错相同。
希望开发者可以更新文档,或更新docker镜像(将符合xdl需求的hdfs置入),能够解决参照官方文档无法跑通demo的问题。
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the failure with `python train.py --run_mode=local --config=config.train.json` in the official Docker image, using the referenced `config.train.json`, `train.py`, and HDFS-backed `data/userbehavoir_tree.pb` setup. Check the reported NumPy/scikit-learn warning and the `DataIO(namenode="http://localhost:9000")` variant; done means the documented demo runs without a segmentation fault, or the documentation/image requirements clearly explain the supported setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, hadoop, python
- Domain
- devops, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100