alibaba / alibaba/x-deeplearning

TDM严格按照文档,使用官方docker无法跑通

Open
#325 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
PureBasic
Stars
4.3k
Forks
1k
PR merge metrics
No merged PRs in 30d

Description

运行至`python train.py --run_mode=local --config=config.train.json`时,提示段错误:

![image](https://user-images.githubusercontent.com/29789135/74223693-3ac4e400-4cf2-11ea-9b3c-ae2146abd4f3.png)

```bash
> python train.py --run_mode=local --config=config.train.json
/usr/local/lib/python2.7/dist-packages/sklearn/utils/__init__.py:12: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
from .murmurhash import murmurhash3_32
config: {u'ps': {u'instance_num': 16, u'memory_m': 64000, u'gpu_cores': 0, u'cpu_cores': 16}, u'dependent_dirs': u'/home/disk4/tdm/tdm_ub_att_ubuntu', u'script': u'train.py', u'worker': {u'instance_num': 20, u'memory_m': 100000, u'gpu_cores': 2, u'cpu_cores': 46}, u'max_local_failover_times': 3, u'auto_rebalance': {u'enable': u'false'}, u'min_finish_worker_rate': 100, u'max_failover_times': 3, u'job_name': u'xdl_tdm', u'docker_image': u'registry.cn-hangzhou.aliyuncs.com/xdl/xdl:ubuntu-cpu-mxnet1.3', u'checkpoint': {u'output_dir': u'/user/paddle/chengmo/tdm/tdm_mock_train/checkpoint'}}
mv data/userbehavoir_tree.pb data/userbehavoir_tree.pb.bak
hadoop fs -get /user/paddle/chengmo/tdm/upload/data/userbehavoir_tree.pb data/userbehavoir_tree.pb
20/02/05 03:33:23 INFO fs.LibdfsLoader: Trying to load the libdfs library...
20/02/05 03:33:23 INFO fs.LibdfsLoader: Loaded the libdfs library
20/02/05 03:33:23 INFO fs.DFileSystem: Loaded the libdfs library
Load successfully, leaf node count:End init
Begin training
Segmentation fault
```

寻找其他issue,也出现了相同的问题且未解决,设置`DataIO(namenode="http://localhost:9000")`后报错相同。

希望开发者可以更新文档,或更新docker镜像(将符合xdl需求的hdfs置入),能够解决参照官方文档无法跑通demo的问题。

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the failure with `python train.py --run_mode=local --config=config.train.json` in the official Docker image, using the referenced `config.train.json`, `train.py`, and HDFS-backed `data/userbehavoir_tree.pb` setup. Check the reported NumPy/scikit-learn warning and the `DataIO(namenode="http://localhost:9000")` variant; done means the documented demo runs without a segmentation fault, or the documentation/image requirements clearly explain the supported setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, hadoop, python
Domain
devops, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.