alibaba / alibaba/x-deeplearning
TDMServing run failed
- Dominant language
- PureBasic
- Stars
- 4.3k
- Forks
- 1k
- PR merge metrics
- No merged PRs in 30d
Description
Hi @woso @lovickie ,
With the userbehavior data, I compiled the tdm and get the converted dense model `model_blaze model_blaze_optimized(model.dat)`. But when I use them in `TDMServing tdm_example`, there is a error below.
```
run log:
I0821 09:38:44.779547 18774 blaze_model.cpp:58] [item_tree_model] blaze load sparse model weight succ, file: test_data/model/blaze_model/sparse_qed
I0821 09:38:44.795930 18774 blaze_model.cpp:70] [item_tree_model] blaze load model succ file: test_data/model/blaze_model/model.dat
E0821 09:38:44.804941 18774 blaze_model.cpp:79] [item_tree_model] blaze can not get predictor
E0821 09:38:44.804960 18774 model_unit.cpp:120] [item_tree_model] init model failed
error log:
[ERROR] [2019-08-16 07:16:33] [4586] [/home/work/open-code/x-deeplearning/blaze/blaze/api/cpp_api/predictor_manager_impl.cc:157] Create Model Predictor failed, test_data/model/blaze_model/model.dat msg=[failed at workspace.h:90]. [/home/work/open-code/x-deeplearning/blaze/blaze/graph/workspace.h:90] input_name: %s data_type not definedblockgrad1
```
It seems that load `model.dat` failed, but I have found `blockgrad1` which exists in `model.dat` and also exists in `graph.txt` at
```
node {
name: "/MxnetBackendOp"
op: "MxnetBackendOp"
... {\n \"op\": \"BlockGrad\", \n \"name\": \"blockgrad1\", \n \"inputs\": [[179, 0, 0]]\n } ...
}
```
and the related `op` in txt-file `model.dat` are
```
op {
type: "Slice"
name: "slice_axis10"
input: "blockgrad1"
output: "slice_axis10"
arg {
name: "axis"
i: 1
}
arg {
name: "start"
i: 1
}
arg {
name: "end"
i: 2
}
}
op {
type: "ReduceSum"
name: "sum0"
input: "blockgrad1"
output: "sum0"
arg {
name: "axis"
i: 1
}
arg {
name: "keepdims"
i: 0
}
}
op {
type: "Mul"
name: "_mul2"
input: "log0"
input: "blockgrad1"
output: "_mul2"
}
external_input {
name: "blockgrad1"
dtype: kFloat
}
```
There is no `blockgrad1` name in `op`.
About these problems, How to fix them.
Thank you!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with blaze_model.cpp and model_unit.cpp, then trace predictor creation through predictor_manager_impl.cc and workspace.h. Reproduce the failure with TDMServing tdm_example and the supplied model.dat and graph.txt; done means the model loads without the blockgrad1 data-type error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100