google / google/ml-compiler-opt
TFRecord generator not working
- Dominant language
- Python
- Stars
- 798
- Forks
- 123
- Avg merge
- 55m
- Merged PRs (30d)
- 4
Description
Thanks for the opensource code!
I have encountered several problems while trying to preproduce the model following the README file:
1. I will really appreciate it if you can provide a explanation on how to use the `extract_ir.py` tool.
Currently I am compiling for the .bc code from source .c file using `clang –c –emit-llvm –O1 X.c`, and using an empty file for X.cmd.
While running generate_default_trace.py, the clang command in inclining_runner.py (i.e. clang +cmds -mllvm -enable-ml-inliner=development ... at line 119) does not generate any log file. I have tried to run the clang command outside of the script, but still no log file is generated. I wish to know if that is caused by having something empty for cmds? If yes, it might be useful to provide some instructions on how to use extract_ir.py to generate a correct .cmd file. If no, that might be worth figuring out why clang is not generating log files.
I get around the log file problem by breaking the clang into two commands:
Firstly use`opt -passes=scc-oz-module-inliner -ml-inliner-ir2native-model= -training-log= -enable-ml-inliner=development -o ` to generate the log file
Then run clang to compile for the native.
I am not sure if this is a good replacement for the original code, and will really appreciate it if you can enlighten me on what's wrong with it.
2. It seems like this script is requiring cuda10, while the others are using cuda11.
3. On running train_bc.py, it hangs forever on
```
experience = next(dataset_iter) (in trainer.py)
```
At a closer at `dataset_iter`, we find that when we read from the TFRecord file in `_file_iterator_fn `(defined in `data_reader.py`), the dataset is actually empty.
I am not sure if it is the problem with how we read the file, or the TFRecord file generated from previous step.
It will be really helpful if you can give some insights on these points!
Contributor guide
Assessment
This issue has not been assessed yet.