bazelbuild / bazelbuild/reclient
reclient doesn't cache any thing if -exec_root is set to any path other than root directory "/"
- Dominant language
- Go
- Stars
- 114
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
I'm invetigating why the cache hit rates is low on one of my project, and it seems like the source path may impact, as my contianer provider assigns me a random path everytime I apply for the build resources like /path/to/xxx, where xxx changes everytime.
Then to confirm this, I setup a local environment like below steps:
1. used opencv as example project, https://github.com/opencv/opencv
2. cloned the source code and placed it to /home/myname/ws/opencv
3. setup reclient environment. Configuration pasted at the end of this post.
4. generated make files: `cmake /home/myname/ws/opencv -DCMAKE_C_COMPILER_LAUNCHER=/path/to/reclient/rewrapper;-cfg=/path/to/reconfig;-exec_root=/ -DCMAKE_CXX_COMPILER_LAUNCHER=/path/to/reclient/rewrapper;-cfg=/path/to/reconfig;-exec_root=/ -B /home/myname/ws/opencv/mybuild`
5. Built opencv for the first time `make -j$(nproc)`, got `RBE Stats: down 3.15 MB, up 0 B, 174 cache hits, 1428 local executions`
6. Built opencv for the second time, got `RBE Stats: down 379.81 MB, up 217.31 KB, 1601 cache hits, 1 local execution` , which shown 100% cache hits
7. To invesitgate if random path impacts, I copied the whole source code from opencv folder to another folder /home/myname/ws/opencv2 and built, the result indicates the cache is not hit: `RBE Stats: down 3.15 MB, up 0 B, 174 cache hits, 1428 local executions`
8. To investigate if -exec_root impacts, I set different paths to exec_path. Set `-exec_root=/home/myname/ws/opencv`, no matter how many times I built the project it always executes locally , hit rates: `RBE Stats: down 3.15 MB, up 0 B, 174 cache hits, 1428 local executions`
Questions:
1. It seems that the path of source code impacts cache, I guess it is probably because cmake generates makefiles using absolute paths. How to address it if my guess is ture?
2. Why exec_root is set to the root(`-exec_root=/home/myname/ws/opencv` of the source code doesn't work as described in the document? But set to root of the file systems(`-exec_root=/`) works, why?
**Configuration**
```
service=remotecache:9980
instance=fuse
server_address=unix:///tmp/reproxy.sock
log_dir=/tmp/reclient
output_dir=/tmp/reclient/output
proxy_log_dir=/tmp/reclient/proxy
depsscanner_address=exec:///reclient/scandeps_server
service_no_auth=true
service_no_security=true
use_rpc_credentials=false
use_application_default_credentials=false
canonicalize_working_dir=false
exec_strategy=local
env_var_allowlist=INCLUDE,LIB,LIBPATH
labels=type=compile,lang=cpp,compiler=clang
reclient_timeout=30s
dump_input_tree=true
```
Contributor guide
Research direction
Reproduce the OpenCV builds described in the issue, comparing -exec_root=/ with -exec_root=/home/myname/ws/opencv and builds from /home/myname/ws/opencv2. Start by tracing reclient's exec_root and cache-key handling; done means identifying why the non-root setting misses the cache and either correcting that behavior or documenting the limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- build-system, devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100