Crash due to AttributeError when using remote execution
- Dominant language
- Python
- Stars
- 140
- Forks
- 45
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 6
Description
[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/1315)
In GitLab by [[Gitlab user @scott]](https://gitlab.com/scott).clarke on May 20, 2020, 13:41
## Summary
[//]: # (Summarize the bug encountered concisely)
I experienced a build failure due to and `AttributeError`, shown below, when using remote execution and having an artifact server defined.
## Steps to reproduce
[//]: # (How one can reproduce the issue - this is very important)
I saw the failure when doing a remote execution build of [this element](https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/blob/valentindavid/bst2/elements/bootstrap-import.bst) with my `project.conf` contaning the lines shown below. I only saw this error when the artifact cache was defined, if commented out this didn't occur.
## What is the current bug behavior?
[//]: # (What actually happens)
Buildstream crashes with an `AttributeError` as seen below.
## What is the expected correct behavior?
[//]: # (What you should see instead)
Buildstream shouldn't crash.
## Relevant logs and/or screenshots
[//]: # (Paste any relevant logs - please use code blocks ``` to format console output, logs, and code as it's hard to read otherwise. You can also add a Snippet and link it here. Check the markdown giude on Gitlab for further tips)
```
An unhandled exception occured:
Traceback (most recent call last):
File "/usr/local/lib64/python3.7/site-packages/buildstream/_scheduler/jobs/job.py", line 709, in child_action
result = self.child_process() # pylint: disable=assignment-from-no-return
File "/usr/local/lib64/python3.7/site-packages/buildstream/_scheduler/jobs/elementjob.py", line 103, in child_process
return self._action_cb(self._element)
File "/usr/local/lib64/python3.7/site-packages/buildstream/_scheduler/queues/buildqueue.py", line 93, in _assemble_element
return element._assemble()
File "/usr/local/lib64/python3.7/site-packages/buildstream/element.py", line 1572, in _assemble
collect = self.assemble(sandbox) # pylint: disable=assignment-from-no-return
File "/usr/lib64/python3.7/contextlib.py", line 119, in __exit__
next(self.gen)
File "/usr/local/lib64/python3.7/site-packages/buildstream/sandbox/sandbox.py", line 328, in batch
batch.execute()
File "/usr/local/lib64/python3.7/site-packages/buildstream/sandbox/_sandboxreapi.py", line 236, in execute
if self.sandbox.run(["sh", "-c", "-e", self.script], self.flags, cwd=first.cwd, env=first.env) != 0:
File "/usr/local/lib64/python3.7/site-packages/buildstream/sandbox/sandbox.py", line 282, in run
return self._run(command, flags, cwd=cwd, env=env)
File "/usr/local/lib64/python3.7/site-packages/buildstream/sandbox/_sandboxreapi.py", line 106, in _run
action_result = self._execute_action(action, flags) # pylint: disable=assignment-from-no-return
File "/usr/local/lib64/python3.7/site-packages/buildstream/sandbox/_sandboxremote.py", line 330, in _execute_action
artifactcache.fetch_missing_blobs(project, local_missing_blobs)
File "/usr/local/lib64/python3.7/site-packages/buildstream/_artifactcache.py", line 408, in fetch_missing_blobs
missing_blobs = self.cas.fetch_blobs(remote, missing_blobs, allow_partial=True)
File "/usr/local/lib64/python3.7/site-packages/buildstream/_cas/cascache.py", line 670, in fetch_blobs
batch.add(digest)
File "/usr/local/lib64/python3.7/site-packages/buildstream/_cas/casremote.py", line 103, in add
self._request.instance_name = self._remote.local_cas_instance_name
AttributeError: 'ArtifactRemote' object has no attribute 'local_cas_instance_name'
```
```
artifacts:
url: https://public.aws.celduin.co.uk
remote-execution:
execution-service:
url: https://push.cache.com:30443
instance-name: remote-execution
server-cert: server.crt
client-cert: push.crt
client-key: push.key
storage-service:
url: https://push.cache.com:30443
instance-name: remote-execution
server-cert: server.crt
client-cert: push.crt
client-key: push.key
action-cache-service:
url: https://push.cache.com:30443
instance-name: remote-execution
server-cert: server.crt
client-cert: push.crt
client-key: push.key
```
## Possible fixes
[//]: # (If you can, link to the line of code that might be responsible for the problem)
## Other relevant information
[//]: # (Add the BuildStream version. Substitute below the "x" by "1", "2" or the right version. Add more than one version if necessary)
* BuildStream version affected:
This happened in a container built from the nightly docker image, the latest buildstream commit at the time of building was: 99063c732b56fb92f6c8b8c32bdb66eb4fb015e6
```
/src> bst --version
1.93.3+76.g99063c73
```
----
Contributor guide
Research direction
Reproduce the failure with the shown project.conf remote-execution and artifacts settings, then trace the stack from sandbox/_sandboxremote.py through _artifactcache.py, _cas/cascache.py, and _cas/casremote.py. Compare the remote artifact-cache objects involved and verify that a remote-execution build with an artifact server no longer crashes with AttributeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100