apache / apache/buildstream

Python exception when running buildstream twice in the same command

Open
#1,960 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
140
Forks
45
Avg merge
1d 3h
Merged PRs (30d)
6

Description

When I run `bst show` to show a diff between two projects (e.g. to do `diff <(bst show ...) <(bst show ...)`) I sometimes get exceptions - the stack trace of one is below.

I was using `-C` to switch to a different buildstream project, if that makes any difference.

I installed recently in a venv, through pip:

```
$ pip show buildstream
Name: BuildStream
Version: 2.3.0
Summary: A framework for modelling build pipelines in YAML
Home-page: https://buildstream.build
Author: The Apache Software Foundation
Author-email: dev@buildstream.apache.org
License: Apache License Version 2.0
Location: /home/john/src/project/.venv/lib64/python3.12/site-packages
Requires: Click, grpcio, Jinja2, pluginbase, protobuf, psutil, pyroaring, ruamel.yaml, ruamel.yaml.clib, setuptools, ujson
Required-by:
```

Traceback (modified only to hide project name):

```
Traceback (most recent call last):
File "/home/john/src/project/.venv/bin/bst", line 8, in
sys.exit(cli())
^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/buildstream/_frontend/cli.py", line 272, in override_main
original_main(self, args=args, prog_name=prog_name, complete_var=None, standalone_mode=standalone_mode, **extra)
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/click/decorators.py", line 45, in new_func
return f(get_current_context().obj, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/buildstream/_frontend/cli.py", line 631, in show
with app.initialized():
^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/contextlib.py", line 137, in __enter__
return next(self.gen)
^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/buildstream/_frontend/app.py", line 250, in initialized
self.context.artifactcache.preflight()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/buildstream/_context.py", line 482, in artifactcache
self._artifactcache = ArtifactCache(self)
^^^^^^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/buildstream/_artifactcache.py", line 36, in __init__
super().__init__(context)
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/buildstream/_assetcache.py", line 306, in __init__
self.cas: CASCache = context.get_cascache()
^^^^^^^^^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/buildstream/_context.py", line 709, in get_cascache
self._cascache = CASCache(self.cachedir, casd=self.get_casd(), remote_cache=bool(self.remote_cache_spec))
^^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/buildstream/_context.py", line 696, in get_casd
self._casd = CASDProcessManager(
^^^^^^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/buildstream/_cas/casdprocessmanager.py", line 109, in __init__
self._logfile = self._rotate_and_get_next_logfile()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/john/src/project/.venv/lib64/python3.12/site-packages/buildstream/_cas/casdprocessmanager.py", line 277, in _rotate_and_get_next_logfile
os.remove(os.path.join(self._log_dir, logfile_to_delete))
FileNotFoundError: [Errno 2] No such file or directory: '/home/john/.cache/buildstream/logs/_casd/1727856548.523361.log'
```

Contributor guide

Open the contributing guide

Research direction

Start in src/buildstream/_cas/casdprocessmanager.py, especially _rotate_and_get_next_logfile(), using the traceback as the entry point. Reproduce the failure by running bst show twice with -C as described, then verify that repeated commands no longer raise the reported FileNotFoundError while rotating CASD logs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.