apache / apache/buildstream

Long running operations should be timed activities

Open
#1,261 12 comments 0 reactions 0 assignees View on GitHub
logging
Dominant language
Python
Stars
140
Forks
45
Avg merge
1d 3h
Merged PRs (30d)
6

Description

[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/1261)
In GitLab by [[Gitlab user @abderrahimk]](https://gitlab.com/abderrahimk) on Feb 9, 2020, 16:23

## Summary

Trying buildstream master, I found a couple instances where seems just hanging there without telling what it is doing.

## Steps to reproduce

Just run `bst build` on a "slow" machine (either actually slow, or with some IO heavy background task).

## What is the current bug behavior?

- [ ] When starting up, bst hangs a little before printing anything as it removes the extract folder
(side note: this isn't very good for parallel installability with bst 1)
- [ ] bst hangs after `SUCCESS Checking sources` before going on to the next step.

## What is the expected correct behavior?

A timed activity, and possibly progress report.

## Possible fixes

- For the first one: https://gitlab.com/BuildStream/buildstream/-/blob/0591b0c8b038aa9f054b295748f3423384711d50/src/buildstream/_context.py#L289
- For the second one: [Stream._enqueue_plan](https://gitlab.com/BuildStream/buildstream/-/blob/0591b0c8b038aa9f054b295748f3423384711d50/src/buildstream/_stream.py#L1323) loops over all elements (in [Queue.enqueue](https://gitlab.com/BuildStream/buildstream/-/blob/0591b0c8b038aa9f054b295748f3423384711d50/src/buildstream/_scheduler/queues/queue.py#L171)) and queries the CAS for every one of them (via [Element._has_all_sources_in_source_cache](https://gitlab.com/BuildStream/buildstream/-/blob/0591b0c8b038aa9f054b295748f3423384711d50/src/buildstream/element.py#L2166))

## Other relevant information

[//]: # (Add the BuildStream version. Substitute below the "x" by "1", "2" or the right version. Add more than one version if necessary)

* BuildStream version affected: master

----

Contributor guide

Open the contributing guide

Research direction

Start with the referenced code in src/buildstream/_context.py around line 289 and Stream._enqueue_plan in src/buildstream/_stream.py, following Queue.enqueue and Element._has_all_sources_in_source_cache. Reproduce with bst build on a slow or I/O-heavy machine. Done means the startup cleanup and source-cache checks report timed activities, with progress where appropriate.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.