Long-running tasks can show no output for a very long time

[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/1029)
In GitLab by [[Gitlab user @jonathanmaw]](https://gitlab.com/jonathanmaw) on May 22, 2019, 11:57

## Summary

There are places in buildstream's start-up where long-running tasks are performed, but no indication of progress is given, making it hard to distinguish a long-running task from a task that is stuck.

For example, we have the "Loading elements" and "Resolving elements" timed activities, where there can be a *very* long time between them starting and them finishing for a sufficiently large set of elements.

## Steps to reproduce

Find a sufficiently-large set of elements - https://gitlab.com/jennis/debian-stretch-bst is good for this purpose.
Load the pipeline for that set with `bst show`.

## What is the current bug behavior?

You will see a *very* long time is spent in "Loading elements" with no other output.

## What is the expected correct behavior?

Some kind of progress indicator would be preferred. A simple "N elements loaded" would be sufficient, but an "N out of M elements loaded" would be nicer.

## Possible fixes

For "loading elements" in particular, the timed_activity occurs in buildstream/_project.py:452 (in load_elements), but an element finishes loading in buildstream/_loader/loader.py:442 (in _collect_element). The length of a Loader's `_meta_elements` is not a useful count of how many elements have been loaded because multiple loaders will exist if junctions are used.

"Resolving elements" has the timed_activity in buildstream/_project.py:457, but an element finishes being resolved in buildstream/element.py:1004 (in _new_from_meta). Unfortunately, there is no list for every resolved element. The closest that exists is that each Element class has a dict of `__instantiated_elements`.

Since there are no good places for a list of current/total elements, the best ways to count progress would be:

1. have a counter in the Context
2. have a Progress object that gets passed down the call stack

Since printing output every time an element is loaded or resolved would consume a lot of resources for marginal benefit, a threshold should be specified for when to print (e.g. every N items, or every N seconds)

----
[//]: # (To review information about possible relevant labels for this issue please view the list of labels: https://gitlab.com/BuildStream/buildstream/labels)

Run `bst show` against the debian-stretch-bst pipeline to reproduce the silent periods. Read `buildstream/_project.py` around `load_elements` and `resolve_elements`, then follow loading in `buildstream/_loader/loader.py` (`_collect_element`) and resolving in `buildstream/element.py` (`_new_from_meta`). Done means long operations visibly report progress at a bounded interval or threshold without excessive output.

Written by the indexing model from the issue text.