AnswerDotAI / AnswerDotAI/fastprogress
Is there a way to deal with variable sized iterables?
- Dominant language
- Jupyter Notebook
- Stars
- 1.1k
- Forks
- 104
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
The problem is this line:
`self.total = len(gen) if total is None else total`
That is executed at the beginning. But `len(gen)` might change throughout the iteration, but progress_bar just assumes it keeps still.
What I really want is to use a timer (so "train for 1 hour" instead of "train for so and so epochs").
```python
import time
class Timer:
def __init__(self, num_seconds):
self.num_seconds = num_seconds
self.i = None
def __len__(self):
if self.i is None: return 1
eps = 1e-7
t = time.time() - self.start
progress = t/num_seconds
return int((num_seconds*self.i)/(progress+eps)) # estimated size. Changes constantly.
def __iter__(self):
self.start = time.time()
self.i, t = 0, 0.0
while t <= self.num_seconds:
t = time.time() - self.start
self.i += 1
yield None
```
With this class, if you write
```python
for t in Timer(3.0):
# do stuff
```
it runs for 3 seconds. But it doesn't work with the `progress_bar` :(
Contributor guide
Assessment
This issue has not been assessed yet.