AnswerDotAI / AnswerDotAI/fastprogress

Is there a way to deal with variable sized iterables?

Open
#64 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1.1k
Forks
104
Avg merge
1m
Merged PRs (30d)
1

Description

The problem is this line:
`self.total = len(gen) if total is None else total`

That is executed at the beginning. But `len(gen)` might change throughout the iteration, but progress_bar just assumes it keeps still.

What I really want is to use a timer (so "train for 1 hour" instead of "train for so and so epochs").

```python
import time
class Timer:
def __init__(self, num_seconds):
self.num_seconds = num_seconds
self.i = None

def __len__(self):
if self.i is None: return 1
eps = 1e-7
t = time.time() - self.start
progress = t/num_seconds
return int((num_seconds*self.i)/(progress+eps)) # estimated size. Changes constantly.

def __iter__(self):
self.start = time.time()
self.i, t = 0, 0.0
while t <= self.num_seconds:
t = time.time() - self.start
self.i += 1
yield None
```

With this class, if you write
```python
for t in Timer(3.0):
# do stuff
```
it runs for 3 seconds. But it doesn't work with the `progress_bar` :(

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.