althonos / althonos/InstaLooter

Instagram limitations \ Time Filtering \ Large Instagram Sets

Open
#206 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2.1k
Forks
257
PR merge metrics
No merged PRs in 30d

Description

## Library version
* instalooter v2.0.3

## Environment
* Windows 10 Pro x64 (1803)
* Python 3.6.0

## Error description - runtime

I see there have been several recent issues discussing the new limitations imposed by Instagram effectively preventing this application from working on large sets of data. One of the recommendations was to use the "-t" option to limit the number of images downloaded in a session. It appears that simply loading all of the image info counts towards the limit and thus even using the "-t" option will not allow downloading of any photos beyond the imposed limit.

On a side note, the "-t" worked fine for me as I originally did "-t 2017-12-31:2017-01-01" and only got 2017 photos with that job and tried again with "-t 2016-12-31:2016-01-01" and only got 2016 photos until I hit the 1000 photo limit.

In my case, assuming a "page" was 50 items (or so I saw in the code), trying to grab older photos from an account would consistently give me the "Query rate exceeded" around page 20 (which would be 1000 photos) and 0 photos would be downloaded. (I was able to successfully process those original photos in previous runs)

**One solution I see, would be to have an option (if even possible) to start the page count at a certain number. (Or build the same queue and ignore the first "x" pages in the queue)** Therefore, if I run it once and it successfully processes 20 pages, I could wait for the cool-off period (I am seeing 30 minutes) and then restart the process starting at page 19 or something. This would remove the burden from InstaLooter from tracking\recording the progress and retrying over time starting at the last known location.

### Reproducible test case

- Find a user with 1000+ images
- Try to download those images with or without the time filter on the command line

### Expected behaviour

Allow an option to start a new run at a different starting page than 1, or have InstaLooter save the progress and continually retry (set a timeout retry value) in order to maintain progress and not redo all of the photos previously processed.

### Actual behaviour

Application can not get beyond around 1000 items with or without the time filter option.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.