not respecting showDupeCount=true; retry without --uniques-only
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.2k
- Forks
- 214
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I'm new to the tool, and don't want to download empty files or files which haven't changed. I tried and got the following. I'm not sure what this means and why it doesn't work...?
❯ waybackpack http://reddit.com/r/self -d ~/Downloads/wayback-reddit --from-date 2008 --to-date 2009 --no-clobber --progress --uniques-only
Traceback (most recent call last):
File "/Users/reagle/.pyenv/versions/3.12.5/bin/waybackpack", line 8, in <module>
sys.exit(main())
^^^^^^
File "/Users/reagle/.pyenv/versions/3.12.5/lib/python3.12/site-packages/waybackpack/cli.py", line 142, in main
snapshots = search(
^^^^^^^
File "/Users/reagle/.pyenv/versions/3.12.5/lib/python3.12/site-packages/waybackpack/cdx.py", line 47, in search
raise WaybackpackException(
waybackpack.cdx.WaybackpackException: Wayback Machine CDX API not respecting showDupeCount=true; retry without --uniques-only.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the failing command and the traceback path through waybackpack/cli.py and waybackpack/cdx.py, especially the search call. Reproduce it with --uniques-only and without that option, then compare the CDX API responses and existing behavior. Done means the reported failure is understood and the command handles the API response without raising the shown exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100