internetarchive / internetarchive/heritrix3

-50 NOTCRAWLED https://podcrto.si/

Open
#258 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
3.3k
Forks
793
Avg merge
1d 8h
Merged PRs (30d)
8

Description

hi,

I try to harvest this site: https://podcrto.si
As National Library we harvest several domains to preserve the information.
I tried with Heritrix 1.14.4 and 3.4 but without success.

I'm getting this:

[code] [status] [seed] [redirect]
-50 NOTCRAWLED https://podcrto.si/
200 CRAWLED https://e-uprava.gov.si/

and

LONGEST#2:
Queue si,podcrto, (p3)
2 items
wakes in: 13m45s77ms
last enqueued: https://podcrto.si/robots.txt
last peeked: https://podcrto.si/robots.txt
total expended: 15 (total budget: -1)
active balance: 2985
last(avg) cost: 1(1)
totalScheduled fetchSuccesses fetchFailures fetchDisregards fetchResponses robotsDenials successBytes totalBytes fetchNonResponses lastSuccessTime
3 1 0 0 1 0 54 54 16 2019-05-23T07:14:29.825Z
SimplePrecedenceProvider
3

Can anyone help or explain what could be the reason for this?
Thank you in advance.

Best,
Matjaž

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.