internetarchive / internetarchive/heritrix3

Bug in non-fatal-error log

Open
#160 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.3k
Forks
793
Avg merge
1d 8h
Merged PRs (30d)
8

Description

While resolving #158 I noticed that the resulting entry in the non-fatal-error log had redundant stacktraces. I.e. one exception triggered the following:

```
2016-05-02T12:58:56.239Z 401 4758 http://aktravel.is/en/fundir-og-radstefnur/framkvaemd-radstefnu - - text/html #001 20160502125856064+163 sha1:KKGWJBFE2H4XPVTWXRNIMTCEQTJ4N76N - -
java.lang.IllegalStateException: Missing auth challenge headers for uri with response status 401: http://aktravel.is/en/fundir-og-radstefnur/framkvaemd-radstefnu
at org.archive.modules.fetcher.FetchHTTP.extractChallenges(FetchHTTP.java:884)
at org.archive.modules.fetcher.FetchHTTP.handle401(FetchHTTP.java:802)
at org.archive.modules.fetcher.FetchHTTP.innerProcess(FetchHTTP.java:743)
at org.archive.modules.Processor.innerProcessResult(Processor.java:175)
at org.archive.modules.Processor.process(Processor.java:142)
at org.archive.modules.ProcessorChain.process(ProcessorChain.java:131)
at org.archive.crawler.framework.ToeThread.run(ToeThread.java:148)
java.lang.IllegalStateException: Missing auth challenge headers for uri with response status 401: http://aktravel.is/en/fundir-og-radstefnur/framkvaemd-radstefnu
at org.archive.modules.fetcher.FetchHTTP.extractChallenges(FetchHTTP.java:884)
at org.archive.modules.fetcher.FetchHTTP.handle401(FetchHTTP.java:802)
at org.archive.modules.fetcher.FetchHTTP.innerProcess(FetchHTTP.java:743)
at org.archive.modules.Processor.innerProcessResult(Processor.java:175)
at org.archive.modules.Processor.process(Processor.java:142)
at org.archive.modules.ProcessorChain.process(ProcessorChain.java:131)
at org.archive.crawler.framework.ToeThread.run(ToeThread.java:148)
```

Probably a bug somewhere in

[`org.archive.crawler.io.NonFatalErrorFormatter.format()`](https://github.com/internetarchive/heritrix3/blob/124907341f8b4b3e921c03721d77d52b33ea96e0/engine/src/main/java/org/archive/crawler/io/NonFatalErrorFormatter.java#L40)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.