internetarchive / internetarchive/heritrix3
Bug in non-fatal-error log
- Dominant language
- Java
- Stars
- 3.3k
- Forks
- 793
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 8
Description
While resolving #158 I noticed that the resulting entry in the non-fatal-error log had redundant stacktraces. I.e. one exception triggered the following:
```
2016-05-02T12:58:56.239Z 401 4758 http://aktravel.is/en/fundir-og-radstefnur/framkvaemd-radstefnu - - text/html #001 20160502125856064+163 sha1:KKGWJBFE2H4XPVTWXRNIMTCEQTJ4N76N - -
java.lang.IllegalStateException: Missing auth challenge headers for uri with response status 401: http://aktravel.is/en/fundir-og-radstefnur/framkvaemd-radstefnu
at org.archive.modules.fetcher.FetchHTTP.extractChallenges(FetchHTTP.java:884)
at org.archive.modules.fetcher.FetchHTTP.handle401(FetchHTTP.java:802)
at org.archive.modules.fetcher.FetchHTTP.innerProcess(FetchHTTP.java:743)
at org.archive.modules.Processor.innerProcessResult(Processor.java:175)
at org.archive.modules.Processor.process(Processor.java:142)
at org.archive.modules.ProcessorChain.process(ProcessorChain.java:131)
at org.archive.crawler.framework.ToeThread.run(ToeThread.java:148)
java.lang.IllegalStateException: Missing auth challenge headers for uri with response status 401: http://aktravel.is/en/fundir-og-radstefnur/framkvaemd-radstefnu
at org.archive.modules.fetcher.FetchHTTP.extractChallenges(FetchHTTP.java:884)
at org.archive.modules.fetcher.FetchHTTP.handle401(FetchHTTP.java:802)
at org.archive.modules.fetcher.FetchHTTP.innerProcess(FetchHTTP.java:743)
at org.archive.modules.Processor.innerProcessResult(Processor.java:175)
at org.archive.modules.Processor.process(Processor.java:142)
at org.archive.modules.ProcessorChain.process(ProcessorChain.java:131)
at org.archive.crawler.framework.ToeThread.run(ToeThread.java:148)
```
Probably a bug somewhere in
[`org.archive.crawler.io.NonFatalErrorFormatter.format()`](https://github.com/internetarchive/heritrix3/blob/124907341f8b4b3e921c03721d77d52b33ea96e0/engine/src/main/java/org/archive/crawler/io/NonFatalErrorFormatter.java#L40)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.