Fix directory deletion in ExtractReuters, recover from errors [LUCENE-6073]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
ExtractReuters in the benchmark module currently fails because it currently creates the output directory, and then calls `IOUtils.rm` on it (which will remove all files in it as well as removes the output directory itself). This is to fix this behaviour.
While I was at it, I also added a bit more logging in case of file errors (the download I had some bad data) and made the task recover in case of issues with one file.
---
Migrated from [LUCENE-6073](https://issues.apache.org/jira/browse/LUCENE-6073) by Ramkumar Aiyengar (@andyetitmoves), updated Nov 30 2014
Contributor guide
Research direction
Start at the ExtractReuters benchmark entry point and inspect how it prepares the output directory before calling IOUtils.rm. Review the file-error handling and logging paths described in the issue, then run the relevant benchmark tests or task checks; done means directory extraction no longer removes its output directory and one bad file does not stop the task.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100