Improve AlreadyClosedException logging [LUCENE-10005]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
If we close the IndexWriter because of a tragic exception and later try to write into the index we got AlreadyClosedException exception like this:
```
2021-05-17 11:35:04,963 ERROR (qtp626742236-67894)----x:ct_returns_shard8_replica_n28-o.a.s.s.HttpSolrCall: null:org.apache.solr.common.SolrException: Error handling 'MERGEINDEXES' action
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:365)
...
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:671)
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:685)
...
Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1877842388-10.181.61.226-1618590425462:blk_1075816117_2075498 file=/solr/ct_returns/core_node31/data/index/_1z_Lucene50_0.doc
at org.apache.hadoop.hdfs.DFSInputStream.refetchLocations(DFSInputStream.java:880)
at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:863)
at
```
This is rather misleading because 2021-05-17 is the date of the AlreadyClosedException, but the tragic exception (BlockMissingException in this case) could be much earlier. This is the exception we saved when we had to close the indexwriter.
We should emphasise that the tragic exception is not a fresh exception, but an exception we saved earlier.
---
Migrated from [LUCENE-10005](https://issues.apache.org/jira/browse/LUCENE-10005) by Andras Salamon (@asalamon74), updated Jun 17 2021
Contributor guide
Research direction
Start by tracing IndexWriter.ensureOpen and the path that saves the tragic exception when the IndexWriter closes. Reproduce or inspect the reported AlreadyClosedException sequence, then verify that logging distinguishes the later closed-writer error from the earlier saved BlockMissingException. Done means the log makes clear that the tragic exception was saved earlier rather than freshly raised.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100