IllegalStateException from CharsetDecoder.flush in RootConditionalWriter.write
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 487
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 13
Description
During local testing of #4970 I saw the following error during or right after the cluster started.
```
java.lang.RuntimeException: java.lang.IllegalStateException: Current state = CODING_END, new state = FLUSHED
at org.apache.accumulo.server.metadata.RootConditionalWriter.write(RootConditionalWriter.java:132) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.server.metadata.RootConditionalWriter.write(RootConditionalWriter.java:67) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.server.metadata.ConditionalTabletsMutatorImpl.writeMutations(ConditionalTabletsMutatorImpl.java:172) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.server.metadata.ConditionalTabletsMutatorImpl.process(ConditionalTabletsMutatorImpl.java:228) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.manager.compaction.coordinator.CompactionCoordinator.reserveCompaction(CompactionCoordinator.java:584) ~[accumulo-manager-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.manager.compaction.coordinator.CompactionCoordinator.getCompactionJob(CompactionCoordinator.java:392) ~[accumulo-manager-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.base/java.lang.reflect.Method.invoke(Method.java:569) ~[?:?]
at org.apache.accumulo.core.trace.TraceUtil.lambda$wrapService$0(TraceUtil.java:202) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at jdk.proxy2/jdk.proxy2.$Proxy35.getCompactionJob(Unknown Source) ~[?:?]
at org.apache.accumulo.core.compaction.thrift.CompactionCoordinatorService$Processor$getCompactionJob.getResult(CompactionCoordinatorService.java:662) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.core.compaction.thrift.CompactionCoordinatorService$Processor$getCompactionJob.getResult(CompactionCoordinatorService.java:639) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:40) ~[libthrift-0.17.0.jar:0.17.0]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:40) ~[libthrift-0.17.0.jar:0.17.0]
at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:147) ~[libthrift-0.17.0.jar:0.17.0]
at org.apache.accumulo.server.rpc.TimedProcessor.process(TimedProcessor.java:50) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:492) ~[libthrift-0.17.0.jar:0.17.0]
at org.apache.accumulo.server.rpc.CustomNonBlockingServer$CustomFrameBuffer.invoke(CustomNonBlockingServer.java:129) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.thrift.server.Invocation.run(Invocation.java:18) ~[libthrift-0.17.0.jar:0.17.0]
at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.base/java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: java.lang.IllegalStateException: Current state = CODING_END, new state = FLUSHED
at java.base/java.nio.charset.CharsetDecoder.throwIllegalStateException(CharsetDecoder.java:996) ~[?:?]
at java.base/java.nio.charset.CharsetDecoder.flush(CharsetDecoder.java:679) ~[?:?]
at java.base/java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:808) ~[?:?]
at org.apache.accumulo.core.metadata.schema.RootTabletMetadata.bytesToUtf8(RootTabletMetadata.java:123) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.core.metadata.schema.RootTabletMetadata.lambda$update$5(RootTabletMetadata.java:162) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.base/java.util.Arrays$ArrayList.forEach(Arrays.java:4204) ~[?:?]
at org.apache.accumulo.core.metadata.schema.RootTabletMetadata.update(RootTabletMetadata.java:161) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.server.metadata.RootConditionalWriter.lambda$write$0(RootConditionalWriter.java:114) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.core.fate.zookeeper.ZooReaderWriter.lambda$mutateExisting$7(ZooReaderWriter.java:221) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.core.fate.zookeeper.ZooReader.retryLoopMutator(ZooReader.java:184) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.core.fate.zookeeper.ZooReaderWriter.mutateExisting(ZooReaderWriter.java:217) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.server.metadata.RootConditionalWriter.write(RootConditionalWriter.java:100) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
```
Contributor guide
Research direction
Start with RootTabletMetadata.bytesToUtf8 and update at the lines shown in the stack trace, then follow the call from RootConditionalWriter.write during cluster startup. Reproduce the failure around compaction coordination and verify that the CharsetDecoder state exception no longer occurs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100