apache / apache/accumulo

IllegalStateException from CharsetDecoder.flush in RootConditionalWriter.write

Open
#4,971 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
1.2k
Forks
487
Avg merge
4d 5h
Merged PRs (30d)
13

Description

During local testing of #4970 I saw the following error during or right after the cluster started.

```
java.lang.RuntimeException: java.lang.IllegalStateException: Current state = CODING_END, new state = FLUSHED
at org.apache.accumulo.server.metadata.RootConditionalWriter.write(RootConditionalWriter.java:132) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.server.metadata.RootConditionalWriter.write(RootConditionalWriter.java:67) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.server.metadata.ConditionalTabletsMutatorImpl.writeMutations(ConditionalTabletsMutatorImpl.java:172) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.server.metadata.ConditionalTabletsMutatorImpl.process(ConditionalTabletsMutatorImpl.java:228) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.manager.compaction.coordinator.CompactionCoordinator.reserveCompaction(CompactionCoordinator.java:584) ~[accumulo-manager-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.manager.compaction.coordinator.CompactionCoordinator.getCompactionJob(CompactionCoordinator.java:392) ~[accumulo-manager-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.base/java.lang.reflect.Method.invoke(Method.java:569) ~[?:?]
at org.apache.accumulo.core.trace.TraceUtil.lambda$wrapService$0(TraceUtil.java:202) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at jdk.proxy2/jdk.proxy2.$Proxy35.getCompactionJob(Unknown Source) ~[?:?]
at org.apache.accumulo.core.compaction.thrift.CompactionCoordinatorService$Processor$getCompactionJob.getResult(CompactionCoordinatorService.java:662) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.core.compaction.thrift.CompactionCoordinatorService$Processor$getCompactionJob.getResult(CompactionCoordinatorService.java:639) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:40) ~[libthrift-0.17.0.jar:0.17.0]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:40) ~[libthrift-0.17.0.jar:0.17.0]
at org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:147) ~[libthrift-0.17.0.jar:0.17.0]
at org.apache.accumulo.server.rpc.TimedProcessor.process(TimedProcessor.java:50) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:492) ~[libthrift-0.17.0.jar:0.17.0]
at org.apache.accumulo.server.rpc.CustomNonBlockingServer$CustomFrameBuffer.invoke(CustomNonBlockingServer.java:129) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.thrift.server.Invocation.run(Invocation.java:18) ~[libthrift-0.17.0.jar:0.17.0]
at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.base/java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: java.lang.IllegalStateException: Current state = CODING_END, new state = FLUSHED
at java.base/java.nio.charset.CharsetDecoder.throwIllegalStateException(CharsetDecoder.java:996) ~[?:?]
at java.base/java.nio.charset.CharsetDecoder.flush(CharsetDecoder.java:679) ~[?:?]
at java.base/java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:808) ~[?:?]
at org.apache.accumulo.core.metadata.schema.RootTabletMetadata.bytesToUtf8(RootTabletMetadata.java:123) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.core.metadata.schema.RootTabletMetadata.lambda$update$5(RootTabletMetadata.java:162) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at java.base/java.util.Arrays$ArrayList.forEach(Arrays.java:4204) ~[?:?]
at org.apache.accumulo.core.metadata.schema.RootTabletMetadata.update(RootTabletMetadata.java:161) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.server.metadata.RootConditionalWriter.lambda$write$0(RootConditionalWriter.java:114) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.core.fate.zookeeper.ZooReaderWriter.lambda$mutateExisting$7(ZooReaderWriter.java:221) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.core.fate.zookeeper.ZooReader.retryLoopMutator(ZooReader.java:184) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.core.fate.zookeeper.ZooReaderWriter.mutateExisting(ZooReaderWriter.java:217) ~[accumulo-core-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
at org.apache.accumulo.server.metadata.RootConditionalWriter.write(RootConditionalWriter.java:100) ~[accumulo-server-base-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]

```

Contributor guide

Open the contributing guide

Research direction

Start with RootTabletMetadata.bytesToUtf8 and update at the lines shown in the stack trace, then follow the call from RootConditionalWriter.write during cluster startup. Reproduce the failure around compaction coordination and verify that the CharsetDecoder state exception no longer occurs.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.