orientechnologies / orientechnologies/orientdb

Unable to repair corrupted class

Open
#9,727 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
5k
Forks
868
Avg merge
15h 6m
Merged PRs (30d)
18

Description

OrientDB Version: 3.0.37
Java Version: Oracle JDK 1.8.0_251
OS: Linux CentOS 7

I'm using OrientDB to store IoT metrics. Because I've a high ingestion ratio, I've configure it to switch in read-only mode when storage free space is below 4GB via "storage.diskCache.diskFreeSpaceLimit" property.
Despite this when the DB operate the switch a table (a class) becomes corrupted.
After adding more space and restarting server, trying to read the last 10 inserted records an exception is logged (the full stack is attached at the end)

[OLocalPaginatedStorage]Exception `21D9DF3B` in storage `plocal:/wos1/orientdb/databases/iot_exp_26a`: 3.0.37 - Veloce (build 6a0e4724c10d51a0b19700fca46da8e41ae006f5, branch 3.0.x)
java.lang.NullPointerException
       at com.orientechnologies.orient.core.storage.cache.chm.AsyncReadCache.releaseFromRead(AsyncReadCache.java:290)
       at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurableComponent.releasePageFromRead(ODurableComponent.java:167)
       at com.orientechnologies.orient.core.storage.cluster.v1.OPaginatedClusterV1.internalReadRecord(OPaginatedClusterV1.java:562)
       at com.orientechnologies.orient.core.storage.cluster.v1.OPaginatedClusterV1.readRecord(OPaginatedClusterV1.java:534)
       at com.orientechnologies.orient.core.storage.cluster.v1.OPaginatedClusterV1.readRecord(OPaginatedClusterV1.java:517)
       at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doReadRecord(OAbstractPaginatedStorage.java:5472)
       at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.readRecord(OAbstractPaginatedStorage.java:2179)

While broken record is skipped on select, a NPE is fired when I try to insert a new record into the table.

CHECK DATABASE command reported no problem. I have however exec REPAIR DATABASE command with no luck.

Corrupted table is a simple class (it doesn't extends vertex or edge) and have mainly simple properties: string, long, embeddedMap of string.

I've also tried to attach the database to latest OrientDB 3.1.x and 3.2.x but no one is able to fix the table.

Table is using a single cluster.

The only way to bypass the problem was a full database export/import into a new one. This action takes hours because the database contains millions of records.

Expected behavior

Have the table corruption detected and repaired by CHECK DATABASE and REPAIR DATABASE commands.

The ideal solution will be to have a CHECK CLASS and REPAIR CLASS command to focus on the problematic class and save time (other classes could be bigger than the corrupted one)

Actual behavior

CHECK and REPAIR commands doesn't identify the problem and the table stay corrupted.

Steps to reproduce

I'm unable to programmatically replicate the problem in a newly created database. I've a full copy of the corrupted one I can use to test a fix against.

orientdb-error.txt

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the attached database and the stack-trace locations AsyncReadCache.java:290 and OPaginatedClusterV1.java:562, then compare the reported corruption with CHECK DATABASE and REPAIR DATABASE behavior. Done means the corruption is detected and repaired, or the proposed class-scoped CHECK CLASS and REPAIR CLASS behavior is clearly assessed.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.