apple / apple/foundationdb

DataDistribution: internal_error

Open
#12,684 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

We are currently seeing this error with Data Distributor
```
trace.10.167.69.22.4592.1770482233.Nvuo9Y.0.2.xml:
/mnt/logs/fdb/trace.10.167.69.22.4592.1770482233.Nvuo9Y.0.2.xml:
```

Any idea on steps we can take to recover the data distributor? It just keeps crash looping and isn't able to do any work. We are fine with losing a small amount of data (the shard) if needed vs losing the cluster.
We are also running fdb 7.3.69

Contributor guide

Open the contributing guide

Research direction

Start with the Data Distributor trace and inspect foundationdb/fdbclient/KeyRangeMap.actor.cpp around line 297, using the reported assertion and backtrace as the entry point. Compare the failure with the 7.3.69 behavior and surrounding error handling. Done means identifying a reproducible cause and documenting a validated recovery path that avoids cluster loss, or clearly establishing whether shard loss is required.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.