apache / apache/accumulo

Accumulo offline scanner is not applying time set via bulk import

Open
#3,270 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
1.2k
Forks
487
Avg merge
4d 5h
Merged PRs (30d)
13

Description

**Describe the bug**

Bulk imports can optionally set a time that will be applied to all keys in the bulk imported files. This timestamp is applied lazily by a low level iterator whenever a bulk import file is read for scan or compaction. While working on #3259 it was discovered the [OfflineScanner](https://github.com/apache/accumulo/blob/8c65e026be2db633d8f2a622aabf05db1c2d6c15/core/src/main/java/org/apache/accumulo/core/clientImpl/OfflineIterator.java#L271-L329) does not appear to lazily apply these timestamps. The OfflineScanner is only used by the Accumulo map reduce when reading an offline table at which point it will directly read the files for a tablet.

**Expected behavior**

The internal OfflineScanner code should lazily apply the timestamps using the TimeSettingIterator and the timestamp from the files entry in the metadata table.

Contributor guide

Open the contributing guide

Research direction

Start with the OfflineScanner code in core/src/main/java/org/apache/accumulo/core/clientImpl/OfflineIterator.java, then trace how bulk-import file timestamps are recorded in the metadata table and how TimeSettingIterator is used. Done means offline scans lazily apply the file-entry timestamp to keys from bulk-imported files.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.