Re-evaluate metadata tables locality groups
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 487
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 13
Description
While running a test of the new external compaction feature I Was looking at the metadata table config. External compactions add a new column family to the metadata table called ecomp. I noticed this new column family was not in the metadata table locality group config. Not sure if it should be. I think these configurations are pretty old and there may be other new column families. Would be good to reassess the locality group config against current column families and metadata table query patterns.
```
root@muchos> config -t accumulo.metadata -f group
-----------+-------------------------+------------------------------------------
SCOPE | NAME | VALUE
-----------+-------------------------+------------------------------------------
table | table.group.server .... | file,log,srv,future
table | table.group.tablet .... | ~tab,loc
default | table.groups.enabled .. |
table | @override .......... | tablet,server
-----------+-------------------------+------------------------------------------
```
```
$ accumulo rfile-info 'hdfs://accucluster/accumulo/tables/!0/table_info/A0002uqa.rf'
Reading file: hdfs://accucluster/accumulo/tables/!0/table_info/A0002uqa.rf
No on disk encryption detected.
RFile Version : 8
Locality group : tablet
Num blocks : 2
Index level 0 : 59 bytes 1 blocks
First key : +rep< ~tab:~pr [] 0 false
Last key : 2< ~tab:~pr [] 52040 false
Num entries : 1,411
Column families : [~tab, loc]
Locality group : server
Num blocks : 10
Index level 0 : 406 bytes 1 blocks
First key : +rep< srv:dir [] 0 false
Last key : 2< srv:time [] 135179 false
Num entries : 7,000
Column families : [log, file, srv, future]
Locality group :
Num blocks : 3
Index level 0 : 116 bytes 1 blocks
First key : 1< last:100000f2b52002c [] 132069 false
Last key : 2< last:100000f2b52001e [] 135179 false
Num entries : 793
Column families : [ecomp, last]
Meta block : BCFile.index
Raw size : 4 bytes
Compressed size : 12 bytes
Compression type : gz
Meta block : RFile.index
Raw size : 884 bytes
Compressed size : 559 bytes
Compression type : gz
```
Contributor guide
Research direction
Start with the metadata table locality-group configuration shown by `config -t accumulo.metadata -f group`, then compare it with the current column families and metadata-table query patterns. Use the `rfile-info` output as a reference, including the default group containing `ecomp` and `last`; done means the configuration has been reassessed and any justified updates are documented or applied.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java
- Domain
- databases
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100