Bingo NoSQL `enumerateId` and `delete` methods apparently out of sync
- Dominant language
- C++
- Stars
- 406
- Forks
- 134
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 24
Description
**Steps to Reproduce**
Using Bingo NoSQL, python 3.x epam.indigo library v1.11.0. My built indices were indicating that there were duplicate entries across all 16 databases, I attempted to enumerate and compare the set of IDs present in each.
Running the following script on a bingo nosql database (split across 16 directories), attempting to delete any entries which were already present in a previously-processed index:
```
def deduplicate(indigo, path):
ids = {}
for idx in tqdm(range(16), total=16, desc=f"Checking ID uniqueness for {path}"):
db = bingo.Bingo.loadDatabaseFile(indigo, f"{path}/db{idx+1:02}")
current_idx = idx + 1
ids[current_idx] = current_ids = set()
result = db.enumerateId()
while result.next():
current_ids.add(result.getCurrentId())
for other_idx in range(idx):
if other_idx < idx:
overlap = current_ids & ids[other_idx+1]
if overlap:
print(f"Remove {len(overlap)} from db{current_idx:02} duplicated with db{other_idx+1:02}")
for id in tqdm(overlap, desc=f"Pruning db{current_idx:02}", leave=False):
try:
db.delete(id)
current_ids.remove(id)
except BingoException as e:
print(f"delete {id} failed: {e}")
print(f"After removal db{current_idx:02} {len(current_ids)}")
db.optimize()
updated = set()
result = db.enumerateId()
while result.next():
updated.add(result.getCurrentId())
print(f"After optimization: {len(updated)}")
db.close()
```
**Expected behavior**
I expected that after "delete", the list generated by `enumerateId` would no longer contain the deleted IDs.
**Actual behavior**
The enumerated IDs still include deleted IDs. An attempt to delete IDs contained in the enumerated IDs list results in a BingoException, `delete xxxxx failed: There is no object with this id`. I have not tested whether the deleted ID still shows up in the queries that were showing duplicates.
**Indigo/Bingo version**
python epam.indigo 1.11.0
**Additional context**
`enumerateId` is not documented nor is its expected usage.
`optimize` is not documented nor is its intended usage.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the supplied deduplicate script with Python epam.indigo 1.11.0, focusing on enumerateId(), delete(), optimize(), and the post-optimization enumeration. Compare the returned IDs with deletion results, then determine the intended documented behavior for these methods and add or update coverage and documentation so the observed mismatch is resolved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100