awslabs / awslabs/amazon-dynamodb-tools
[verb] `signature` - for comparing tables across regions or time
- Dominant language
- Python
- Stars
- 181
- Forks
- 50
- Avg merge
- 4h 11m
- Merged PRs (30d)
- 36
Description
We could do a "signature" verb that gave, for example, a hash of the table or parts of a table. This would allow efficient comparisons of two tables across regions or time. Similar to how you use a signature to check two files match, but here it's tables.
Regions: You could compare two endpoints of a global table without copying any table data out of region (as you'd have to with a `diff`).
Time: You can store the signature of a table before a backup, then compare the signature later after a restore.
One way to build this would be to do a segmented scan with X segments, generate a hash of each segment, then a hash of the hashes. If the hash of hashes doesn't match, you can know in what segment the mismatch was. The output would be the X+1 hashes (to S3 probably).
We'd need a hash algorithm that could take a stream of items serialized in some very deterministic way to build the hash up.
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by locating the existing verb implementation and DynamoDB scan behavior, then review how S3 outputs are handled. Done should mean a decided signature design, including deterministic item serialization, segmented results, and comparison behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100