crate / crate/cratedb-toolkit

[I/O] DynamoDB: Load data from tables using single-table design

Open
#232 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10
Forks
5
Avg merge
7d 5h
Merged PRs (30d)
7

Description

## About
Loading data from DynamoDB into CrateDB may need special treatment compared with other databases, because DynamoDB's relational design paradigms are special.

## Details
According to the blog post [Single-table vs. multi-table design in Amazon DynamoDB](https://aws.amazon.com/blogs/database/single-table-vs-multi-table-design-in-amazon-dynamodb/):

> For people learning about [Amazon DynamoDB](https://docs.aws.amazon.com/dynamodb), the idea of [single-table design](https://www.alexdebrie.com/posts/dynamodb-single-table/) is one of the most mind-bending concepts out there. Rather than the relational notion of having a table per entity, **DynamoDB tables often include multiple different entities in a single table**.

> DynamoDB isn’t a relational database and you shouldn’t use it like one. The learning curve feels steep, but there are really only three or four key concepts you need to learn, and everything else flows from that. Once you understand these basics, you’ll be able to make more informed decisions about how many tables to use in your application.

## Thoughts

Based on reports by others

> every row has a unique schema based on the "type" of the record

we think it might make sense to dispatch records to different target tables in CrateDB instead, based on this information.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.