Dynamic tablename/config.json use case

I'm looking at a use case where I'd like to bring in CSVs into their own individual tables, created at/right before load time. This is for data submitted by internal users. It benefits us to isolate each CSV (relatively small in size) to it's own table. This table would be determined by the filename and the s3 directories in which the CSV is submitted to.

Option 1 is to load all these CSVs into a single, pre-created table, and then have a lambda run off of an SNS msg to create a table and move data to this table for each file. I think this gets a little tricky when considering concurrency.

Option 2 is to first run a custom lambda function that will create the end table we'd load into, and then insert into dynamodb a config setting for this table. This lambda function then would, at the end, move the s3 file into a location that would trigger the aws redshift lambda loader and use the newly pushed config.

I'm inclined to go with option 2, but I'm wondering if there is anything to be aware of when doing this. It seems like to me that I need to insert into dynamodb a new config that contains the tablename, s3 location, etc as a part of the pre-process lambda. Am I missing anything else?

Non vengono indicati file o test. Inizia leggendo i punti di ingresso del loader e della configurazione del repository, quindi segui il modo in cui gli eventi S3 e le impostazioni di DynamoDB selezionano una tabella Redshift. Il lavoro sarà completato quando sarà disponibile un design approvato da un maintainer che copra la creazione di tabelle per file e i caricamenti concorrenti.

Scritto dal modello di indicizzazione a partire dal testo della issue.