aws-samples / aws-samples/amazon-textract-idp-cdk-constructs
chore:Maintain S3 Prefix
- Dominant language
- TypeScript
- Stars
- 33
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Description
At the moment TextractGenericSyncSfnTask loses the original S3 prefix of a document in the temp output path, as a result the CSV and JSON constructs also loses the prefix in the final outputs. For example if uploads/ contain my_docs/claim_forms/ or my_docs/discharge_summary. I think we should preserve the prefixes across the three constructs, and for the CSV and JSON constructs we should also generate a manifest (JSON) file along with the final output, which will contain the location/path of the document and the Textract output JSON, such as {"doc_path": …., "output_path": "….csv or .json"} this will make it easy for any custom post-processing *outside* of the step functions workflow. Imagine a workflow where someone sorts and classifies the documents by prefix in S3 and then runs this task, the output ultimately loses the prefixes. The prefixes should be preserved under the top level prefixes for the construct’s output i.e. for example Async task will put files under {s3_temp_output_prefix}/{my}/{original}/{prefix}/{job_id}/ and subsequently for CSV or JSON tasks will generate files {s3_output_prefix}/{my}/{original}/{prefix}/my_doc_xxxx/my_doc.json and a {s3_output_prefix}/{my}/{original}/{prefix}/my_doc_xxxx/my_doc_manifest.json
Contributor guide
Assessment
This issue has not been assessed yet.