aws-samples / aws-samples/amazon-textract-idp-cdk-constructs

chore:Maintain S3 Prefix

Open
#20 0 comments 0 reactions 0 assignees View on GitHub
chore
Dominant language
TypeScript
Stars
33
Forks
14
PR merge metrics
No merged PRs in 30d

Description

At the moment TextractGenericSyncSfnTask loses the original S3 prefix of a document in the temp output path, as a result the CSV and JSON constructs also loses the prefix in the final outputs. For example if uploads/ contain my_docs/claim_forms/ or my_docs/discharge_summary. I think we should preserve the prefixes across the three constructs, and for the CSV and JSON constructs we should also generate a manifest (JSON) file along with the final output, which will contain the location/path of the document and the Textract output JSON, such as {"doc_path": …., "output_path": "….csv or .json"} this will make it easy for any custom post-processing *outside* of the step functions workflow. Imagine a workflow where someone sorts and classifies the documents by prefix in S3 and then runs this task, the output ultimately loses the prefixes. The prefixes should be preserved under the top level prefixes for the construct’s output i.e. for example Async task will put files under {s3_temp_output_prefix}/{my}/{original}/{prefix}/{job_id}/ and subsequently for CSV or JSON tasks will generate files {s3_output_prefix}/{my}/{original}/{prefix}/my_doc_xxxx/my_doc.json and a {s3_output_prefix}/{my}/{original}/{prefix}/my_doc_xxxx/my_doc_manifest.json

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.