MarketSquare / MarketSquare/robotframework-aws
Enhancement Request: List keys (files) in S3 bucket using paginators
@teaglebuilt is already working on this.
Since Dec 15, 2019.
- Dominant language
- Python
- Stars
- 25
- Forks
- 21
- PR merge metrics
- No merged PRs in 30d
Description
I would like to connect to an existing S3 bucket and list the keys (files). This will then drive further test cases on a key-by-key basis as we fetch files in order and perform data quality checks such as row counts, hash value, average or total of numerical columns, etc..
There is a good write-up [here](https://alexwlchan.net/2019/07/listing-s3-keys/) which explains how this could be implemented using [paginators.](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/paginators.html) Paginators simplify paging complexities for buckets that contain large amounts of files.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.