Reading S3 object metadata
- Dominant language
- C++
- Stars
- 60
- Forks
- 100
- Avg merge
- 1h 50m
- Merged PRs (30d)
- 25
Description
It would be nice to be able to query [S3 object metadata](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html) directly.
Maybe it could use the `parquet_kv_metadata` as an interface?
```sql
SELECT * FROM parquet_kv_metadata('s3://bucket/file.parquet');
```
```
file_name,key,value
's3://bucket/file.parquet', pandas,
's3://bucket/file.parquet', ARROW:schema,
's3://bucket/file.parquet', s3_metadata, {"ETag": "awefawefawef"}
```
(apologies if this is already possible, I couldn't find anything about it in the docs)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the existing parquet_kv_metadata interface and the AWS S3 object metadata documentation linked in the issue. Determine how S3 metadata could be exposed through the suggested SQL query, then verify that querying an S3 Parquet file returns its object metadata alongside the existing key-value metadata.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, cpp
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100