aws-samples / aws-samples/data-lake-as-code

Submission Summary table in RODA Clinvar dataset not queryable in Athena

Open
#27 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
173
Forks
46
PR merge metrics
No merged PRs in 30d

Description

I'm trying to demo Athena using the pre-built RODA Clinvar dataset. I deployed using the stack here: https://github.com/aws-samples/data-lake-as-code/tree/roda#readme.

The stack deploys without any issues noted, and after that I'm able to query most of the tables from Athena. However, when I run the following command in Athena:

`SELECT * FROM "clinvar_summary_variants_dl-awsroda"."submission_summary";`

The count of records returned is as-expected, but the values in every column are empty.

Notably, there appears to be a disconnect between the column names in the glue data catalog and the column names in the corresponding parquet metadata: in the data catalog columns are assigned distinct names (e.g. variationid, clinicalsignificance,...) in accordance with the official clinvar file definition README, but in the parquet file for this table each column appears to be named 'col' + autoincremented integer (e.g. col0, col1,...).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.