SACGF / SACGF/variantgrid

Analysis node CSV export: plain .csv with a shorter filename

Open
#1,866 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
30
Forks
3
Avg merge
9h 28m
Merged PRs (30d)
42

Description

User feedback (via SACGF/variantgrid_sapath#441): ".csv files need unzipping and have long names which adds extra steps to the process"

The node grid CSV export is zipped because it reuses the writer added for the whole cohort/sample annotated exports in #1171, where files ran to millions of rows. A node export is what the user has already filtered the grid down to, and they open it straight away, so the zip is an extra step for little gain (node export files are purged after `ANALYSIS_NODE_EXPORT_CACHE_DAYS` anyway). On Windows "Extract All" also puts the CSV in a folder with the zip's name, doubling the path length.

Change:

* Node grid CSV export is a plain `.csv` (cohort/sample annotated exports keep `.csv.zip`, VCFs keep `.vcf.gz`)
* Shorter export filename: `{node name or class}_{sample}_a{analysis}_n{node}_v{version}.csv` - analysis pk, node pk and version keep it unique
* `text/csv` added to nginx `gzip_types` so plain CSVs are still compressed on the wire

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the analysis node grid CSV export entry point and inspect the writer added for the whole cohort/sample annotated exports in #1171. Check the nginx gzip_types configuration as well. Done means node exports are plain .csv files with the specified shorter unique filename, existing cohort/sample and VCF formats remain unchanged, and text/csv is compressed on the wire.

Written by the indexing model from the issue text.

Assessment

Tech stack
nginx, python
Domain
backend, devops
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.