JuliaDataCubes / JuliaDataCubes/EarthDataLab.jl

Better handling of the testdata

Open
#202 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Triage
Dominant language
Julia
Stars
34
Forks
14
PR merge metrics
No merged PRs in 30d

Description

Currently the testcube is always downloaded from a S3Cube bucket and this takes at my location 72 seconds for the handling of the s3cube and then also 120 seconds for the download and saving of the germanycube.
These are 192 seconds of the tests which take in total 340 seconds approximately.
The resulting cube is 120 kilobyte in size.
Wouldn't it be better, to save the germanycube directly in a data folder in the test folder, because then we won't have to do the download every time we want to test.
Another optiion would be, that we handle the testdata with the new artifacts system, because then we also won't have to download the testdata for every invocation.
But I think, that this is unnecessary, until we have larger testdata sets, and for the moment, we can put the testdata in the test folder.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the test setup that downloads the testcube and germanycube from the S3Cube bucket, then inspect how test data is stored and whether the test-folder data or artifacts system fits better. Done means the existing tests use local or otherwise cached test data without downloading it on every invocation, while still passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
performance, testing
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.