JuliaIO / JuliaIO/FileIO.jl

Save / Load with multiple data inputs / outputs

Open
#192 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
228
Forks
81
Avg merge
1d 29m
Merged PRs (30d)
1

Description

I'm frequently dealing with a really weird tabular data structure called a PCL file - basically, the first n rows contain various forms of metadata (first column for these rows is metadata name), then below that it has a sample x feature matrix. So for example, I might have my_data.pcl:

age 12 14 18
gender m f f
sampleID a b c
feature1 0.2 0.4 0.3
feature2 0.3 0.2 0.3
feature3 0.1 0.6 0.7

Since most of the operations occur on the numerical table part, and storing this all in a single dataframe (or whatever) would generally lead to columns with type Any, what I'd like to be able to do is have load/save functions that make/take two iterable tables, that share the sampleID row, eg the two tables would be:

metadatadf:

sampleID a b c
age 12 14 18
gender m f f

featuredf:

sampleID a b c
feature1 0.2 0.4 0.3
feature2 0.3 0.2 0.3
feature3 0.1 0.6 0.7

And I'd like to be able to do something like:

(x, y) = load("my_data.pcl", id_row="sampleID")
metadatadf = DataFrame(x)
featuredf = DataFrame(y)

save("new_table.pcl", metadatadf, featuredf)

Can I get some guidance on whether this is possible / makes sense to use the FileIO framework for this?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review FileIO's load/save interface against the proposed two-table PCL examples and the id_row="sampleID" option. Determine whether multi-input/output dispatch and this metadata/feature split fit the framework, then document a concrete API decision and its expected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.