JuliaData / JuliaData/DataAPI.jl

Initial Roadmap

Open
#4 5 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
35
Forks
13
PR merge metrics
No merged PRs in 30d

Description

I'm opening this issue as a place to discuss the path forward for this package and for people to give their feedback. Here's where things are at in my mind:

  • I've opened PRs for Tables.jl, DataValues.jl, CategoricalArrays.jl, StatsBase.jl, and DataFrames.jl showing how packages would take a dependency on DataAPI.jl (once registered obviously) and extend functions from DataAPI.jl instead of defining themselves or using Requires.jl
  • I haven't made any changes to packages using the refX DataAPI.jl functions; while I understand the basics, I'm not as familiar w/ the implementations, but I'm willing to take a stab at it if people would like. @nalimilan and @piever are much more aware of how packages like PooledArrays, CategoricalArrays, and StructArrays can take advantage of sharing the common ref functions. I think it was also suggested at some point that we may want a RefArrays.jl package that was home to various sorting/grouping optimization routines that DataFrames/StructArrays could then share. I'm happy to push forward on making those changes, but I'll need to have some discussions w/ @nalimilan and @piever for guidance.
  • I've made @davidanthoff and @piever admins in this repo, in addition to the regular JuliaData maintainers who have access by default (@nalimilan, @bkamins, @andyferris, @andreasnoack, @ararslan , etc.); I think it's important that at least the initial set of package maintainers who depend on DataAPI.jl have an equal say in how the package works/is maintained and have some control over things.

In terms of steps forward, here's what I think:

  • @nalimilan, @piever, and myself (and anyone else who'd like) work on making sure the ref* function API is solid and draft PRs showing how packages could share these functions
  • Confirm w/ @davidanthoff that the DataValues.jl PR looks good (it's very minimal code, so hopefully should be uncontroversial); I know he's been swamped lately, but hopefully we can find a few minutes to sync on this
  • Register DataAPI.jl officially; I'm planning on the initial release being 1.0.0
  • Merge PRs to respective packages taking on the dependency

Please ping anyone else who might be interested or have something useful to add to the discussion here. I don't think there's a super rush on any of this, but I know DataFrames is approaching a 1.0 release in the next few months and it would be good to cleanup its dependencies soon.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the linked package PRs for Tables.jl, DataValues.jl, CategoricalArrays.jl, StatsBase.jl, and DataFrames.jl, then examine the existing refX functions and the discussion about RefArrays.jl. Done would require agreement on the API, guidance from the maintainers, official registration of DataAPI.jl, and the dependent-package PRs being ready to merge.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.