JuliaData / JuliaData/TableOperations.jl

Add method to horizontally concatenate two (or more) tables of possibly different type

Open
#30 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
48
Forks
8
PR merge metrics
No merged PRs in 30d

Description

This has come up in ML workflows; see https://github.com/alan-turing-institute/MLJ.jl/issues/915. Would TableOperations.jl be the appropriate place for this?

What I have in mind is a simple concatenation - not a fancy join. So, if a column name of table1 appears in table2, then the table2 column just gets added with its name modified.

The tricky part is deciding on what the return type should be. I don't have fixed ideas about this, but perhaps if the tables do have the same type, and that is a sink type, then that is also the return type.

Although it is not a part of the public API, I see that TableTransforms.jl has an implementation. (To get the final table, the type of the first table is materialized.):

julia> table1
3×2 DataFrame
 Row │ x        z        
     │ Char     Float64  
─────┼───────────────────
   1 │ 𘂯       0.673471
   2 │ \U3f846  0.360792
   3 │ \Ud50cb  0.68075

julia> table2
(x = [0.41754294943943493, 0.7713462387833814, 0.9189998773436003], y = ['\U84fa1', '\U5e144', '\U872a4'])

julia> TableTransforms.tablehcat([table1, table2])
3×4 DataFrame
 Row │ x        z         x_        y       
     │ Char     Float64   Float64   Char    
─────┼──────────────────────────────────────
   1 │ 𘂯       0.673471  0.417543  \U84fa1
   2 │ \U3f846  0.360792  0.771346  \U5e144
   3 │ \Ud50cb  0.68075   0.919     \U872a4

cc @ExpandingMan @juliohm

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the linked MLJ.jl issue and the TableTransforms.jl implementation in src/transforms/parallel.jl, especially tablehcat. Then inspect the TableOperations.jl and Tables.jl interfaces to determine how heterogeneous inputs and duplicate column names should be handled. Done means a documented horizontal concatenation operation with an agreed return-type rule and coverage for the shown table forms.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.