SciML / SciML/RuntimeGeneratedFunctions.jl

Would this package be useful in DataFramesMeta?

Open
#33 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
112
Forks
19
Avg merge
6h 11m
Merged PRs (30d)
9

Description

I just finished a big PR in DataFramesMeta to try and reduce latency.

Given an expression of the form

f(:x, :y)

we make an anonymous function

(x, y) -> f(x, y)

This carries a compilation cost of creating the anonymous function. So my PR made an optimization where if it saw f(:x, :y) it would just return f, since it's the same as the anonymous function above.

But more other expressions are still problematic. Consider

:x + 1

This will always have to turn into

x -> x + 1

for purposes of the src => fun => dest syntax in DataFrames.

It looks like this package will let me cash :(x -> x + 1) so that the anonymous function of that form is only compiled once, and later calls are taken from a lookup. Is that correct?

If so, DataFramesMeta seems like a good application of this package.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read the linked DataFramesMeta PR 221 and compare its generated-function optimization with RuntimeGeneratedFunctions' handling of expressions such as :(x -> x + 1). Verify whether repeated expressions reuse a compiled lookup; done is a documented answer about applicability or a follow-up integration decision.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.