JuliaDiff / JuliaDiff/AbstractDifferentiation.jl
Comparison with DifferentiationInterface.jl
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 138
- Forks
- 18
- PR merge metrics
- No merged PRs in 30d
Description
Hi there,
Adrian Hill (@adrhill) and I recently started a new unified differentiation package called DifferentiationInterface.jl:
I'm gonna try to show why we did that, how we proceeded, and what we expect to gain from it.
The discussion had started in #129 but did not go very far.
The main idea was to be less generic than AbstractDifferentiation from the start, in order to reduce the surface of error and focus on testing + performance.
This led to severe feature regressions and a complete rewrite, which is why I felt it was best to start a new project.
Whenever I say below that something is not implemented in AbstractDifferentiation, I don't mean it can never be.
I just mean it might require significant changes, not unlike the ones Adrian and I proposed.
Design principles
| AbstractDifferentiation | DifferentiationInterface | Reason | Refs | |
|---|---|---|---|---|
| Backend specification | Custom structs | Borrowed from ADTypes.jl | Used by SciML and Lux.jl | #88 |
| Backend requirements | @primitive macro |
Method implementation | Easier to understand | #13 |
| Fallback mechanism | Based on jacobian |
Based on pushforward / pullback |
Makes more theoretical sense | #123 |
| Fallback implementations | Lots of closures | No closure at all | Type stability | #109 #121 |
| Differentiation order | First and second order | First and second order | Enough for all practical purposes | |
| Input / output number | Anything | One in, one out | No need to deal with tuples or concatenate | #53 #65 #99 |
| Input / output types | Anything | AbstractArray or Number |
Some backends can't do more anyway + helps simplify | |
| Differentiation cache | Not yet | Yes | Performance with ForwardDiff / ReverseDiff | #41 |
| Mutating functions | Not yet | Yes | Performance with Enzyme and others | #14 |
| Sparsity | Not yet | Yes | Natural with ADTypes and SparseDiffTools |
Features
| AbstractDifferentiation | DifferentiationInterface | Reason | Refs | |
|---|---|---|---|---|
| Enzyme compatibility | Not yet | Yes | Easier with one array in, one array out | #40 #84 #85 |
| Benchmarking utilities (for users) | Not yet | Yes | Easier with one array in, one array out | #45 |
| Testing utilities (for users) | Not yet | Yes | Easier with one array in, one array out | #45 |
Performance comparison
I'm gonna add AbstractDifferentiation as an extension to DifferentiationInterface, so that we can include its backends in our benchmark suite and compare.
The results will be posted here in the coming days.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the design and feature comparison tables in this issue, then review the referenced issues and DifferentiationInterface.jl. The issue says AbstractDifferentiation should be added as an extension to the new package so its backends can be included in the benchmark suite; completion would require the promised performance comparison results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100