JuliaDiff / JuliaDiff/DifferentiationInterface.jl
Inconsistency in handling empty arguments
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 313
- Forks
- 35
- PR merge metrics
- No merged PRs in 30d
Description
When differentiating with respect to an empty array, the results tend to vary:
using DifferentiationInterface, ForwardDiff, ReverseDiff, Mooncake, Enzyme
ADTYPES = [
AutoForwardDiff(),
AutoReverseDiff(),
AutoMooncake(; config=nothing),
AutoEnzyme(; mode=Forward),
AutoEnzyme(; mode=Reverse),
# and more...
]
for adtype in ADTYPES
DifferentiationInterface.value_and_gradient(sum, adtype, Float64[])
end
ReverseDiff, Mooncake, and reverse Enzyme all happily return (0.0, []) 😄
Forward Enzyme tries to use a batch size of 0 and errors:
And ForwardDiff tries to construct a GradientResult which errors:
Funnily enough gradient with ForwardDiff (rather than value_and_gradient) is fine because it doesn't try to construct the GradientResult. I imagine the other operators would also have varying behaviour.
I suppose it is a bit of a trivial edge case, but would it be possible to unify the behaviour of the AD backends?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked empty-argument handling in DifferentiationInterface/ext/DifferentiationInterfaceEnzymeExt/utils.jl and DifferentiationInterface/ext/DifferentiationInterfaceForwardDiffExt/onearg.jl, then run the provided value_and_gradient example across the listed backends. Done means the affected backends and operators have deliberately consistent behavior for empty arguments.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100