TuringLang / TuringLang/DynamicPPL.jl

Do we still need parametric types in model definition for autodiff?

Open
#823 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Julia
Stars
286
Forks
41
Avg merge
1d 11h
Merged PRs (30d)
34

Description

We currently use the following DynamicPPL syntax to support AutoDiff.

@model function f(N, ::Type{T}=Vector{Float64}) where {T}
    a = T(undef, N)
    a .~ Normal()
    m = sum(a)
    o = T(undef, N)
    o .~ Normal(m, 1)
end

We still need the parametric type (..., ::Type{T}=Vector{Float64}) where {T} for operator-overloading approaches like ForwardDiff and ReverseDiff. However, this is no longer necessary for Mooncake, which is source-transformation-based.

There might be a way to work around such requirements for ForwardDiff and ReverseDiff by implementing a (simple) model (AST) expression transform similar to Mooncake's. Such an approach is a bit speculative at the moment, but it is worth considering to simplify the modelling syntax.

cc @willtebbutt

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the current DynamicPPL model syntax with Mooncake's source-transformation approach, then investigate whether ForwardDiff and ReverseDiff can avoid the parametric type requirement. Done means a clear decision or design direction for simplifying model definitions, including the implications for each autodiff backend.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.