FluxML / FluxML/Tracker.jl

[gsoc] Remove the `scan` method.

Open
#164 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
54
Forks
37
Avg merge
5h 49m
Merged PRs (30d)
1

Description

### Motivation and description

Currently the `scan` method is used to mark the nodes before applying the actual backpropagation in the graph.
https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L19
We can achieve the same results without employing the this pre-marking of nodes. In this proposal the `back` function shall not use the `scan` method, but rather take care of the nodes itself, without a pre-marking.
https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L74

The functions and files involved are only where the `ref` field is used: https://github.com/FluxML/Tracker.jl/blob/master/src/Tracker.jl#L41.

The following structures/functions/algos have to be understood in order to remove `scan`:
1. struct [Tracked{T}](https://github.com/FluxML/Tracker.jl/blob/master/src/Tracker.jl#L40) and it's members
2. the [scan](https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L19), [back](https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L47) and [back](https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L74) methods in the file `back.jl`.

Note the function [back(g::Grads, x::Tracked, Δ)](https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L118) is used for higher-gradients and we should not focus on this use-case in this proposal.

Contributor guide

No contributing guide indexed for this repository

Research direction

Read src/Tracker.jl around the Tracked{T} ref field, then inspect scan and the back methods in src/back.jl. Trace how ref is used to mark nodes before backpropagation and compare the behavior with back handling nodes directly. Done means scan is no longer used by back while preserving existing backpropagation behavior; higher-gradient back(g::Grads, x::Tracked, Δ) is out of scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
machine-learning
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.