JuliaDiff / JuliaDiff/FiniteDifferences.jl

to_vec doesn't preserve equality

Open
#132 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
318
Forks
32
PR merge metrics
No merged PRs in 30d

Description

Firstly, I'm very fond of to_vec -- it's served us well over the last couple of years. We know it has some shortcomings though, that we will need to address at some point.

Presently we have implementations of to_vec like this instead of implementations that look like this.

IIRC the reason for this is that if you define things in the latter fashion then you'll find if you do something like

A = randn(5, 4)
A_vec, _ = to_vec(A)
At_vec, _ = to_vec(transpose(A))

then A_vec == At_vec. My suspicion is that at some point in time, we were trying to compare the equality of two things based on their to_vec representation and it came up that things weren't being compared properly. We do a similar thing with Diagonal etc.

Unfortunately, it seems quite straightforward to construct examples which don't admit such a straightforward resolution as we employed for Adjoint, Transpose, and Diagonal. For a trivial example, consider that

A = randn(5)
A == (A, ) # is false -- a `Tuple` is never equal to a `Vector`

However, since to_vec(::Tuple) simply concatenates to_vec of each of the fields, we'll find that

to_vec(A)[1] == to_vec((A, ))[1]

This leads me to believe that it's likely not possible to define to_vec such that equality is preserved.

As such, I propose that we

  1. embrace, and explicitly document, that knowing whether to_vec(x)[1] == to_vec(y)[1] in general tells you nothing about whether x == y, and vice versa. We should point this out to explicitly warn people away from trying to check whether two things are equal based on their to_vec representation.
  2. refactor some of the things (like Tranpose and Adjoint) to use the struct representation, which is more computationally efficient.
  3. to_vec should purely be thought of as a way to make FiniteDifferences' life easier. We might even consider ceasing to export it.

Now that @oxinabox has added some new approximate equality checking to ChainRulesTestUtils, this ought to be less of a problem anyway, because it's now more straightforward to check whether things are approximately equal.

Of course, if anyone can see a different path forward, I'm all ears.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the to_vec implementations in src/to_vec.jl around the linked lines, along with the issue's examples for Transpose, Adjoint, Diagonal, and Tuple. Determine the scope and preferred path for documenting equality limitations, refactoring representations, and reconsidering the export; done requires an agreed design because the issue proposes alternatives rather than a single change.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
backend, documentation
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.