clj-commons / clj-commons/byte-streams

Late declarations of lower-cost conversions are ineffective

Open
#61 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Clojure
Stars
430
Forks
31
PR merge metrics
No merged PRs in 30d

Description

Declaring new conversions via `def-conversion` which would make conversion between two types less costly are ineffective if the conversion in question has occurred at least once before. This is caused by the global [converter memoization](https://github.com/clj-commons/byte-streams/blob/master/src/clj_commons/byte_streams.clj#L139-L142) which captures the state of the conversion graph at the point in time of the very first invocation for a given pair of source and dest types.

Reproducer:

```clojure
(defrecord Foo [data])
(defrecord Bar [data])

(defn run [n]
(prn :begin n)
(bs/convert (Foo. "foo") String)
(prn :end n)
(newline))

(bs/def-conversion ^{:cost 0} [Foo Bar]
[x _]
(prn :foo->bar)
(Bar. (:data x)))

(bs/def-conversion ^{:cost 0} [Bar String]
[x _]
(prn :bar->string)
(:data x))

;; At this point we can only indirectly convert from `Foo` to `String` via `Bar`
(run 1)

;; Now we declare a direct conversion path from `Foo` to `String`
(bs/def-conversion ^{:cost 0} [Foo String]
[x _]
(prn :foo->string)
(:data x))

;; But because the first invocation has already memoized the more costly path, it has no effect
(run 2)
```

Output:

```clojure
:begin 1
:foo->bar
:bar->string
:end 1

:begin 2
:foo->bar
:bar->string
:end 2
```

In contrast, moving both `run` invocations after the last `bs/def-conversion` outputs:

```clojure
:begin 1
:foo->string
:end 1

:begin 2
:foo->string
:end 2
```

This is a bit of a gotcha which might at least be worth documenting. Alternatively, `def-conversion` could reset the memo which should solve this issue.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in src/clj_commons/byte_streams.clj at the converter memoization linked in the issue, then trace how def-conversion updates the conversion graph. Reproduce the Foo-to-String example and determine whether late lower-cost conversions should invalidate the memoized result or be documented as a limitation; done means the chosen behavior is covered or clearly explained.

Written by the indexing model from the issue text.

Assessment

Tech stack
clojure
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.