practicalli / practicalli/clojure

Lazy sequences verses recursion

Open
#200 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

clojure-core
Dominant language
Makefile
Stars
117
Forks
36
PR merge metrics
No merged PRs in 30d

Description

understand the difference between how a recursive function using lazy-seq differs from what for returns.
My experience is that with lazy sequence returned by for is that when iterating it with loop the portion of the sequence that gets evaluated remains referenced in some way, so that none of the elements from previous iterations are garbage collected. This is assuming the application's iteration code doesn't involve retaining a reference, of course.

I've got some example code that shows the difference through causing an OOM in the for case. https://gist.github.com/Hamled/cab41245abd208b4f6cb67026f135e34 (The gist also has some examples of how this interacts weirdly with futures from the manifold library, in case anyone has used that and has some insights... that's more representative of my actual use case.)

Oh maybe this is a result of for using chunk-cons ? When I change the parameters of my example code so that the sequence is 100 elements long, but the total amount of memory allocated is still the same, the for version also works.

I guess if the amount of memory needed for all entries in single chunk is more than can be allocated, then it OOMs?

to be clear, for is not a loop and doesn't use looping internally - it is built out of "recursive" calls that are lifted out via lambda encapsulation
IOW exactly how you'd use lazy-seq and self calls in any other lazy function you construct

in general, I would not use lazy seqs to do things over a buffer, I would use loop/recur to have better control over when things happen
with lazy seqs, you give up control over when realization happens (chunking usually happens in chunks of 32)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue provides no repository file or test entry point. Start by running the linked gist and comparing its lazy-seq and for cases, including the chunking and manifold-future examples; done means documenting a clear, reproducible explanation of the observed memory behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
clojure
Domain
developer-experience
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.