`boot.pod/make-pod` (and thus `boot.core/aot`) fail with envs that are too large
- Dominant language
- Clojure
- Stars
- 1.7k
- Forks
- 178
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the bug**
`boot.pod/make-pod` (and thus its dependents like `boot.core/aot`) fails with environments that are sufficiently large. I came across this bug while working on a project with a lot of dependencies (760 jars) and thus a `:boot-class-path` of 83324 chars. The reason for the failure is that environments beyond a certain size cannot be embedded in code within `boot.pod/init-pod!`:
https://github.com/boot-clj/boot/blob/8b5fd047bd1c6fd33438e31062780084aca37dee/boot/pod/src/boot/pod.clj#L850
**To Reproduce**
Steps to reproduce the behavior:
1. From the REPL (though this bug also occurs from the commandline, it's easier to reproduce at the REPL), run `(boot.pod/make-pod env)` where `env` is a "sufficiently large" Boot environment. In my case it failed with a `:boot-class-path` of 83324 chars (also failed with `:fake-class-path`), but didn't care about the `:dependencies` size (it failed when I kept `:boot-class-path` in the env, whether or not I set `:dependencies` to `[]`).
2. Observe that the following exception occurs:
```
;; Given a dependency of Clojure 1.10
java.lang.IllegalArgumentException:
clojure.lang.Compiler$CompilerException: Syntax error compiling fn* at (0:0).
;; Given a dependency of Clojure 1.8 or 1.9
java.lang.ClassFormatError: Unknown constant tag 112 in class file clojure/core$eval21
clojure.lang.Compiler$CompilerException: java.lang.ClassFormatError: Unknown constant tag 112 in class file clojure/core$eval21, compiling:(NO_SOURCE_PATH:0:0)
```
**Expected behavior**
`boot.pod/make-pod` and dependents (`boot.core/aot` among them) should gracefully handle very large Boot environments.
**Desktop**
- OS: Mac OS 10.14.1
- Java: 1.8
**Additional context**
This is not due to the one classpath string being too big, as I tried splitting it up at compile-time via `partition-all`, embedding that in code, and recombining at runtime.
The workaround I'm using right now is to spit the environment to a file and read it in again:
```clojure
(defn- init-pod!
[env pod]
;; This was added
(let [temp-file (doto (java.io.File/createTempFile "env" ".edn") (.deleteOnExit))]
;; This was added
(spit temp-file env)
(doto pod
(require-in "boot.pod")
(with-invoke-in (boot.pod/set-worker-pod! worker-pod))
(with-eval-in
(require 'boot.util 'boot.pod)
(reset! boot.util/*verbosity* ~(deref util/*verbosity*))
;; This was changed
(alter-var-root #'boot.pod/env (constantly (slurp ~(.getPath temp-file))))
(create-ns 'pod)
(dosync (alter @#'clojure.core/*loaded-libs* conj 'pod))
(alter-var-root #'*ns* (constantly (the-ns 'pod)))
(clojure.core/refer-clojure)))))
```
Contributor guide
Research direction
Start in boot/pod/src/boot/pod.clj at init-pod! around the referenced line, then reproduce the failure with boot.pod/make-pod and a large environment such as one containing an 83324-character :boot-class-path. Compare this path with the reported temporary-file workaround; done means make-pod and dependents such as boot.core/aot handle very large environments without the shown compiler or class-format errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clojure
- Domain
- build-system, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100