boot-clj / boot-clj/boot

AOT works on REPL but not in script

Open
#534 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Clojure
Stars
1.7k
Forks
178
PR merge metrics
No merged PRs in 30d

Description

When I use `(boot (aot :namespace '#{ns1 ns2}))` in the REPL this works fine. When I run the same code as a script the AOT does not happen, the `sparkling.serialization.Registrator` class does not compile, and I get these AOT-related exceptions:

org.apache.spark.SparkException: Failed to register classes with Kryo
java.lang.ClassNotFoundException: sparkling.serialization.Registrator

Here is the code. To reproduce this issue save this to a file called `chk.clj`. Then `chmod 755 chk.clj`. Then run `./chk.clj`

To see the code work start `boot repl` and paste this file into it.

```clojure
#!/usr/bin/env boot
(set-env!
:dependencies '[
[org.apache.spark/spark-core_2.10 "2.0.2"]
[org.apache.spark/spark-sql_2.10 "2.0.2"]
[gorillalabs/sparkling "1.2.5"]])

(require '[sparkling.conf :as conf])
(require '[sparkling.core :as spark])
(require '[sparkling.serialization])
(require '[sparkling.destructuring])

(boot (aot :namespace '#{sparkling.serialization
sparkling.destructuring } ))

(defonce sc (-> (conf/spark-conf)
(conf/app-name "My App")
(conf/master "local")
(spark/spark-context)))

(->> (range 100) (spark/parallelize sc) (spark/collect))
```

Output of `boot --version`.
```
#http://boot-clj.com
#Wed Nov 30 01:20:40 PST 2016
BOOT_CLOJURE_NAME=org.clojure/clojure
BOOT_CLOJURE_VERSION=1.7.0
BOOT_VERSION=2.6.0
```

Contributor guide

Open the contributing guide

Research direction

Start with chk.clj and reproduce the difference between running ./chk.clj and pasting the same file into boot repl. Trace the (boot (aot :namespace ...)) invocation and verify whether sparkling.serialization.Registrator is compiled; done means script execution performs AOT and no ClassNotFoundException or Kryo registration error occurs.

Written by the indexing model from the issue text.

Assessment

Tech stack
clojure
Domain
build-system, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.