JuliaPy / JuliaPy/PythonPlot.jl
PythonPlot + PackageCompiler + IJulia: kernel does not start (IJulia not precompiling)
- Langage dominant
- Julia
- Étoiles
- 85
- Forks
- 9
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
I want a faster startup time and time-to-first-plot when opening or restarting a notebook.
(Currently about ~~30~~ 15 seconds total; compared to about 3.2 seconds TTFP with a Python notebook).[^1],[^2]
[^1]: not all of this time is due to PythonCall and PythonPlot: about 10 seconds is IJulia startup itself: https://github.com/JuliaLang/IJulia.jl/issues/1090
[^2]: after upgrading to julia 1.9.3, from 1.9.0-beta3, I saw an impressive decrease in TTFP, from 30 s, to 15 seconds. :D. About 8.6 seconds is due to IJulia startup still. Which leaves 7.0 seconds for PythonCall & PythonPlot load and startup, and plot display.
Putting PythonPlot in a system image with PackageCompiler.jl works, and reduces loading and plotting time **amazingly**, _in the terminal_.
But when trying to use an IJulia kernel with that sysimg, the kernel does not start.
Error in jupyter terminal:
```
[I 14:32:27.364 NotebookApp] Kernel started: 31647825-a827-4c36-86f6-f715ba470675, name: julia-mysys-1.9
[ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
ERROR: LoadError: OutOfMemoryError()
Stacktrace:
[1] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
@ Base .\loading.jl:1001
[2] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
@ Base .\loading.jl:1413
[3] _require(pkg::Base.PkgId, env::String)
@ Base .\loading.jl:1690
[4] _require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base .\loading.jl:1567
[5] macro expansion
@ .\loading.jl:1555 [inlined]
[6] macro expansion
@ .\lock.jl:267 [inlined]
[7] require(into::Module, mod::Symbol)
@ Base .\loading.jl:1518
[8] include
@ .\Base.jl:456 [inlined]
[9] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
@ Base .\loading.jl:1952
[10] top-level scope
@ stdin:2
in expression starting at C:\Users\tfiers\.julia\packages\IJulia\6TIq1\src\IJulia.jl:1
in expression starting at stdin:2
ERROR: LoadError: Failed to precompile IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a] to "C:\\Users\\tfiers\\.julia\\compiled\\v1.9\\IJulia\\jl_19EC.tmp".
Stacktrace:
[1] error(s::String)
@ Base .\error.jl:35
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
@ Base .\loading.jl:2195
[3] compilecache
@ .\loading.jl:2068 [inlined]
[4] _require(pkg::Base.PkgId, env::String)
@ Base .\loading.jl:1712
[5] _require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base .\loading.jl:1567
[6] macro expansion
@ .\loading.jl:1555 [inlined]
[7] macro expansion
@ .\lock.jl:267 [inlined]
[8] require(into::Module, mod::Symbol)
@ Base .\loading.jl:1518
in expression starting at C:\Users\tfiers\.julia\packages\IJulia\6TIq1\src\kernel.jl:1
[I 14:32:45.348 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
[ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffedc8a89ae -- jl_table_assign_bp at C:/workdir/src\iddict.c:48
in expression starting at C:\Users\tfiers\.julia\packages\IJulia\6TIq1\src\IJulia.jl:36
jl_table_assign_bp at C:/workdir/src\iddict.c:48
ijl_idtable_rehash at C:/workdir/src\iddict.c:24
jl_restore_system_image_from_stream_ at C:/workdir/src\staticdata.c:3165
jl_restore_package_image_from_stream at C:/workdir/src\staticdata.c:3314
jl_restore_incremental_from_buf at C:/workdir/src\staticdata.c:3360 [inlined]
ijl_restore_package_image_from_file at C:/workdir/src\staticdata.c:3473
_include_from_serialized at .\loading.jl:1001
jfptr__include_from_serialized_26122 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
tojlinvoke79009 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
j__include_from_serialized_25818 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
_require_search_from_serialized at .\loading.jl:1413
_require at .\loading.jl:1690
_require_prelocked at .\loading.jl:1567
jfptr__require_prelocked_45234 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
tojlinvoke79134 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
j__require_prelocked_25262 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
macro expansion at .\loading.jl:1555 [inlined]
macro expansion at .\lock.jl:267 [inlined]
require at .\loading.jl:1518
jfptr_require_25249 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1874 [inlined]
call_require at C:/workdir/src\toplevel.c:466 [inlined]
eval_import_path at C:/workdir/src\toplevel.c:503
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:731
jl_eval_module_expr at C:/workdir/src\toplevel.c:203 [inlined]
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:715
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:856
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:856
ijl_toplevel_eval at C:/workdir/src\toplevel.c:921 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:971
eval at .\boot.jl:370 [inlined]
include_string at .\loading.jl:1806
_include at .\loading.jl:1866
tojlinvoke77691 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
include at .\Base.jl:456 [inlined]
include_package_for_output at .\loading.jl:1952
jfptr_include_package_for_output_34600 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1874 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:226
eval_stmt_value at C:/workdir/src\interpreter.c:177 [inlined]
eval_body at C:/workdir/src\interpreter.c:624
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:762
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:912
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:856
ijl_toplevel_eval at C:/workdir/src\toplevel.c:921 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:971
eval at .\boot.jl:370 [inlined]
include_string at .\loading.jl:1806
tojlinvoke76665 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
include_string at .\loading.jl:1816 [inlined]
exec_options at .\client.jl:305
jfptr_exec_options_46043 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
tojlinvoke77001 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
j_exec_options_42205 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
_start at .\client.jl:522
jfptr__start_42203 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1874 [inlined]
true_main at C:/workdir/src\jlapi.c:573
jl_repl_entrypoint at C:/workdir/src\jlapi.c:717
mainCRTStartup at C:/workdir/cli\loader_exe.c:59
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 3674 (Pool: 3657; Big: 17); GC: 0
```
Not expecting this to work or be worked on, just noting it down here.
`versioninfo()`:
```
Julia Version 1.9.0-beta3
Commit 24204a7344 (2023-01-18 07:20 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: 8 × Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
Threads: 7 on 8 virtual cores
Environment:
JULIA_EDITOR = code.cmd
JULIA_NUM_THREADS = 7
JULIA_PYTHONCALL_EXE = C:\Users\tfiers\mambaforge\python
```
**Package compilation & kernel install scripts:**
`build.jl`:
```julia
using PackageCompiler
create_sysimage(
["PythonPlot"],
sysimage_path = "mysys.dll",
precompile_execution_file = "compile_me.jl",
# Note: not `script`.`
)
```
`compile_me.jl`:
```julia
using PythonPlot
pygui(false)
fig, ax = pyplot.subplots()
plot([1,2,1])
PythonPlot.display_figs()
# Might have to call PythonPlot.__init__() in nb
```
`add_ipython_kernel.jl`:
```julia
using IJulia
path = abspath(joinpath(@__DIR__, "mysys.dll"))
IJulia.installkernel(
"Julia-mysys",
"--project=@.",
"--sysimage=$path",
)
```
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Évaluation
Cette issue n'a pas encore été évaluée.