JuliaGenAI / JuliaGenAI/HuggingFaceDatasets.jl

Issue loading Pycall and HuggingFaceDatasets simultaneously

Open
#32 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
38
Forks
1
PR merge metrics
No merged PRs in 30d

Description

I'm getting an error when loading Pycall and HuggingFaceDatasets simultaneously in Julia Version 1.11.5.

julia(59194,0x1ffb40140) malloc: *** error for object 0x166e842b0: pointer being freed was not allocated
julia(59194,0x1ffb40140) malloc: *** set a breakpoint in malloc_error_break to debug

[59194] signal 6: Abort trap: 6
in expression starting at REPL[3]:1
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 48449096 (Pool: 48442458; Big: 6638); GC: 41

Pkg status

julia> Pkg.status()
Status `~/.julia/environments/v1.11/Project.toml`
  [fbb218c0] BSON v0.3.9
  [052768ef] CUDA v5.8.2
  [35d6a980] ColorSchemes v3.29.0
  [5ae59095] Colors v0.13.1
  [8f4d0f93] Conda v1.10.2
  [b4f34e82] Distances v0.10.12
  [31c24e10] Distributions v0.25.120
  [24df6009] ExactOptimalTransport v0.2.5
  [587475ba] Flux v0.16.4
⌃ [28b8d3ca] GR v0.72.8
  [d94b9a45] HuggingFaceDatasets v0.3.4
  [916415d5] Images v0.26.2
  [98e50ef6] JuliaFormatter v2.1.2
  [5ab0869b] KernelDensity v0.6.9
  [eb30cadb] MLDatasets v0.7.18
⌃ [b8f27783] MathOptInterface v1.40.2
  [6f286f6a] MultivariateStats v0.10.3
  [d8a4904e] MutableArithmetics v1.6.4
  [872c559c] NNlib v0.9.30
⌃ [7e02d93a] OptimalTransport v0.1.9
  [3bd65402] Optimisers v0.4.6
  [d96e819e] Parameters v0.12.3
  [34689656] ParetoGAN v1.0.0-DEV `~/.julia/dev/ParetoGAN`
  [14b8a8f1] PkgTemplates v0.7.55
  [f0f68f2c] PlotlyJS v0.18.16
⌃ [91a5bcdd] Plots v1.40.7
  [92933f4c] ProgressMeter v1.10.4
  [438e738f] PyCall v1.96.4
  [295af30f] Revise v3.8.0
  [4c63d2b9] StatsFuns v1.5.0
  [6dd1b50a] Tulip v0.9.7
⌃ [e88e6eb3] Zygote v0.7.8
  [37e2e46d] LinearAlgebra v1.11.0

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the crash by loading PyCall and HuggingFaceDatasets together in Julia 1.11.5, using the package environment shown in the issue. Investigate the interaction between these packages and the reported allocator failure; done means both packages load without the malloc error or abort.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia, python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.