CliMA / CliMA/ClimaCore.jl

Some matrix field broadcasting results in `Failed to compile PTX code (ptxas received signal 9)` errors

Open
#2,035 0 comments 0 reactions 0 assignees View on GitHub
bug GPU
Dominant language
Julia
Stars
117
Forks
19
Avg merge
3d 4h
Merged PRs (30d)
41

Description

For example: https://buildkite.com/clima/climacore-ci/builds/4458#01924384-ee2a-4328-a106-c6501a46657e.

Per Tim Besard on slack:
> signal 9 comes from outside; OOM killer maybe?

I confirmed with him, this is OOM during compilation.

I'm not sure how we can fix this. I think we'll need to profile some simpler cases to see where the compiler is mostly allocating.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the linked Buildkite failure and reproduce the matrix field broadcasting case that produces the PTX compilation error. Profile simpler broadcasting cases as suggested in the issue to identify where compilation memory grows. Done means the affected compilation no longer fails from an out-of-memory condition, with the relevant behavior verified on a representative case.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
compilers, hpc, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.