4paradigm / 4paradigm/OpenMLDB
Using Kyro serialization may crash for the offline jobs
- Dominant language
- C++
- Stars
- 1.7k
- Forks
- 331
- Avg merge
- 12d 12h
- Merged PRs (30d)
- 1
Description
**Bug Description**
Now the offline jobs may crash when adding `--conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer'` for offline jobs. It may works but Kyro serialization is not suggested for OpenMLDB offline jobs.
Contributor guide
Research direction
The issue mentions offline jobs and a Spark configuration using KryoSerializer. Look for Spark job submission code and serialization handling in the OpenMLDB codebase, particularly around offline execution paths. Check for any existing warnings or documentation about serialization support. Running an offline job with the given configuration to reproduce the crash would be a first step.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- spark
- Domain
- backend, databases, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100