4paradigm / 4paradigm/OpenMLDB

Using Kyro serialization may crash for the offline jobs

Open
#2,497 0 comments 0 reactions 1 assignee Claimed by @tobegit3hub View on GitHub
bug
Dominant language
C++
Stars
1.7k
Forks
331
Avg merge
12d 12h
Merged PRs (30d)
1

Description

**Bug Description**

Now the offline jobs may crash when adding `--conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer'` for offline jobs. It may works but Kyro serialization is not suggested for OpenMLDB offline jobs.

Contributor guide

Open the contributing guide

Research direction

The issue mentions offline jobs and a Spark configuration using KryoSerializer. Look for Spark job submission code and serialization handling in the OpenMLDB codebase, particularly around offline execution paths. Check for any existing warnings or documentation about serialization support. Running an offline job with the given configuration to reproduce the crash would be a first step.

Written by the indexing model from the issue text.

Assessment

Tech stack
spark
Domain
backend, databases, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.