4paradigm / 4paradigm/OpenMLDB
Using Kyro serialization may crash for the offline jobs
Offen
#2,497
0 Kommentare
0 Reaktionen
1 zugewiesene Person
Beansprucht von @tobegit3hub
Auf GitHub ansehen
bug
- Vorherrschende Sprache
- C++
- Sterne
- 1.7k
- Forks
- 331
- Ø Merge
- 12 T. 12 Std.
- Gemergte PRs (30 T.)
- 1
Beschreibung
**Bug Description**
Now the offline jobs may crash when adding `--conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer'` for offline jobs. It may works but Kyro serialization is not suggested for OpenMLDB offline jobs.
Beitragsleitfaden
Rechercherichtung
The issue mentions offline jobs and a Spark configuration using KryoSerializer. Look for Spark job submission code and serialization handling in the OpenMLDB codebase, particularly around offline execution paths. Check for any existing warnings or documentation about serialization support. Running an offline job with the given configuration to reproduce the crash would be a first step.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- spark
- Bereich
- backend, databases, machine-learning
- Issue-Typ
- Bug
- Schwierigkeit
- 3/5
- Geschätzter Aufwand
- 1-2 Tage
- Aktivitätsstatus
- Veraltet
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 45/100