Observer crashes with SIGSEGV in ObSystemConfigKey constructor during DBMS_SCHEDULER automatic stats gathering job

Environment

• Version: observer (OceanBase seekdb)
• Build Info:
  • REVISION: 1-3f1ca66dcb3103dc15c90b834bbe7259032dac18
  • BUILD_BRANCH: master
  • BUILD_TIME: May 29 2026 23:31:01
  • BUILD_FLAGS: RelWithDebInfo|Sanity
  • BUILD_INFO: obbuild-sanity-master-101066
• Is this an RD-provided temporary version?: No

Description

The observer process crashed with a segmentation fault (SIGSEGV) while executing an automatic statistics gathering job scheduled by DBMS_SCHEDULER. The crash occurred in the ObSystemConfigKey::ObSystemConfigKey constructor, causing the observer on port 27035 to terminate. Subsequent external connections to this port resulted in "Connection refused" errors.

Steps to Reproduce / Scenario

Reproducibility: Consistently reproducible.

Scenario: The crash was triggered internally by the observer while executing a DBMS_SCHEDULER job for automatic statistics collection. The specific action was the execution of DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(...).

Details:

• A core dump file was generated at /obdata/data/coredump/core-T1_ReqWorker-2479102-1780236175 (approx. 559 MB).
• GDB analysis of the core shows the crashing thread's stack top is ObSystemConfigKey::ObSystemConfigKey (SIGSEGV).
• The call chain originates from ObDBMSSchedJobExecutor::run_dbms_sched_job with job_id=7.
• The core file is preserved at the path above.

Impact

The observer process terminated, making the database instance on port 27035 unavailable. This disrupts any service or workload depending on this instance.

Logs / Stack Trace

---
pid: 3062091
cwd: "/obdata/data"
command: "gdb -q -batch -ex \"set pagination off\" -ex \"thread 1\" -ex \"bt\" \"/obdata/data/minan/4/ob/obm.z1.obs0/bin/observer\" \"/obdata/data/coredump/core-T1_ReqWorker-2479102-1780236175\""
started_at: 2026-06-01T07:32:11.972Z
running_for_ms: 75052
---
[New LWP 2479184]
... (multiple threads) ...
Core was generated by `/obdata/data/minan/4/ob/obm.z1.obs0/bin/observer --port 27035 --data-dir /obdat'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000055c8f6a78b55 in oceanbase::common::ObSystemConfigKey::ObSystemConfigKey (this=0x7f18c0450298) at ./src/share/config/ob_system_config_key.h:56
56	./src/share/config/ob_system_config_key.h: No such file or directory.
[Current thread is 1 (Thread 0x7f18c0477240 (LWP 2479184))]
[Switching to thread 1 (Thread 0x7f18c0477240 (LWP 2479184))]
#0  0x000055c8f6a78b55 in oceanbase::common::ObSystemConfigKey::ObSystemConfigKey (this=0x7f18c0450298) at ./src/share/config/ob_system_config_key.h:56
56	in ./src/share/config/ob_system_config_key.h
#0  0x000055c8f6a78b55 in oceanbase::common::ObSystemConfigKey::ObSystemConfigKey (this=0x7f18c0450298) at ./src/share/config/ob_system_config_key.h:56
#1  oceanbase::common::ObConfigStorage::load_all_configs(oceanbase::common::ObSystemConfig&)::$_0::operator(oceanbase::share::ObSQLiteRowReader&) const (this=<optimized out>, reader=...) at ./src/share/config/ob_config_storage.cpp:90
#2  std::__invoke_impl<int, oceanbase::common::ObConfigStorage::load_all_configs(oceanbase::common::ObSystemConfig&)::$_0&, oceanbase::share::ObSQLiteRowReader&>(std::__invoke_other, oceanbase::common::ObConfigStorage::load_all_configs(oceanbase::common::ObSystemConfig&)::$_0&, oceanbase::share::ObSQLiteRowReader&) (__f=..., __args=...) at ./deps/3rd/usr/local/oceanbase/devtools/lib/gcc/x86_64-redhat-linux/12/../../../../include/c++/12/bits/invoke.h:61
... (full backtrace continues, ending in thread start and clone) ...
---
exit_code: 0
elapsed_ms: 75693
ended_at: 2026-06-01T07:33:27.665Z
---

Additional Context

• Test Changes: Not specified.
• Other Environment Abnormalities/Changes: None reported (no clog disk full, manual interventions, or injected exceptions).
• Work Item Type: This appears to be an internal bug report, not a customer-reported issue.

Start with the preserved core dump using GDB and inspect ObSystemConfigKey::ObSystemConfigKey at src/share/config/ob_system_config_key.h:56, then follow ObConfigStorage::load_all_configs at src/share/config/ob_config_storage.cpp:90. Trace the call from ObDBMSSchedJobExecutor::run_dbms_sched_job for job_id=7. Done means the automatic statistics job no longer terminates observer or leaves port 27035 unavailable.