[helm] source-oracle OOM-killed on abctl/kind 2.1.0 — JDBC buffer estimator reads host memory (25GB) instead of the 2Gi container limit
- Dominant language
- Python
- Stars
- 22.1k
- Forks
- 5.3k
- PR merge metrics
- PR metrics pending
Description
### Helm Chart Version
2.1.0
### What step the error happened?
During the Sync
### Relevant information
## Summary
On a self-managed abctl (kind) deployment, source-oracle reads are consistently OOM-killed on a small table (~72,466 rows / ~89 MB). The TwoStageSizeEstimator reports a max memory of ~25 GB (the host's total RAM) despite the source pod being launched with a 2Gi container limit. The JDBC buffer target is sized ~15 GB and the adaptive fetch size climbs to ~4.17 million rows, so the Oracle driver preallocates far beyond 2Gi and the pod is kernel-OOM-killed at ~70,000 records every run.
No documented memory control changes the reported max memory — it stays at ~25 GB regardless. This looks like the connector JVM not detecting the container cgroup memory limit in a kind/nested-container environment.
Possibly related to (but distinct from) the older frozen issue #27846 — this is specifically the TwoStageSizeEstimator reading host RAM instead of the 2Gi cgroup limit under abctl/kind, not a gradual leak.
## Environment
- abctl (kind single-node), Airbyte App/Chart version 2.1.0
- Host: OCI VM, ~25 GB RAM
- source-oracle: reproduced on both 0.5.7 and 0.5.8 (identical failure)
- destination-postgres 3.0.13, orchestrator/workload-init 2.1.0
- Sync mode: Full refresh | Append; 1 stream, ~72,466 rows, ~89 MB, 46 columns
## Timeline
- DB in continuous use since ~Nov 2025 across earlier Airbyte version(s).
- Current kind cluster rebuilt 2026-05-17 onto platform 2.1.0, carrying the existing DB forward.
- OOM on this table appears after the move to 2.1.0; other connections on the same instance sync fine.
## What I already ruled out (all evidence-confirmed)
1. JOB_MAIN_CONTAINER_MEMORY_LIMIT set to 6Gi — present in workload-launcher env, but source pod still launches at 2Gi.
2. CONNECTOR_SPECIFIC_RESOURCE_DEFAULTS_ENABLED=false — confirmed live; source pod still 2Gi.
3. DB-level resource_requirements — null on actor_definition, actor, and connection rows.
4. defaultRowPrefetch in JDBC params — fetch size still climbs to ~4.17M rows (per-statement setFetchSize overrides it).
5. JOB_DEFAULT_ENV_JAVA_OPTS=-Xmx1500m on the launcher — applied and restarted; source JVM still reports ~25GB max memory.
6. Connector version 0.5.7 vs 0.5.8 — identical failure; changelog shows no read-path change between them.
## Questions
- Is there a supported way to force the source connector JVM to honor the container cgroup limit (or a hard -Xmx) on abctl/kind 2.1.0?
- Is the workload-launcher expected to ignore JOB_MAIN_CONTAINER_MEMORY_LIMIT / CONNECTOR_SPECIFIC_RESOURCE_DEFAULTS_ENABLED for connector pods in 2.1.0?
- Should TwoStageSizeEstimator fall back to the cgroup limit rather than Runtime.maxMemory() when containerized?
- Interim: this stream has a JOURNAL_DATE cursor — would switching to incremental avoid the large-fetch preallocation, or does the initial backfill hit the same OOM?
### Relevant log output
```shell
[source] image: airbyte/source-oracle:0.5.8 resources: limits={memory=2Gi, cpu=2}, requests={memory=1Gi, cpu=1}
TwoStageSizeEstimator(getTargetBufferByteSize) Max memory limit: 25232932864, JDBC buffer size: 15139759718
AdaptiveStreamingQueryConfig(accept) Set new fetch size: 4170733 rows
Records read: 70000 (82 MB)
No exit code file found after waiting 10s. Container may have been killed (OOM or other signal). Returning fallback exit code 1.
Source process exited with non-zero exit code 1
sync summary: recordsEmitted 72405, recordsCommitted 0, status failed, failureOrigin source
```
---
**Internal Tracking:** https://github.com/airbytehq/oncall/issues/13107
Contributor guide
Assessment
This issue has not been assessed yet.