debezium / debezium/dbz

Use multiple flashback query threads for Oracle snapshots [DBZ-4918]

Open
#602 0 comments 0 reactions 0 assignees View on GitHub
component/oracle-connector migrated-from-jira type/enhancement
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-4918](https://issues.redhat.com/browse/DBZ-4918)

Hi, I am using debezium to read snapshot from oracle table which has 60 columns and 300000 rows and it's about 220MB. After many times testing I conclude that it should take 14 minutes to finish that snapshot task. I have try to change the option `snapshot.fetch.size` bigger or smaller but got no effection. So it would be greate if you can help to supply some advise to due with this situation.

Here's my configuration of the task.
{code:java}
// code placeholder
connector.class = io.debezium.connector.oracle.OracleConnector
database.tablename.case.insensitive = false
log.mining.strategy = online_catalog
include.schema.changes = false
tombstones.on.delete = false
log.mining.continuous.mine = true
database.history.skip.unparseable.ddl = true
database.history.instance.name = 5fa0ac61-7e7d-4ce8-923c-e704242dc35a
snapshot.fetch.size = 10000
database.user = xxx
database.dbname = xxx
database.url = jdbc:oracle:thin:@//xxx
database.server.name = oracle_logminer
schema.whitelist = xxx
offset.flush.interval.ms = 9223372036854775807
database.password = *****
name = engine
table.include.list = xxx.xxx
snapshot.mode = initial {code}

Contributor guide

Open the contributing guide

Research direction

Start with the OracleConnector snapshot path and the snapshot.fetch.size configuration described in the issue. Determine how Oracle snapshots currently execute and define the changes needed to use multiple flashback query threads; done means the requested snapshot completes with parallel query threads and its behavior is covered by appropriate tests.

Written by the indexing model from the issue text.

Assessment

Domain
database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.