imicroseq / imicroseq/muse

Many JDBCConnectionExceptions are thrown when muse is under heavy load

Open
#86 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

critical
Dominant language
Java
Stars
1
Forks
2
PR merge metrics
No merged PRs in 30d

Description

Details:
- It looks like muse is unsuccessfully attempting to get JDBC connections for transactionals (see logs observed during heavy submission load bellow)
- JDBC connections is used for flyway but the data operation should be via R2DBC connections
- Most likely there is a mismatch between db drivers injection some where (or could be something else)
- the overload of exceptions eventually makes muse unhealthy and it gets restarted (which is bad if we have inflight submissions as they are lost)

Logs while muse was running during some submissions:

![image](https://user-images.githubusercontent.com/34698315/144094791-51a27ed8-7d6e-4ab5-8caa-82555bec640e.png)

Expected Behavior:
- muse should be consistent with its db connections

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the JDBC connection exceptions during heavy submission load and trace which transactional operations request JDBC connections instead of R2DBC connections. Use the reported logs as the initial symptom; done means database operations use the intended connection type and Muse remains healthy without losing inflight submissions.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.