OHDSI / OHDSI/WebAPI

Cohort Delete thows CleanupCohortTasket JDBC errors

Open
#2,334 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
151
Forks
183
Avg merge
14m
Merged PRs (30d)
2

Description

Expected behavior

(Using WebAPI 2.14.0 / Atlas 2.14.1)

When deleting a cohort, the org.ohdsi.webapi.cohortdefinition.CleanupCohortTasklet fires without error, and the cohort records are deleted from the pertinent cohort tables for all CDMs in which we've executed the cohort definition.

Actual behavior

(I've confirmed our JDBC connections and the service account's ability to delete from CDM results schemas.)

A few different things to cover here:

  1. It looks like this tasklet is attempted against all CDMs in WebAPI, even for those that haven't been executed. So we see JDBC errors for those, perhaps some additional logic to determine which to delete from would help here.

  2. The tasklet is unable to execute against Redshift:

ERROR taskExecutor-3 org.ohdsi.webapi.cohortdefinition.CleanupCohortTasklet - [] - Error deleting results for cohort: 992, cause: Could not get JDBC Connection; nested exception is java.sql.SQLNonTransientConnectionException: [Amazon][JDBC](10100) Connection Refused: [Amazon][JDBC](11640) Required Connection Key(s): PWD, UID; [Amazon][JDBC](11480) Optional Connection Key(s): AccessKeyID, AuthMech, AutoCreate, BlockingRowsMode, ClusterID, DbGroups, DisableIsValidQuery, DriverLogLevel, EndpointUrl, FilterLevel, IAMDuration, Language, loginTimeout, OpenSourceSubProtocolOverride, plugin_name, profile, Region, SecretAccessKey, SessionToken, socketTimeout, ssl, sslcert, sslfactory, sslkey, sslpassword, sslrootcert, SSLTruststore , SSLTrustStorePath, tcpKeepAlive, TCPKeepAliveMinutes, unknownLength

  1. The tasklet is unable to execute against Snowflake. The JDBC connection has a user name, so not sure why this is an error:

ERROR taskExecutor-3 org.ohdsi.webapi.cohortdefinition.CleanupCohortTasklet - [] - Error deleting results for cohort: 992, cause: Could not get JDBC Connection; nested exception is net.snowflake.client.jdbc.SnowflakeSQLLoggedException: JDBC driver internal error: missing user name for opening session.

  1. I've run the deletes from https://github.com/OHDSI/WebAPI/blob/v2.14.0/src/main/resources/resources/cohortdefinition/sql/cleanupResults.sql manually with the same JDBC connection details without issue.
Steps to reproduce behavior
  1. Create and generate a cohort
  2. Delete it
  3. View webapi logs
  4. Also you can see the cohort records still exist in the appropriate CDM results schemas

Tagging @konstjar

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with org.ohdsi.webapi.cohortdefinition.CleanupCohortTasklet and the referenced resources/cohortdefinition/sql/cleanupResults.sql. Reproduce the cohort deletion, inspect WebAPI logs for Redshift and Snowflake JDBC errors, and compare tasklet behavior with the manual deletes. Done means deleting a cohort cleans its records from the appropriate CDM results schemas without errors for unexecuted or affected CDMs.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.