USACE / USACE/cwms-data-api

Investigate using AV_TSV directly and implementing logic of retrieve_ts in java/jooq

Open
#1,685 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Java
Stars
28
Forks
25
Avg merge
4d 22h
Merged PRs (30d)
20

Description

Is your feature request related to a problem? Please describe.
In review of the retrieve_ts methods, there are up 5 loops over the data to handle various scenarios like datum conversions or versioned data. This adds up rather high.

Describe the solution you'd like
It is likely possible using a combination of Java and JOOQ query building to conditionally implement those loops in a more performant way using the AV_TSV table directly, or more likely AV_TSV_DQU to handle unit conversions.

Describe alternatives you've considered
Attempt to speed up retrieve_ts... I don't really see that happening, the design does make sense if you're stuck within PLSQL

Additional context
This would also allow for easier streaming of results for such things as a csv or json-lines format.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the retrieve_ts methods in the Java code and tracing their loops for datum conversion and versioned data. Compare that flow with the AV_TSV and AV_TSV_DQU tables and the JOOQ query-building entry points; done means a decided, measured approach that preserves those scenarios and supports streaming results.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, databases, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.