USACE / USACE/cwms-data-api

Timeseries POST gives Null Pointer for null Values

Open
#876 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug timeseries
Dominant language
Java
Stars
28
Forks
25
Avg merge
4d 22h
Merged PRs (30d)
20

Description

Using POST for Timeseries to set NULL values

Background

Working with @inguyen314 we were discussing the finer points of how to replace-all values in a timeseries with null values.

The idea being you want to write some data from a future forecast, but other data you'd want to make sure was null. Or perhaps the forecast changes and those values are now null for those dates.

Agreement

We both agreed that one should be able to interlace null values in a valid POST request where you may also want to write valid values.

Workaround

One could of course call the DELETE method and specify the date range.

Reasons not to?

But in our humble opinion Why not both?

Example

Doing the following results in a NULL POINTER exception in the logs:

  const payloadDeleteGraysPt = {
                    "name": "Grays Pt-Mississippi.Stage.Inst.~1Day.0.netmiss-fcst",
                    "office-id": "MVS",
                    "units": "ft",
                    "values": [
                        [
                            getDateWithTimeSet(0, 6, 0),
                            null,
                            0
                        ],
                        [
                            getDateWithTimeSet(1, 6, 0),
                            8675309,
                            0
                        ],
                        [
                            getDateWithTimeSet(2, 6, 0),
                            8675309,
                            0
                        ],
                        [
                            getDateWithTimeSet(3, 6, 0),
                            null,
                            0
                        ],
                        [
                            getDateWithTimeSet(4, 6, 0),
                            null,
                            0
                        ],
                        [
                            getDateWithTimeSet(5, 6, 0),
                            null,
                            0
                        ],
                        [
                            getDateWithTimeSet(6, 6, 0),
                            null,
                            0
                        ],
                    ]

This was done in javascript. I do not have a CURL command readily available but here is the Vanilla JS done to test this:

async function deleteTS() {
    // Create an array of promises to handle multiple payloads
    let promises = payloadDelete.map(ts_payload => {
        return fetch("[https://wm.mvs.ds.usace.army.mil/mvs-data/timeseries?store-rule=REPLACE%20ALL"](https://wm.mvs.ds.usace.army.mil/mvs-data/timeseries?store-rule=REPLACE%20ALL%22), {
            method: "POST",
            headers: {
                "accept": "*/*",
                "Content-Type": "application/json;version=2",
            },
            body: JSON.stringify(ts_payload)
        }).then(async r => {
            // Get the response message and status
            const message = await r.text();
            const status = r.status;
            return { 'message': message, 'status': status };
        }).catch(error => {
            // Handle fetch errors
            return { 'message': error.message, 'status': 'fetch_error' };
        });
    });
 
    // Wait for all promises to resolve
    const return_values = await Promise.all(promises);
    console.log("Return values from deleteTS:", return_values);
 
    // Check for errors based on status and message content
    const has_errors = return_values.some(v => v.status !== 200 || v.message.includes("error") || v.message.includes("fail"));
    return has_errors;
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the timeseries POST endpoint using store-rule=REPLACE ALL and reproduce the request from the JavaScript example, including interlaced null values. Trace the request handling until the null pointer occurs; done means the request completes successfully and replaces the specified timeseries values with null or valid values as submitted.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, javascript
Domain
api, backend-api-design
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.