SciCatProject / SciCatProject/frontend

Dataset size and numberOfFiles becomes incorrect when dataset is created with initial values before OrigDatablock creation

Open
#2,248 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
29
Forks
44
Avg merge
20h 2m
Merged PRs (30d)
20

Description

Summary

The current implementation updates the dataset size incrementally when a new OrigDatablock is created:

    const createOrigDatablock: CreateOrigDatablockDto = {
      ...createDatasetOrigDatablockDto,
      datasetId: pid,
      ownerGroup: dataset.ownerGroup,
      accessGroups: dataset.accessGroups,
      instrumentGroup: dataset.instrumentGroup,
    };
    const datablock =
      await this.origDatablocksService.create(createOrigDatablock);

    const updateDatasetDto: PartialUpdateDatasetObsoleteDto = {
      size: dataset.size + datablock.size,
      numberOfFiles: dataset.numberOfFiles + datablock.dataFileList.length,
    };
    await this.datasetsService.findByIdAndUpdate(dataset.pid, updateDatasetDto);

This assumes that dataset.size and dataset.numberOfFiles always represent the sum of existing OrigDatablocks.

However, users can create a dataset with a predefined size and numberOfFiles before any OrigDatablock is created. When the first OrigDatablock is later added, the dataset values are incremented on top of the existing values, resulting in incorrect totals.

Steps to Reproduce

Create dataset

/api/v3/datasets
{
 ...otherfileds,
 size: 1000,
 numberOfFiles: 2
}

Create origDatablock

api/v3/datasets/:pid/origdatablocks
{
  ...otherfields,
   "dataFileList": [
    {
      "path": "string/string",
      "size": 1111,
      "time": "2026-03-04T09:37:02.960Z",
      "chk": "teststring",
    }
  ],

}
Current Behaviour

Fetch created dataset

api/v3/datasets/:pid

{
 ...otherfields,
 size: 2111,
 numberOfFiles:3
}
Expected Behaviour

Fetch created dataset

api/v3/datasets/:pid

{
 ...otherfields,
 size: 1111,
 numberOfFiles:1
}
Extra Details

Provide all the relevant details related to this issues. Include screenshot, wireframes and mockups if helpful.

PS: make ssure to add the proper tags to the issue

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the /api/v3/datasets/:pid/origdatablocks flow, especially origDatablocksService.create and datasetsService.findByIdAndUpdate. Trace how the dataset's initial size and numberOfFiles are combined with the first OrigDatablock, then verify the dataset response after creating the dataset with initial values and adding one OrigDatablock. Done means the returned totals match the OrigDatablocks rather than double-counting the initial values.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.