SciCatProject / SciCatProject/backend

Ingestion of two proposals without attached datasets fails

Open
#151 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
TypeScript
Stars
27
Forks
37
Avg merge
17h 32m
Merged PRs (30d)
38

Description

Issue Name

Ingestion of two proposals without attached datasets fails

Summary

We tried to ingest two proposals in an empty database created by scicat-backend-next, which turned out not to be possibile due to an index error in the mongo db.

Steps to Reproduce
  1. Ingest a proposal e.g. using the same metadata as in the tests
{
  "proposalId": "first_proposal",
  "email": "proposer%40uni.edu",
  "title": "A test proposal",
  "abstract": "Abstract of test proposal",
  "ownerGroup": "20170251-group",
  "MeasurementPeriodList": []
}

the response will be code 201 with the following body

{
  "ownerGroup": "20170251-group",
  "accessGroups": [],
  "proposalId": "first_proposal",
  "email": "proposer%40uni.edu",
  "title": "A test proposal",
  "abstract": "Abstract of test proposal",
  "MeasurementPeriodList": [],
  "attachments": [],
  "datasets": [],
  "_id": "first_proposal",
  "__v": 0,
  "id": "first_proposal"
}
  1. Try to ingest a second dataset with modified id
{
  "proposalId": "second_proposal",
  "email": "proposer%40uni.edu",
  "title": "A test proposal",
  "abstract": "Abstract of test proposal",
  "ownerGroup": "20170251-group",
  "MeasurementPeriodList": []
}

the answer will be a code 500

{
  "statusCode": 500,
  "message": "Internal server error"
}

and in the server logs we see

MongoServerError: E11000 duplicate key error collection: scicat.Proposal index: datasets.pid_1 dup key: { datasets.pid: null }
  1. Manually playing the database e.g. chaning the entry of the first entry from
{
    _id: 'first_proposal',
    ownerGroup: '20170251-group',
    accessGroups: [],
    proposalId: 'first_proposal',
    email: 'proposer%40uni.edu',
    title: 'A test proposal',
    'abstract': 'Abstract of test proposal',
    MeasurementPeriodList: [],
    attachments: [],
    datasets: [],
    __v: 0
}

to

    datasets: [
        {
            pid: 'something'
        }
    ]

enables the ingestion of the second dataset.

Extra Details

To me it looks like the unique indes on "datasets.pid" in the Proposal collection realy is a problem.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the two-proposal ingestion sequence against an empty scicat-backend-next database and inspect the MongoDB Proposal collection, especially the unique datasets.pid_1 index. Trace the TypeScript proposal ingestion path and existing proposal tests, if present. Done means multiple proposals with empty datasets can be ingested without a duplicate-key error and the regression is covered by a test.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, typescript
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.