loopbackio / loopbackio/loopback-next

Robust handling of ObjectID type for MongoDB

Open
#3,720 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

epic major Repository
Dominant language
TypeScript
Stars
5.1k
Forks
1.1k
Avg merge
2d 21h
Merged PRs (30d)
27

Description

MongoDB is tricky - see https://github.com/strongloop/loopback-next/issues/1875

- It uses a custom `ObjectID` type for primary keys.
- `ObjectID` is represented as a `string` when converted to JSON
- In queries, string values must be cast to ObjectID, otherwise they are not considered as the same value: `'some-id' !== ObjectID('some-id')`.

As a result, both PK and FK properties must use `ObjectID` as the type, and coercion must be applied where necessary.

Ideally, I'd like LB4 to define MongoDB PK and FKs as follows:

- `{type: 'string', mongodb: {dataType: 'ObjectID'}}`

Even better, `dataType: 'ObjectID'` should be automatically applied by the connector for PK and FKs referencing ObjectID PKs.

For example:

```ts
@model()
class Product {
@property({
type: 'string',
generated: true,
// ^^ implies dataType: 'ObjectID'
})
id: string;

@property({
type: 'string',
references: {
model: () => Category,
property: 'id',
},
// ^^ implies dataType: 'ObjectID' when Category is attached to MongoDB
})
categoryId: string;
}
```

For v1, I suppose we can ask developers to provide dataType manually.

```ts
@model()
class Product {
@property({
type: 'string',
generated: true,
mongodb: {dataType: 'ObjectID'},
})
id: string;

@property({
type: 'string',
mongodb: {dataType: 'ObjectID'},
})
categoryId: string;
}
```

With this setup in place, `id` and `categoryId` properties should be always returned as strings from DAO and connector methods.

## Related discussions

- The issue that started the discussion for LB4: Type ObjectID for model property #1875
- Model inclusion resolvers are tricky when ObjectID gets involved - see https://github.com/strongloop/loopback-next/pulls
- Long-term proposal for fixing ObjectID in LB 3.x: https://github.com/strongloop/loopback/issues/1874
- Short-term fix for LB 3.x that was never finished: https://github.com/strongloop/loopback-datasource-juggler/pull/778
- Recent improvements in loopback-connector-mongodb to honor `dataType: 'mongodb'`: https://github.com/strongloop/loopback-connector-mongodb/pull/517 and https://github.com/strongloop/loopback-connector-mongodb/pull/525

## Acceptance criteria

- For every property defined as `{type: 'string', mongodb: {dataType: 'ObjectID'}}`, including properties defined in nested/embedded models:

- When the MongoDB connector returns data from database, it converts ObjectID values to strings.
- When the MongoDB connector writes data to database, it converts string values to ObjectID
- When the MongoDB connector queries database (think of `filter.where`, but also `findById` and `replaceById`), it converts string values to ObjectID. The conversion is applied to non-trivial conditions too, e.g. `{where: {id: { inq: ['my-objectid-1', 'my-objectid-2'] }}}`

- Documentation page for MongoDB users explaining extra configuration needed

- Blog post announcing the improvements

**Tasks**

- Model.toObject() should preserve prototypes (e.g. Date and ObjectID values) #3607
- Spike (initial research & PoC implementation): https://github.com/strongloop/loopback-next/issues/3456

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin with the spike in issue #3456 and the linked MongoDB connector discussions; inspect connector handling for returned data, writes, filter.where, findById, and replaceById, along with the Model.toObject() task #3607. Done means ObjectID values are converted in all listed cases, including nested models and complex conditions, with MongoDB user documentation and a blog post added.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, typescript
Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.