loopbackio / loopbackio/loopback-next
Robust handling of ObjectID type for MongoDB
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 5.1k
- Forks
- 1.1k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 27
Description
MongoDB is tricky - see https://github.com/strongloop/loopback-next/issues/1875
- It uses a custom `ObjectID` type for primary keys.
- `ObjectID` is represented as a `string` when converted to JSON
- In queries, string values must be cast to ObjectID, otherwise they are not considered as the same value: `'some-id' !== ObjectID('some-id')`.
As a result, both PK and FK properties must use `ObjectID` as the type, and coercion must be applied where necessary.
Ideally, I'd like LB4 to define MongoDB PK and FKs as follows:
- `{type: 'string', mongodb: {dataType: 'ObjectID'}}`
Even better, `dataType: 'ObjectID'` should be automatically applied by the connector for PK and FKs referencing ObjectID PKs.
For example:
```ts
@model()
class Product {
@property({
type: 'string',
generated: true,
// ^^ implies dataType: 'ObjectID'
})
id: string;
@property({
type: 'string',
references: {
model: () => Category,
property: 'id',
},
// ^^ implies dataType: 'ObjectID' when Category is attached to MongoDB
})
categoryId: string;
}
```
For v1, I suppose we can ask developers to provide dataType manually.
```ts
@model()
class Product {
@property({
type: 'string',
generated: true,
mongodb: {dataType: 'ObjectID'},
})
id: string;
@property({
type: 'string',
mongodb: {dataType: 'ObjectID'},
})
categoryId: string;
}
```
With this setup in place, `id` and `categoryId` properties should be always returned as strings from DAO and connector methods.
## Related discussions
- The issue that started the discussion for LB4: Type ObjectID for model property #1875
- Model inclusion resolvers are tricky when ObjectID gets involved - see https://github.com/strongloop/loopback-next/pulls
- Long-term proposal for fixing ObjectID in LB 3.x: https://github.com/strongloop/loopback/issues/1874
- Short-term fix for LB 3.x that was never finished: https://github.com/strongloop/loopback-datasource-juggler/pull/778
- Recent improvements in loopback-connector-mongodb to honor `dataType: 'mongodb'`: https://github.com/strongloop/loopback-connector-mongodb/pull/517 and https://github.com/strongloop/loopback-connector-mongodb/pull/525
## Acceptance criteria
- For every property defined as `{type: 'string', mongodb: {dataType: 'ObjectID'}}`, including properties defined in nested/embedded models:
- When the MongoDB connector returns data from database, it converts ObjectID values to strings.
- When the MongoDB connector writes data to database, it converts string values to ObjectID
- When the MongoDB connector queries database (think of `filter.where`, but also `findById` and `replaceById`), it converts string values to ObjectID. The conversion is applied to non-trivial conditions too, e.g. `{where: {id: { inq: ['my-objectid-1', 'my-objectid-2'] }}}`
- Documentation page for MongoDB users explaining extra configuration needed
- Blog post announcing the improvements
**Tasks**
- Model.toObject() should preserve prototypes (e.g. Date and ObjectID values) #3607
- Spike (initial research & PoC implementation): https://github.com/strongloop/loopback-next/issues/3456
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin with the spike in issue #3456 and the linked MongoDB connector discussions; inspect connector handling for returned data, writes, filter.where, findById, and replaceById, along with the Model.toObject() task #3607. Done means ObjectID values are converted in all listed cases, including nested models and complex conditions, with MongoDB user documentation and a blog post added.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb, typescript
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100