drogonframework / drogonframework/drogon

Eager Loding/Relationship Caching

Open
#1,028 10 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
14.3k
Forks
1.4k
Avg merge
1d 13h
Merged PRs (30d)
14

Description

**Is your feature request related to a problem? Please describe.**
The raltionships are not cached by the ORM, which leads to unecessary DB access. Also not having a eager loading system can cause major performance issues.

**Describe the solution you'd like**
Add in the generated model file a way to eager load and use the cached relationship (this second part can be controlled by the model.json file for each relationship)

**Describe alternatives you've considered**
I was think something similar to Eloquent from Laravel:
```php
$user = User::query()
->with('posts', function(Builder $query) {
$query->with('comments')
->with('banner', function(Builder $query) {
$query->with('metadata');
});
})
->get();
```
This will run 5 queries, one for users, one for posts, one for the comments, one for the banner and one for metadata.
But since I can't think a way to load the relationship only using the name as a string a thought in generate some static methods in the model class for each relationship so it can be eager loaded.
Instead of passing a Query Builder to the anonymous function
```c++
DbClientPtr client = drogon::app().getDbClient();
Mapper mp(client);
std::vector users = mp.findAll()
Users::loadPosts(users, [](std::vector& posts) {
Posts::loadComments(posts);
Posts::loadBanner(posts, [](std::vector& banners) {
Banners::loadMetadata(banners);
})
});
```
The type of the static function would be `void(std::vector& model)` (It can't be const std::vector& because the entities inside the vector need to be modified).
As you can see the callback is optional, because maybe we don't want to load nothing after that relationship.
The eager loading functions only accepts a vector because N+1 problems don't occur when you have a single model and want to retrive it's relationship.
The generated sql for the eager loading would be something like this:
```sql
select * from posts where user_id IN ($1, $2, ...);
select * from comments where post_id IN ($1, $2, ...);
select * from banners where post_id IN ($1, $2, ...);
select * from meta_data where banner_id IN ($1, $2, ...);
```
After this I would loop over the relationships and distribute then for the correct parent.
Everything would be stored in a new member variable inside each model (`std::shared_ptr`). For example:
1. Users model would have `std::vector`
2. Posts model would have `std::vector`, `Banner` and `User` (inverse relationship);
3. Banners model would have `Metadata`
4. Metadata model would have `Banner` (inverse relationship)

**OBS**: The inverse relationships would not be loaded since would be a nightmare to manage the references

Now for retrive this relationships would be something like:
```c++
user.getPosts(client,
[](std::vector& posts) {},
[](const DrogonDbException& e) {}); //this would still run the query and after it sets the posts member inside user
//It's passed by reference because it's a reference to the posts member inside user

std::vector& posts = user.getPosts(); //this would only retrive from the member inside user
```
**Not determined**
1. For optional relationships I was thinking in using `std::optional` as a return type, but this can also be and exception (because std::optional is not widly used in the framework). If this is the case the optionality of the relationship would be configured in the model.json (this is more a problem for has one/belongs to relationships, because for has many we can just return a empty vector).
2. I was also thinking about change the sql in the relationship functions to use the `Mapper` functions, opnions are also welcome.
3. I'm still thinking about many to many relationships
4. Having a `std::future` as a way to retrive the relationship, similar to:
```c++
Mapper mp(client);
std::future user = mp.findFutureByPrimaryKey(id);
```
It would be:
```c++
Mapper mp(client);
Users user = mp.findByPrimaryKey(id);
std::future&> posts = user.getFuturePosts(client);
//this case would be similar to the new way of retriving and updating the cache but it would result in more redable code
//and would throw an exception if something goes wrong
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.