MongoEngine / MongoEngine/mongoengine
Upsert on bulk insert
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.3k
- Forks
- 1.2k
- Avg merge
- 4h 41m
- Merged PRs (30d)
- 11
Description
I want to upsert on bulk insert.
Here is my code snippets.
from datetime import datetime
from mongoengine import *
connect('test')
class Exclusions(Document):
user_id = LongField(required=True)
target_user_id = LongField(required=True)
category = StringField(required=True)
created_at = DateTimeField(default=datetime.now().replace(microsecond=0))
meta = {
'index_background': True,
'indexes': [
{
'fields': ['+user_id'],
},
{
'fields': ('user_id', 'target_user_id'),
'unique': True
}
]
}
current_time = datetime.now().replace(microsecond=0)
ex1 = Exclusions(user_id=1, target_user_id=2, category='likes', created_at=current_time)
ex2 = Exclusions(user_id=3, target_user_id=4, category='likes', created_at=current_time)
go = []
go.append(ex1)
go.append(ex2)
Exclusions.objects.insert(go)
If I execute twice, it throw errors.
pymongo.errors.BulkWriteError: batch op errors occurred
mongoengine.errors.NotUniqueError: Document must not have _id value before bulk write (batch op errors occurred)
I know it is because of index. But I want to insert if it doesn't exist and update if it exists on bulk insert method.
Is there any options for it?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the Exclusions.objects.insert(go) entry point and trace MongoEngine's bulk-insert handling into the PyMongo bulk-write behavior. Define how repeated inserts should update documents matching the unique user_id and target_user_id pair, then verify that duplicates no longer produce the reported errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb, python
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100