MongoEngine / MongoEngine/mongoengine
Calling `.reload()` on an object before `FileField.replace` breaks the FileField
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.3k
- Forks
- 1.2k
- Avg merge
- 4h 41m
- Merged PRs (30d)
- 11
Description
Setup:
from mongoengine.document import Document
from mongoengine.fields import StringField, FileField
import pickle
import pandas as pd
class TestObject(Document):
name = StringField(primary_key=True)
file = FileField()
testdata1 = pd.DataFrame([[1, 2, 3], [0, 4, 6], [9, 3, 5]], columns=['A', 'B', 'C'])
testdata2 = pd.DataFrame([[5, 6, 7], [2, 4, 8], [0, 6, 1]], columns=['A', 'B', 'C'])
Minimal working example:
db_obj = TestObject(name='test1')
db_obj.file = pickle.dumps(testdata1)
db_obj.save()
db_obj.file.replace(pickle.dumps(testdata2))
db_obj.save()
print(pickle.loads(TestObject.objects().get(name='test1').file.read()))
Prints:
A B C
0 5 6 7
1 2 4 8
2 0 6 1
Minimal broken example:
db_obj = TestObject(name='test2')
db_obj.file = pickle.dumps(testdata1)
db_obj.save()
db_obj.reload() ##
db_obj.save() ##
db_obj.file.replace(pickle.dumps(testdata2))
db_obj.save()
print(pickle.loads(TestObject.objects().get(name='test2').file.read()))
Raises:
TypeError: a bytes-like object is required, not 'NoneType'
I compared the file ids for the local and DB objects at each step using:
print(db_obj.file, TestObject.objects().get(name='test1').file)
In both cases, the file IDs stay the same until the .replace:
<GridFSProxy: None (5f8cdc7ed4010827a9cac125)> <GridFSProxy: None (5f8cdc7ed4010827a9cac125)>
After the .replace, the database loses its file, while the local object has a new one:
<GridFSProxy: None (5f8cdc7ed4010827a9cac127)> <GridFSProxy: <no file> (5f8cdc7ed4010827a9cac125)>
In the working example, the .save() resolves this discrepancy:
<GridFSProxy: None (5f8cdc7ed4010827a9cac127)> <GridFSProxy: None (5f8cdc7ed4010827a9cac127)>
But in the broken example, there is no change after .save(), and the file ID discrepancy cannot be resolved. If other fields are included on the object and modified, they update properly with that final .save(), but the FileField does not.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the minimal broken example with FileField, reload(), save(), and replace(), then inspect the FileField and GridFSProxy behavior across those calls. Add a regression test covering the reload-before-replace sequence and verify that the replacement file remains available after the final save.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb, python
- Domain
- backend, database
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100