MongoEngine / MongoEngine/mongoengine

Calling `.reload()` on an object before `FileField.replace` breaks the FileField

Open
#2,397 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.3k
Forks
1.2k
Avg merge
4h 41m
Merged PRs (30d)
11

Description

Setup:

from mongoengine.document import Document
from mongoengine.fields import StringField, FileField
import pickle
import pandas as pd

class TestObject(Document):
    name = StringField(primary_key=True)
    file = FileField()

testdata1 = pd.DataFrame([[1, 2, 3], [0, 4, 6], [9, 3, 5]], columns=['A', 'B', 'C'])
testdata2 = pd.DataFrame([[5, 6, 7], [2, 4, 8], [0, 6, 1]], columns=['A', 'B', 'C'])

Minimal working example:

db_obj = TestObject(name='test1')
db_obj.file = pickle.dumps(testdata1)
db_obj.save()
db_obj.file.replace(pickle.dumps(testdata2))
db_obj.save()
print(pickle.loads(TestObject.objects().get(name='test1').file.read()))

Prints:

   A  B  C
0  5  6  7
1  2  4  8
2  0  6  1

Minimal broken example:

db_obj = TestObject(name='test2')
db_obj.file = pickle.dumps(testdata1)
db_obj.save()
db_obj.reload()  ##
db_obj.save()  ##
db_obj.file.replace(pickle.dumps(testdata2))
db_obj.save()
print(pickle.loads(TestObject.objects().get(name='test2').file.read()))

Raises:

TypeError: a bytes-like object is required, not 'NoneType'

I compared the file ids for the local and DB objects at each step using:

print(db_obj.file, TestObject.objects().get(name='test1').file)

In both cases, the file IDs stay the same until the .replace:

<GridFSProxy: None (5f8cdc7ed4010827a9cac125)> <GridFSProxy: None (5f8cdc7ed4010827a9cac125)>

After the .replace, the database loses its file, while the local object has a new one:

<GridFSProxy: None (5f8cdc7ed4010827a9cac127)> <GridFSProxy: <no file> (5f8cdc7ed4010827a9cac125)>

In the working example, the .save() resolves this discrepancy:

<GridFSProxy: None (5f8cdc7ed4010827a9cac127)> <GridFSProxy: None (5f8cdc7ed4010827a9cac127)>

But in the broken example, there is no change after .save(), and the file ID discrepancy cannot be resolved. If other fields are included on the object and modified, they update properly with that final .save(), but the FileField does not.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the minimal broken example with FileField, reload(), save(), and replace(), then inspect the FileField and GridFSProxy behavior across those calls. Add a regression test covering the reload-before-replace sequence and verify that the replacement file remains available after the final save.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, python
Domain
backend, database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.