collective / collective/collective.exportimport

Timezone dropped from non-naive datetimes during conversion

Open
#206 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
19
Forks
21
PR merge metrics
No merged PRs in 30d

Description

Hey folks!

I'm seeing that the default serialization / conversion of a datetime is dropping a timezone.

The effect will be an incorrect time during import. (unless you're British and it's winter).

A localized time is exported to json without a timezone, then imported as naive, which will likely become a UTC timezone,. and then an incorrect on the imported system.

For example, if you inspect a datetime field in 'global_dict_hook()'

contentexport/export_content.py(110)global_dict_hook()

(Pdb) obj.effective
<bound method DexterityContent.effective of <Item at /RFA/english/1_ly1s3g8j_1_2i53wwps_1.mp4>>

(Pdb) obj.effective()
DateTime('2023/05/25 09:55:00 GMT-4')

(Pdb) obj.EffectiveDate()
'2023-05-25T09:55:00-04:00'

(Pdb) item['effective']
'2023-05-25T09:55:00' 

Note that item['effective'] is a string with the localized time 9:55AM but the "Eastern Time" or rather "UTC-4" offset was dropped.

This essentially becomes 9:55AM UTC on import. 3:55AM Eastern. Resulting in the time changing.

Should this be a fix here, in exportimport, or go upstream to restapi?
I don't want to break Volto. This issue may be pedantic to the React UI, but it's essential for the correctness of data migrations.

Thoughts?

My field was a Zope DateTime, so this was appropriate. Maybe datetimes are ok when you don't muck around as much as I do?

def global_dict_hook()
    ....     
    effective_date = obj.EffectiveDate()
    if effective_date:
        date = obj.effective().utcdatetime()  #convert to UTC before dropping timezone info.
        date = (f"{date.isoformat('T','seconds')}")
        item['effective'] = date

May 30:

Interesting, that in the exported file... some dates are good, some are not.

This is a test object. Created and published nearly instantly to create some test data.

Note how some dates come out as UTC, but the "UI important ones" come out as Localized and Naive:

"created": "2023-05-15T14:41:17+00:00",
...
"effective": "2023-05-15T10:45:00",
"expires": "2025-05-16T10:45:00",
...
"modified": "2023-05-15T14:45:24+00:00"

May I be so bold as to put forth a theory that this IS a modification to the restAPI to make the frontend better, but was fixed in the wrong place?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at contentexport/export_content.py:110 and inspect global_dict_hook() to trace how localized Zope DateTime values are serialized. Compare the export/import path with the mentioned exportimport and restapi layers to identify where the offset is lost. Done means a localized datetime retains enough timezone information for an import round trip without changing the represented time.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.