Video Timestamping Bugs with GoPro MAX

Extracting timestamps from GoPro MAX video files is basically completely broken in mapillary_tools. However, it is not purely mapillary_tools’ fault. Multiple bugs and design blunders compound here.

1. GoPro MAX has no timezone support.
   Hence, its RTC basically always runs in local time. Of course, only as long as the user has set the RTC correctly. This is the default case.
   1. Rarely occasionaly, the user may deliberately set the RTC to UTC. However, there is no way to tell programmatically. Well, maybe if you are ready to do some algorithmic educated guessing against the GPMF’s GPS clock signal.
2. The extracted video start timestamp is always strangely many seconds or more off from creation_time (the RTC timestamp), even when you have properly synced the RTC via https://gopro.github.io/labs/control/precisiontime before video recording and/or the video file has no GPS signal in its GPMF stream.
3. Because GoPro MAX has no timezone support creation_time is always written in local time, although the spec requires creation_time timestamps to be in UTC. So, this is broken by design by GoPro. Again, unless the user deliberately sets the RTC to UTC you may get away with everything.
4. When you sample_video the frames are timestamped to UTC (OffsetTimeOriginal == +00:00) but DateTimeOriginal and GPSDateStamp/GPSTimeStamp are falsely written local time values and are weirdly off from GPMF timestamps. It is a nonissue if the user has set the RTC to UTC.
5. When you video_process a video file that has a GPMF stream but no clock signal in it with a GPX file then mapillary_tools errors out with [{"error":{"message":"Empty GPS data found","type":"MapillaryGPXEmptyError"},"filename":"video.mp4","filetype":"video"}]. In this case, the GPS data should be sourced from the GPX file as specified. A GPX file and a video file without a GPMF stream always work fine though.
6. GoPro MAX always writes an RTC sourced timecode, which fortunately is in sync with the creation_time tag and is also the preferred means by which to sync video edits. The RTC is and actually should be the source for proper time keeping on GoPro MAX.

So, as you can see it is a total mess, mostly caused by GoPro though. For the least amount of fuss, a user should record GPMF videos with GPS positions and clock signal or set the RTC to UTC in order to work around all this hassle. The vast majority of users however understandably do not set the RTC to UTC.

Why All of This Actually Matters?

Unless a contributor actually pays attention to timezones or only uploads video files with GPS and clock signal data in the GPMF stream, there is a high chance that already many sequences have been uploaded with timestamps in the incorrect timezone. Of course, this does not apply to users who capture in the UTC timezone by default and set the RTC correctly.

What Needs a Fix

• sample_video for GoPro MAX and cameras that write local time to creation_time.
  Although users of such cameras can use the --video_start_time option to rectify timestamping, it uses a proprietary timestamp format in UTC only. It might be perhaps better to add a --timezone option. Alternatively, make the --video_start_time (also) accept the ISO 8601 format because it includes timezone support.

  GoPro MAX is a Mapillary recommended camera. Hence, imho this camera should receive extra care and attention, despite or perhaps because of all its quirks. Not all users upload GPMF with GPS data video files directly. MAX2 solves all these issues through full timezone support.

  • The strange time discrepancy between GPMF sourced timestamps and RTC sourced creation_time tag and timecode timestamps.

• video_process: GPMF video files without GPS data should fall back to sourcing data from --geotag_source gpx.

Start with the sample_video and video_process entry points and trace how GoPro MAX creation_time, GPMF timestamps, and --geotag_source gpx are handled. Reproduce the reported cases, including a GPMF stream without a clock signal, then define completion as correct timezone-aware timestamping and GPX fallback without the reported error.