EducationalTestingService / EducationalTestingService/Confero

Add screen capture video frame times as events in the iohub hdf5 file.

Open
#70 0 comments 0 reactions 0 assignees View on GitHub
idea Low Priority
Dominant language
Python
Stars
9
Forks
1
PR merge metrics
No merged PRs in 30d

Description

During post processing of the video files, a script is available which converts video frame times to the same time base as iohub events during the session.

Instead of saving out a separate file that has the video frame - iohub event mapping table in it, should we just treat video frames as another event type. The processing script could add a 'videocapture' device within the events section of the hdf5 file, and this device could have a ScreenShotEvent table in it. The script could add one event per video frame per file for each session processed. The ScreenShotEvent type could have the following fields (the important ones are listed here):

**session_id**
**recording_period_id**
**device_time**: in this case would be the original frame time provided from opencv
**time**: the converted device time to the iohub event time base.
**frame_number**: the opencv frame number returned for the frame

This could replace the extra npz file being created, and the processing module could be changed to support returning other iohub events selected based on any of:

**session_id**,
**video_id**,
**starting_frame_number**
**end_frame_number** (optional)
**start_video_time** : in event time base, so the time from some other event could be used as the start of the search time.
**end_video_time**: (optional) in event time base, so the time from some other event could be used as the end of the search time period.

I'll let you decide if this would be a good way of doing things based on how you will want to slice and dice data. If you want it this way, please assign priority 1 to this issue and assign back to me.

Thanks!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.