boltgolt / boltgolt/howdy

How does IR face recognition work with a model trained on RGB?

Open
#930 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
7.8k
Forks
412
PR merge metrics
No merged PRs in 30d

Description

Sorry, this might not strictly qualify as an issue but it feels like the documentation is a bit lacking in this context and I don't see any other place to ask...

I've been trying to make sense out of how Howdy works on the inside but I am struggling a bit.

As far as I can tell the DLIB model used by `howdy` has been trained on RGB images of people (https://github.com/davisking/dlib-models#dlib_face_recognition_resnet_model_v1datbz2) but howdy is supposed to use images taken with an IR webcam. The `VideoCapture` class seems to return both an RGB frame as well as a grayscale frame (https://github.com/boltgolt/howdy/blob/beta/howdy/src/recorders/video_capture.py#L97) which is confusing since OBS is giving me what I would call grayscale when using my IR camera (where is the RGB coming from?). Howdy then proceeds to use the grayscale frame for face detection (https://github.com/boltgolt/howdy/blob/beta/howdy/src/cli/add.py#L171) but the RGB frame for getting the actual encoding of the face (https://github.com/boltgolt/howdy/blob/beta/howdy/src/cli/add.py#L201) which feels quite sketchy since it would defeat the whole thing with the IR camera...

Could someone please explain this to me? What am I missing?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with howdy/src/recorders/video_capture.py around line 97, then trace the grayscale and RGB frames through howdy/src/cli/add.py around lines 171 and 201. Compare those paths with the linked dlib RGB model documentation and document how IR input is handled and whether the encoding path is valid.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
authentication, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.