karpathy / karpathy/convnetjs

Data input is important for perception algo of FSD?

Open
#125 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
11.2k
Forks
2.1k
PR merge metrics
No merged PRs in 30d

Description

Hello,

I'm very interested about the github course.
and I'm wondoring about data input of the FSD autonomous driving algorithm perception solution, based on RGB image data as input or Camera raw as data input?

Here is some matetials on the AI day.
On the 2021 AI day,it shows that Perception algo is based the camera raw data ,with the type "Bayer raw ,12bit ,no tonemapping". However ,On the 2022 AI day, it shows that
perception algo is based the rgb(png/jpg) data ,with the type "rgb ,8bit ,isp processed".

My questions about the data input as follows:
Q1:So the data input type description above is true ? what is the best data type for perception input.
Q2:Shadow mode collects data such as Video clips after isp process and encode process,so the video image is the input for training? if not ,with the bayer raw as input,how to reduce the distribution gap between original raw data and rgb data。

Thank you very much.
![20240807-102553](https://github.com/user-attachments/assets/7504b46e-dbfe-4d20-9b8c-eb60ee9d363b)
![20240807-102548](https://github.com/user-attachments/assets/1d6e5d35-a5ff-4330-939f-86cd77e3f573)

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no repository files, tests, or entry points; it asks general questions about Bayer raw versus ISP-processed RGB inputs for perception and shadow-mode training. First verify whether this repository contains related documentation or code, then seek maintainer guidance; done would require an accepted, repository-specific explanation or documentation target.

Written by the indexing model from the issue text.

Assessment

Domain
autonomous-driving, machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
10/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.