Raspberry PI's ISP (libcamera/picamera2) crops IMX500's inference detection region when cropping parameters of feed are inputted with cropping coordinates.

Open
#1,204 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
30/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
python, raspberry-pi
Domain
computer-vision, embedded-iot

Research direction

Start with the picamera2.setcontrols("ScalerCrop") call and the interaction between ScalerCrop, the displayed video feed, and IMX500 object-detection coordinates. Check the picamera2 and libcamera behavior for whether cropping changes the inference region or only the output stream. Done means confirming a supported API or documenting that the requested full-resolution inference behavior is unavailable.

Written by the indexing model from the issue text.

Description

Hello,

I am using picamera2 library (Raspberry Pi's ISP) to crop and show video feed of IMX500 based on people detection (person detected coordinates) obtained from AI inference on IMX500 using object detection model.

Ideal situation, when I am using ISP cropping method using picamera2 for showing cropped video feed output (x,y,w,h) of custom coordinates, it should show cropped video feed and at the same time, AI detection running on the full resolution which is 4056 * 3040. But this is not the case. When I feed (x,y,w,h) coordiantes for showing cropped video streaming, it runs AI inferencing on (x,y,w,h) coordinates rather than running inference on full resolution (4056*3040).

I am using picamera2.setcontrols("ScalerCrop":(x,y,w,h,))
Please let me know if I am missing something or API dedictaed for above application or this is a missing functionality in using ISP of PI.

Dominant language
Python
Stars
1.2k
Forks
258
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from raspberrypi/picamera2

All issues in raspberrypi/picamera2

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.