justadudewhohacks / justadudewhohacks/opencv4nodejs

How to work with the desktop screen using robotjs and opencv4nodejs?

Open
#754 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
5.1k
Forks
826
PR merge metrics
No merged PRs in 30d

Description

I used one of the examples. I want to take a screenshot of the desktop.
And find on it the picture I'm uploading. The problem is that I do not understand how to correctly convey the screenshot of the desktop. I am getting a gray window and an incorrect image.

```
const cv = require('opencv4nodejs');
const robot = require('robotjs');
var Jimp = require( 'jimp' );

const findWaldo = async () => {

var screen = robot.screen.capture(0, 0, 1024, 768); // I take screenshot here

const originalMat = new cv.Mat(screen.image, 1024, 768, cv.CV_8UC3); //Here's the processing

const waldoMat = await cv.imreadAsync(`${__dirname}/bobber.png`);
const matched = originalMat.matchTemplate(waldoMat, 5);


const minMax = matched.minMaxLoc();
const { maxLoc: { x, y } } = minMax;

// Draw bounding rectangle
originalMat.drawRectangle(
new cv.Rect(x, y, waldoMat.cols, waldoMat.rows),
new cv.Vec(0, 255, 0),
1,
cv.LINE_8
);

cv.imshow('We\'ve found Waldo!', originalMat);
cv.waitKey();

};
```
I am getting such a gray image. [https://i.stack.imgur.com/7AJ4F.png](https://i.stack.imgur.com/7AJ4F.png)

How can a full-screenshot of desktop image be obtained for recognition. In this case, do not save the file but work with the buffer?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.