Automattic / Automattic/node-canvas

getImageData returns Changed Data

Open
#1,651 1 comment 0 reactions 0 assignees View on GitHub
Help wanted
Dominant language
JavaScript
Stars
10.7k
Forks
1.2k
Avg merge
4d 8h
Merged PRs (30d)
1

Description

## Issue or Feature
I'm trying to recreate the pixel data of an **JPEG** image from base64 or URL and compare it to the pixel values generated in Python's [Pillow](https://python-pillow.org/) library, which is common regarded as a solid benchmark for Image Processing in Python. I noticed that an image read by node-canvas differs significantly from the same image in Python. Here are the scripts I've used to visualize this. I thought to compare node-canvas and python to jpeg-js as well, and found that all three of them differ.

_An interesting point to note, while JPEG differs, the pixel data for PNG images in Python and node-canvas match._

## Steps to Reproduce
Script used to generate a random image and then read and convert to pixel data (generates a comma-separated pixel file).
```python
import numpy as np
from PIL import Image

arr = np.zeros((32,32,3), dtype=np.uint8)

for i in range(32):
for j in range(32):
arr[i,j] = np.random.randint(255, size=(1,3))

im = Image.fromarray(arr)
im.save('test.jpg')

img_8u = np.array(Image.open('test.jpg').getdata(), dtype=np.uint8)
with open('img_py.txt', 'w') as f:
f.write(','.join(map(str, img_8u.flatten().tolist())))
```

Script used to generate pixel data in Nodejs (generates a comma-separated pixel file):
```js
const Canvas = require("canvas")
const fs = require("fs")
const jpeg = require("jpeg-js")

function drawCanvas(url, shape) {
var cvs = Canvas.createCanvas(shape[0], shape[1]);
var context = cvs.getContext('2d');
var img = new Canvas.Image;
img.onload = function() {
context.drawImage(img, 0, 0);
};
img.src = url;
return context;
}

function preprocessAndStore(imageData, fileName) {
var rgbFP32 = new Float32Array(cleanAndStripAlpha(imageData))
fs.writeFile(fileName, rgbFP32.join(','), (err) => {
if (err) throw err;
})
}

function cleanAndStripAlpha(imageData) {
const width = imageData.width;
const height = imageData.height;
const npixels = width * height;

const rgbaU8 = imageData.data;

// Drop alpha channel and retain rgb
const rgbU8 = new Uint8Array(npixels * 3);
for (let i = 0; i < npixels; ++i) {
rgbU8[i * 3] = rgbaU8[i * 4];
rgbU8[i * 3 + 1] = rgbaU8[i * 4 + 1];
rgbU8[i * 3 + 2] = rgbaU8[i * 4 + 2];
}
return rgbU8;
}

// Node-Canvas
var width = 32
var height = 32
var imgData = drawCanvas("test.jpg", [width, height])
preprocessAndStore(imgData.getImageData(0, 0, width, height), 'image_canv.txt')

// JPEG JS
// var jpegdata = fs.readFileSync('test.jpg')
// var rawImgData = jpeg.decode(jpegdata)
// preprocessAndStore(rawImgData, 'image_jpegjs.txt')
```

The two text files should be comparable with any diff checker. If you wish to recreate all my experiments, you will find that monochrome images or images with a uniform color will be regenerated correctly but the moment variation is introduced (an image of a dog for example), the colors returned are different, even though the image may look similar.

What might be causing this and how can this be resolved?

## Your Environment
* Version of node-canvas (output of `npm list canvas` or `yarn list canvas`): 2.6.1
* Environment (e.g. node 4.2.0 on Mac OS X 10.8): node v12.16.1

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.