Warm tip: This article is reproduced from stackoverflow.com, please click
image-processing javascript node.js puppeteer

How to traverse over pixel colours of a Puppeteer created screenshot image and draw rectangle and sa

发布于 2020-04-08 15:42:57

I am new to JavaScript/Node.js and image processing and would like to take the output of puppeteer screenshot (preferably in memory) then be able to traverse colors of each pixel in the image, then stroke a rectangle border around certain coordinated of the image (x, y, width, height), then write save the image with the rectangle highlights to file using Node.js.

What I get from puppeteer is

returns: <Promise<string|Buffer>> Promise which resolves to buffer or a base64 string (depending on the value of encoding) with captured screenshot.

I read that I can create a canvas object using puppeteer to do this but I am not sure about this solution especially to retrieve pixel colors. I don't know if I should be using a package like PureImage for this or some other package.

Questioner
Ibrahim
Viewed
92
Ibrahim 2020-02-06 01:12

Here is my code that....

  1. Handle a base64 screenshot taken by Puppeteer.
  2. Create an image element.
  3. Wait for the image onload event.
  4. set the src attribute of an image to the screenshot.
  5. Create canvas not attached to DOM and write image to it.
  6. Stroke two rectangles on the canvas.
  7. return the canvas data.

async function (rectangle1, rectangle2, screenshot) {
        let screenshotHighlighted = await driver.page.evaluate(async function (rectangle1, rectangle2, screenshot) {
            let image = new Image();
            let imgLoadPromise = async function(){ return new Promise((resolve, reject) => {
                image.onload = () => {console.log('loaded'); return resolve};
                image.onerror = reject;
            });}
            image.src = 'data:image/png;base64,' + screenshot;
            await imgLoadPromise;
            let canvas = document.createElement("CANVAS");
            let context = canvas.getContext("2d");
            canvas.width = image.naturalWidth;
            canvas.height = image.naturalHeight;
            context.drawImage(image, 0, 0);
            //solid black and dashed red
            context.strokeStyle = "black";
            context.strokeRect(rectangle1.minX, rectangle1.minY, rectangle1.width, rectangle1.height);
            context.setLineDash([5, 10]);
            context.strokeStyle = "red";
            context.strokeRect(rectangle1.minX, rectangle1.minY, rectangle1.width, rectangle1.height);
            context.setLineDash([]);
            //solid black and dashed yellow
            context.strokeStyle = "black";
            context.strokeRect(rectangle2.minX, rectangle2.minY, rectangle2.width, rectangle2.height);
            context.setLineDash([5, 10]);
            context.strokeStyle = "yellow";
            context.strokeRect(rectangle2.minX, rectangle2.minY, rectangle2.width, rectangle2.height);
            return canvas.toDataURL();
        }, rectangle1, rectangle2, screenshot);
        return screenshotHighlighted;
    }

To save the file in Node.JS

fs.writeFile(path.join(directory,imageFileName),highlightedScreenshot.split(',')[1], 'base64', 
  function(err) {
        if(err) {
                    console.log('ERROR IN SAVING IMAGE');
                    console.log(err);
                } 
            });

Traversing pixels of an HTML canvas can be found in the following answer

Get Pixels from HTML Canvas