Mastering images with HTML5 canvas – part 1

Canvas image basics, grayscale and photo filters

Published 2/9/2015

There can be no doubts that the canvas technology was one of the most notable achievements of HTML5. Even though canvas still seems to be very limited and slow in comparison with the native OS graphics, it's hard to overrate the conceptual importance on this invention. Together with AJAX, CSS3 and other newest techniques, canvas turns a modern browser to the real full-featured dynamic application platform with no prostheses and crutches through various add-ons and plug-ins.

It goes without saying that canvas couldn't go without a possibility of image processing. While in the old classic WEB an image was considered a static thing, canvas lets us access the image row data and manipulate it on the level of pixels, so we can make changes in an image on the fly. Doing it on the client side we free the server from unnecessary load, save storage space and significantly reduce the amount of downloaded resources. Moreover, canvas can venture the dynamic image effects which we could haven't even imagined if every change would require a client-server connection trip.

In this part of the article we'll understand the basics of the image processing with canvas, implement some simple manipulations and see how it can be used in the practical WEB design. In the upcoming parts I'll cover the more complicated cases.

The basics of the image manipulations with canvas

In fact, every canvas element is an image by itself. This image contains N = W * H pixels where W is the image's width and H is the image's height. Each pixel is a set of four RGBA bytes where the first three bytes sequentially contain brightness values of red, green and blue color components and the last byte contains the alpha component which defines the pixel's opacity as shown on the below picture.

RGBA pixel model

Therefore, the total size of image row data S = 4 * W * H bytes.

Initially all the data bytes equal zero, so the image is absolutely black and transparent. When something is drawn into canvas it changes the values of the affected pixels. Every next drawing goes on top of the previous result and changes the pixels again accordantly to used colors and transparency. For example, if we draw a red rectangle into an empty canvas and then add an overlapping green rectangle with transparency 50% we'll receive the following result:

Align with CSS Caricature

To access the pixels data we should use the getImageData method which copies the pixels values for the specified rectangle on a canvas into its data property. Then we can do everything we want to do with these pixels, and finally put the result back onto canvas using the method putImageData. The following pseudo code illustrates this process:

get canvas context
var ctx = canvas.getContext("2d");
 
get image data object
var imageData = ctx.getImageData(0, 0, width, height);
 
get the pixels
var data = imageData.data;
for (var i = 0; i < data.length; i += 4) {
  • make some processing
}
 
put the pixels back onto canvas
ctx.putImageData(imageData, 0, 0);

To work with a picture or a video frame we can draw it into canvas using the method drawImage as following:

ctx.drawImage(img, 0, 0);

where the first parameter is an image or video DOM object and the next two parameters are the x and y coordinates where to place the image on the canvas. Naturally, that image should be completely loaded before this operation.

CORS limitation and workaround

When talking about the use of images in canvas, we must not forget the security limitations. By the CORS protocol, the access to image data is allowed only in the same domain scope. Specifically, if the image is downloaded from the domain our page belongs to then we have no problem, but if it's downloaded from another domain, the method getImageData fails and throws the security exception. I.e., we still can draw the images from the external domains into canvas, but we cannot access their pixels and change them. For example, we'll not be able to play with the user's avatar picture coming from his social network account or Gravatar.

Usually it's not a problem to place on the server the images you want to manipulate on the client side, but it can be a problem if you use a site which isn't under your control. For example, CodePen doesn't allow you to upload your custom resources unless you have a paid account, and so you cannot place an image on the same domain. However, if you still aren't ready to pay, you can trick it by a simple and legal workaround. With a help of a free online tool like DataURL Maker in a couple of seconds you can convert the image to the embeddable Base64 encoded URL and paste it as an HTML image object's src property value. Such code doesn't look very beautiful, but it works.

Turn an image to grayscale

Now, that we have the necessary background, we can start the actual image processing. To begin with, let's convert a regular color image to grayscale. There are a number of color-to-grayscale conversion methods; the most accurate of them is the luminosity method that doesn't simply averages the red, green and blue values, but forms a weighted average to account for human perception. The luma component (or Y in the YUV color space) can be computed from the RGB using the following quick formula:

Pixel luma value computation

So, to convert the image we should pass the entire image pixel by pixel, calculate the luma value and set the result to red, green and blue color components.

for (var i = 0; i < data.length; i += 4) {
  • calculate the luma value
    var luma = 0.299 * data[i] + 0.587 * data[i+1] + 0.114 * data[i+2];
     
    set it to RGB
    data[i] = Math.round(luma);
    data[i+1] = Math.round(luma);
    data[i+2] = Math.round(luma);
}

Note that we increment the index by 4 on every cycle to jump to the next pixel. When we're done, our canvas contains the grayscale reflection of the source image.

Photo filters

Well, just a convertion of an image to grayscale is really simple, but not so sexy. To get something really useful for the WEB design we should suggest further evolvement of the idea. For instance, let's think about the colored photo filters effect.

Although colored photo filters can be used with any image, the most interesting effects are achieved by applying them just on the grayscale images. For example, sepia filter on grayscale creates a nice simulation of the old black and white photo from the grandmother's album.

The colored filter is defined using two parameters. The first one is the RGB color combination. There are twenty standard color combinations like various warming and cooling filters, orange, sepia, underwater, etc. The second parameter is a filter density. The denser filter we use, the more strongly the result image will be colored.

To realize the filtering, the same way as for grayscale we should pass the image pixel by pixel and compute the luma component value. Then using the filter RGB values combination and the density we should compute the intensity for each color plan as following:


Color intensity computation

where Ic is the color intensity (Ir is the red intensity, Ig is the green intensity, Ib is the blue intensity), Fc is the filter color value (Fr is the red value, Fg is the green value, Fb is the blue value) and D is the density value in the range from 0 to 100 percents.

Finally we set the pixel color values as

Filtered color value computation

where C is the color value (i.e. R, G and B), Ic is the color intensity and Y is the pixel luma value.

Now we can write our filtering function:

function filterImage(img, filter, density) {
  •  
    compute color intensity for the entire filter and density
    var rIntensity = (filter.r * density + 255 * (100 - density)) / 25500;
    var gIntensity = (filter.g * density + 255 * (100 - density)) / 25500;
    var bIntensity = (filter.b * density + 255 * (100 - density)) / 25500;
     
    create canvas and load image
    var canvas = document.createElement("canvas");
    canvas.width = img.width;
    canvas.height = img.height;
    var ctx = canvas.getContext("2d");
    ctx.drawImage(img, 0, 0);
     
    get image data and process the pixels
    var imageData = ctx.getImageData(0, 0, img.width, img.height);
    var data = imageData.data;
    for (var i = 0; i < data.length; i += 4) {
    • var luma = 0.299 * data[i] + 0.587 * data[i+1] + 0.114 * data[i+2];
      data[i] = Math.round(rIntensity * luma);
      data[i+1] = Math.round(gIntensity * luma);
      data[i+2] = Math.round(bIntensity * luma);
    }
     
    put the image data back into canvas
    ctx.putImageData(imageData, 0, 0);
     
    return canvas;
}

The only thing left to do is an initialization of the standard filter set:

function flterColors(r, g, b) {
  • this.r = r;
    this.g = g;
    this.b = b;
}
 
var filters = new Array();
filters.push(new flterColors(0xFF, 0xFF, 0xFF)); grayscale
filters.push(new flterColors(0xEC, 0x8A, 0x00)); Warming (85)
filters.push(new flterColors(0xFA, 0x96, 0x00)); Warming (LBA)
filters.push(new flterColors(0xEB, 0xB1, 0x13)); Warming (81)
filters.push(new flterColors(0x00, 0x6D, 0xFF)); Coolling (80)
filters.push(new flterColors(0x00, 0x5D, 0xFF)); Cooling (LBB)
filters.push(new flterColors(0x00, 0xB5, 0xFF)); Cooling (82)
filters.push(new flterColors(0xEA, 0x1A, 0x1A)); Red
filters.push(new flterColors(0xF3, 0x84, 0x17)); Orange
filters.push(new flterColors(0xF9, 0xE3, 0x1C)); Yellow
filters.push(new flterColors(0x19, 0xC9, 0x19)); Green
filters.push(new flterColors(0x1D, 0xCB, 0xEA)); Cyan
filters.push(new flterColors(0x1D, 0x35, 0xEA)); Blue
filters.push(new flterColors(0x9B, 0x1D, 0xEA)); Violet
filters.push(new flterColors(0xE3, 0x18, 0xE3)); Magenta
filters.push(new flterColors(0xAC, 0x7A, 0x33)); Sepia
filters.push(new flterColors(0xFF, 0x00, 0x00)); Deep Red
filters.push(new flterColors(0x00, 0x22, 0xCD)); Deep Blue
filters.push(new flterColors(0x00, 0x8C, 0x00)); Deep Emerald
filters.push(new flterColors(0xFF, 0xD5, 0x00)); Deep Yellow
filters.push(new flterColors(0x00, 0xC1, 0xB1)); Underwater

and we can see the colored photo filtering in action:

Use of colored filters in the practical WEB

To complete this part I'd like to show how the colored photo filters can be used in two common cases of practical WEB development. The first one is an image gallery. Imagine we have a set of images on the page. Initially we want to show them in an old album black and white style, while the hovered image should become colored. To meet this goal we'll use the filterImage function wrote before with the sepia filter and 50 percent density. Upon image loading we'll prepare a related canvas containing the filtered image and place it on top of the source image element. Initial canvas element's opacity is set to 1 so it fully covers the image. On hover we'll change the opacity of canvas to 0 and see the source image. To make this change smooth we'll use the half second CSS transition.

HTML
a href#img idimg1a
 
CSS
.img-cover {
  • positionabsolute;
  • opacity1.0;
  • -webkit-transitionall .5s ease-in-out;
  • transitionall .5s ease-in-out
}
 
.img-cover:hover {
  • opacity0;
}
 
JavaScript
$('#img1').load(function() {
  • create canvas with the filtered image
    var filteredCanvas = filterImage($(this).get(0), filters[15], 50);
     
    set canvas element's parameters and insert it into HTML
    $(filteredCanvas).attr('class', 'img-cover');
    $(filteredCanvas).css('left', $('#img1').position().left);
    $(filteredCanvas).css('top', $('#img1').position().top);
    $(filteredCanvas).insertAfter($(this));
});

Hover on the image to see it in action.

The second common case is the necessity to show some image-related information when the mouse hovers it. Here we need the image to be filtered on hover, so we'll exchange the two opacity values in the 'img-cover' class. The Initial opacity value will be set to 0 and the ':hover' value to 1. Of course we should also prepare the text element that will be shown on hover:

HTML
a href#
  • img idimg2
    div idimgTitle2divdivVINCENT VAN GOGHbrAUTOPORTRAITdiv
a
 
CSS
.img-cover {
  • positionabsolute;
  • opacity0;
  • -webkit-transitionall .5s ease-in-out;
  • transitionall .5s ease-in-out
}
 
.img-cover:hover {
  • opacity1.0;
}
 
#imgTitle2 {
  • displayinline-block;
  • positionabsolute;
  • font-family'Oswald', sans-serif;
  • font-size20px;
  • font-weightbold;
  • text-shadow2px 2px #606060;
  • text-aligncenter;
  • colorwhite;
  • opacity0;
  • -webkit-transitionall .25s ease-in-out;
  • transitionall .25s ease-in-out
}
 
#imgTitle2:hover {
  • opacity1.0;
}
 
#imgTitle2 div {
  • displayblock;
  • height0;
  • displayblock;
  • -webkit-transitionall .25s ease-in-out;
  • transitionall .25s ease-in-out
}
 
#imgTitle2:hover div {
  • height40%
}

For change, let's use the underwater filter with 75 percent density:

JavaScript
$('#img2').load(function() {
  • create canvas with the filtered image
    var filteredCanvas = filterImage($(this).get(0), filters[20], 75);
     
    set canvas element's parameters and insert it into HTML
    $(filteredCanvas).attr('class', 'img-cover');
    $(filteredCanvas).css('left', $('#img2').position().left);
    $(filteredCanvas).css('top', $('#img2').position().top);
    $(filteredCanvas).insertAfter($(this));
     
    set the text element parameters
    $("#imgTitle2").css("left", $("#img2").position().left);
    $("#imgTitle2").css("top", $("#img2").position().top);
    $("#imgTitle2").css("width", $("#img2").width);
    $("#imgTitle2").css("height", $("#img2").height);
}

Hover the image to see it in action.

Conclusion

In this part we understood the basics of image manipulation with HTML5 canvas. Using this knowledge we have developed a simple algorithm for colored photo filters and saw how it can be utilized in the WEB pages.

In the next upcoming part we'll deal with image clipping and combine it with the variable transparency and brightness to get some cool visual effects.

Enjoyed this Article?
Recommend it to friends and colleagues!

3 Reader Comments