Mastering images with HTML5 canvas – part 1

There can be no doubts that the canvas technology was one of the most notable achievements of HTML5. Even though canvas still seems to be very limited and slow in comparison with the native OS graphics, it's hard to overrate the conceptual importance on this invention. Together with AJAX, CSS3 and other newest techniques, canvas turns a modern browser to the real full-featured dynamic application platform with no prostheses and crutches through various add-ons and plug-ins.

It goes without saying that canvas couldn't go without a possibility of image processing. While in the old classic WEB an image was considered a static thing, canvas lets us access the image row data and manipulate it on the level of pixels, so we can make changes in an image on the fly. Doing it on the client side we free the server from unnecessary load, save storage space and significantly reduce the amount of downloaded resources. Moreover, canvas can venture the dynamic image effects which we could haven't even imagined if every change would require a client-server connection trip.

In this part of the article we'll understand the basics of the image processing with canvas, implement some simple manipulations and see how it can be used in the practical WEB design. In the upcoming parts I'll cover the more complicated cases.

The basics of the image manipulations with canvas

In fact, every canvas element is an image by itself. This image contains N = W * H pixels where W is the image's width and H is the image's height. Each pixel is a set of four RGBA bytes where the first three bytes sequentially contain brightness values of red, green and blue color components and the last byte contains the alpha component which defines the pixel's opacity as shown on the below picture.

Therefore, the total size of image row data S = 4 * W * H bytes.

Initially all the data bytes equal zero, so the image is absolutely black and transparent. When something is drawn into canvas it changes the values of the affected pixels. Every next drawing goes on top of the previous result and changes the pixels again accordantly to used colors and transparency. For example, if we draw a red rectangle into an empty canvas and then add an overlapping green rectangle with transparency 50% we'll receive the following result:

To access the pixels data we should use the getImageData method which copies the pixels values for the specified rectangle on a canvas into its data property. Then we can do everything we want to do with these pixels, and finally put the result back onto canvas using the method putImageData. The following pseudo code illustrates this process:

To work with a picture or a video frame we can draw it into canvas using the method drawImage as following:

where the first parameter is an image or video DOM object and the next two parameters are the x and y coordinates where to place the image on the canvas. Naturally, that image should be completely loaded before this operation.

CORS limitation and workaround

When talking about the use of images in canvas, we must not forget the security limitations. By the CORS protocol, the access to image data is allowed only in the same domain scope. Specifically, if the image is downloaded from the domain our page belongs to then we have no problem, but if it's downloaded from another domain, the method getImageData fails and throws the security exception. I.e., we still can draw the images from the external domains into canvas, but we cannot access their pixels and change them. For example, we'll not be able to play with the user's avatar picture coming from his social network account or Gravatar.

Usually it's not a problem to place on the server the images you want to manipulate on the client side, but it can be a problem if you use a site which isn't under your control. For example, CodePen doesn't allow you to upload your custom resources unless you have a paid account, and so you cannot place an image on the same domain. However, if you still aren't ready to pay, you can trick it by a simple and legal workaround. With a help of a free online tool like DataURL Maker in a couple of seconds you can convert the image to the embeddable Base64 encoded URL and paste it as an HTML image object's src property value. Such code doesn't look very beautiful, but it works.

Turn an image to grayscale

Now, that we have the necessary background, we can start the actual image processing. To begin with, let's convert a regular color image to grayscale. There are a number of color-to-grayscale conversion methods; the most accurate of them is the luminosity method that doesn't simply averages the red, green and blue values, but forms a weighted average to account for human perception. The luma component (or Y in the YUV color space) can be computed from the RGB using the following quick formula:

So, to convert the image we should pass the entire image pixel by pixel, calculate the luma value and set the result to red, green and blue color components.

Note that we increment the index by 4 on every cycle to jump to the next pixel. When we're done, our canvas contains the grayscale reflection of the source image.

Photo filters

Well, just a convertion of an image to grayscale is really simple, but not so sexy. To get something really useful for the WEB design we should suggest further evolvement of the idea. For instance, let's think about the colored photo filters effect.

Although colored photo filters can be used with any image, the most interesting effects are achieved by applying them just on the grayscale images. For example, sepia filter on grayscale creates a nice simulation of the old black and white photo from the grandmother's album.

The colored filter is defined using two parameters. The first one is the RGB color combination. There are twenty standard color combinations like various warming and cooling filters, orange, sepia, underwater, etc. The second parameter is a filter density. The denser filter we use, the more strongly the result image will be colored.

To realize the filtering, the same way as for grayscale we should pass the image pixel by pixel and compute the luma component value. Then using the filter RGB values combination and the density we should compute the intensity for each color plan as following:

where I_c is the color intensity (I_r is the red intensity, I_g is the green intensity, I_b is the blue intensity), F_c is the filter color value (F_r is the red value, F_g is the green value, F_b is the blue value) and D is the density value in the range from 0 to 100 percents.

Finally we set the pixel color values as

where C is the color value (i.e. R, G and B), I_c is the color intensity and Y is the pixel luma value.

Now we can write our filtering function:

The only thing left to do is an initialization of the standard filter set:

and we can see the colored photo filtering in action:

Use of colored filters in the practical WEB

To complete this part I'd like to show how the colored photo filters can be used in two common cases of practical WEB development. The first one is an image gallery. Imagine we have a set of images on the page. Initially we want to show them in an old album black and white style, while the hovered image should become colored. To meet this goal we'll use the filterImage function wrote before with the sepia filter and 50 percent density. Upon image loading we'll prepare a related canvas containing the filtered image and place it on top of the source image element. Initial canvas element's opacity is set to 1 so it fully covers the image. On hover we'll change the opacity of canvas to 0 and see the source image. To make this change smooth we'll use the half second CSS transition.

Hover on the image to see it in action.

The second common case is the necessity to show some image-related information when the mouse hovers it. Here we need the image to be filtered on hover, so we'll exchange the two opacity values in the 'img-cover' class. The Initial opacity value will be set to 0 and the ':hover' value to 1. Of course we should also prepare the text element that will be shown on hover:

For change, let's use the underwater filter with 75 percent density:

Hover the image to see it in action.

Conclusion

In this part we understood the basics of image manipulation with HTML5 canvas. Using this knowledge we have developed a simple algorithm for colored photo filters and saw how it can be utilized in the WEB pages.

In the next upcoming part we'll deal with image clipping and combine it with the variable transparency and brightness to get some cool visual effects.