Tutorial: Photorealistic Painting of a Cat

Originally Published December 2010

If you’ve ever painted with traditional water-based media, such as watercolor or acrylics, you may have painted with a liquid rubber mask, such as Maskoid, to protect areas from paint. An example of this is a watercolor of a cat peeking out from some dense foliage. You might cover the cat with liquid masking to protect it. When the mask is dry, you can paint the foliage without worrying about getting any paint on the cat. Once the foliage is dry, simply peel the mask off the cat. It remains untouched, and ready to paint.

In digital art, of course, there are different methods of masking out areas of a painting. We’ve seen layer masks in use quite a bit here on Digital Image Magazine. But today I’m going to show another method that you may find useful. It’s not necessarily better than using a layer mask, but it may suit your workflow better, depending on the painting you’re working on. Here’s how it works. Let’s use the example from above: a cat peeking out from dense foliage. With this method, we’ll use the painted foliage to create its own mask, allowing us to reselect areas with ease during the later stages of the painting.

Some readers have asked me to show how I create a painting, so this tutorial is in response to that request. So far, I have finished the foliage, and the cat remains to be painted (see above). So that’s where we’ll end this first part, and begin the cat next week. By the way, this tutorial applies just as well to Painter as Photoshop. Ready? Let’s get started!

The photo (by Sande Hamilton) used for this painting can be downloaded for free from Stock.Xchg (link). Once you’ve downloaded the photo, you can head over to this recent post which shows how to create nice line drawing in about 30 seconds, using Photoshop’s Minimum filter. In a nutshell, here’s how you do it:

1. In the Layers palette, drag the Background image onto the New Layer icon. Desaturate this copy (preferably with the Black and White image adjustment tool).

2. Copy this desaturated copy. Then, go Image>Adjust>Invert. Change the layer blending mode to Color Dodge. The image appears to go white, or nearly so.

3. Use the Minimum filter (Filter>Other>Minimum), with a small numeric setting. If the line drawing is too thin, increase the setting a bit.

4. Flatten the image and save as “Cat Drawing.psd.”

So, at this point you’ve got two files: the original photo, and the line drawing file. This drawing file is really the painting file, because that’s where you’ll be adding the layers for the different elements of the painting. Right now, it’s got one layer, called Background. Double-click on the name, and change it to Drawing. Change the blend mode of this layer to Multiply. This layer will remain on top of all the layers we’ll be adding. Because it’s in Multiply mode, the drawing will remain visible on top of the other layers.

Insert a new layer, by clicking on the New Layer icon, or go Layer>New Layer. Click, hold, and drag this layer beneath the Drawing layer. Turn off the eyeball on the Drawing layer. Now you can see that this new layer has a tiny checkerboard pattern. Photoshop displays this pattern wherever there is no image data–in other words, the layer is completely empty, and transparent. It’s like a pane of clean glass without a smudge on it. We’re going to paint on this layer, and then use these pixels to create a mask. Go ahead and make the Drawing layer visible again by clicking on its eyeball. Choose a soft paintbrush, pick a good green (we’ll modify it shortly), and make sure that flow and opacity are at 100%. Also, make sure Layer 1 is active. Now choose a blade (I think these are long iris leaves) to paint, and use the line drawing to guide you. Paint right up to the lines. Use the eraser if you go over.

By using the paintbrush at 100% opacity, we have created an area on Layer 1 that is completely non-transparent. If you click on the transparency lock, you will only be able to paint on the existing pixels on Layer 1. This is what I meant before: the paint has created its own mask. Is that cool or what?

To see how this works, try painting on Layer 1, both on the blade of green and on the transparent area. See? You can only paint on the existing pixels. Which is just fine by us, because we want to develop this blade now, and we won’t have to worry about staying inside the lines! Lower the brush’s opacity and flow both to about 40. With the original photo open next to your painting, begin trying to match the photo as closely as possible. Use the eyedropper to sample colors if you like (that’s what I do). Vary the opacity and the softness/hardness of the brush to paint the dark “veins” in the blade. Here’s what mine looks like when done.

Once you’re done painting this blade, we’ll want to save the selection. Go Select>Load Selection, and a dialog box comes up, already pre-filled with “Layer 1 Transparency” in the Channel field. Perfect. Click OK.

The blade is surrounded by dancing ants. Or marching ants. (Why ants?) Now we will save the selection. Go Select>Save Selection, and type in “Layer 1″ or whatever you’d like. When you click OK, a new channel is created. Go to the Channels palette to see it there. I like to keep the channel named after the layer it goes with, for simplicity sake. At this point, this layer is done, so to protect it from accidental painting, let’s lock it. Click on the black lock, three over from the transparency lock.

From here on it’s simply a matter of add a new layer, paint a blade (or area of a blade) at 100% opacity, lock the transparency, refine the blade, then save the selection and lock the layer. I ended up with 19 layers. You could have more or less, depending on how much you want to rely on separate layers for elements. Elements which are closer to the viewer will go towards the top of the layer stack, so that they overlap the elements “in back of them.” You’ll notice that two of the blades are out of focus. You can paint them that way too: just paint the blade with a hardish edge, and then apply a Gaussian blur, setting around 30. This creates areas of semi-transparency, which work just fine with the transparency lock.

I hope you’ll give it a try, and let me know how it goes for you. If you run into trouble, just leave a comment, or drop me an email (bobnolin at digitalimagemagazine.com). It took me about 4 hours to paint all the foliage and the dark background areas. So give it some time and be patient. I hope you enjoy this exercise!

Tutorial: Photorealistic Portrait of a Cat, Part 2