November 17, 2003
SSX 3 - It's You Against the Mountain
November 09, 2003
Is all that we see or seem but a dream within a dream?
Note to writers everywhere: DON'T DO THIS! The plot device where the whole unbelievable episode takes place while the main character is predictably unconscious or in a coma is soooooo tired and overused. Just an humble request to please strive for something a bit more creative.
I believe it was Edgar Allan Poe who asked, Is all that we see or seem but a dream within a dream? Two centuries ago it was still rather original. When it comes to twenty-first century entertainment, however, the answer should be a resounding "No!"
Spidering Hacks
My most recent minor claim to fame---two of the one-hundred hacks in the book are by yours truly. I received my complimentary copy on Friday. The focus of the book is on scripts which spider or scrape existing web resources in order to repurpose the data. My first hack is a simple script which will publish your IE favorites to your web site (here's an example that uses the default template). The second shows how to hack latitude and longitude data out of Mappoint and then use those results to determine the geographic distance between two places.
If you're interested in ways to grab and repurpose data from the web, there's a ton of practical stuff in here, and as with all of the O'Reilly "hacks" line of books, code for everything is provided. Happy hacking!
November 07, 2003
Is your nurse cute?
Following Thomas's surgery this week---for those of you that don't know me, the whole story about Thomas's cancer is here---his class at school made cards for him. This was the first one I saw. It's hilarious. Ritina said, "Well, you can tell the kids are getting older." Click on the thumbnail to bring up the larger image and read the card.
November 05, 2003
The Matrix Revolutions
The special effects, while no less spectacular than in the predecessors, seemed commonplace---the "wow" effect is gone. Thus, this third installment needed a great story to be a great movie. Instead, it was a good movie with an okay story. Too many inconsistencies, too many unanswered questions left unresolved. Seeing it on an IMAX screen, however, was very cool---crystal clear five-story high cutting edge special effects can make up for a lot of bad writing. 
November 01, 2003
Da Funk and Da Noise
Lots of megapixels do not necessarilly a fine digital photograph make. Digital cameras, to varying degrees, introduce noise into the images they record. When displayed or printed, this noise can show up as pixellation or grain, the tendency of individual pixels or small groups of pixels to standout among their neighbors in regions of otherwise consistent color, creating an unsmooth image that appears much lower in resolution than it really is. The problem is exacerbated if the image is digitally manipulated---sharpening, an often utilized process for digital prints, is possibly the greatest culprit---because the error pixels often become more pronounced compared to their surroundings.
About three months ago I became frustrated with the output produced by my 3.3 megapixel Canon G1 and 2400 dpi Lexmark inkjet. The prints were worse than my former setup, a 1 megapixel HP C320 and a 600 dpi Deskjet. Researching how that could possibly be led me to an understanding of the problem, dissatisfaction with several existing solutions becuase they either didn't work or were way too costly, and ultimately to the creation of a set of Photoshop actions that perform, in my clearly biased opinion, as well as most of the existing tools I was able to find.
So, for what it's worth and with the hope that other digital photographers might benefit, presented here is the Photoshop action set along with a summary of what I learned.
- A Few Examples
- The Photoshop Action Set
- A Quick Guide to Using the Actions
- The Digital Noise Problem
- Conception of the RP DNR Actions
- How the RP DNR Actions Work
- A Comparison of Noise Reduction Methods
- Advanced Use of the DNR Action Set
- Making High Quality Prints
- Recommendations
First, a few examples are probably in order to demonstrate the issue. I have a test suite of about a dozen images that I've been working with for the past few months. Below are three photos taken with my Canon G1 that demonstrate various levels of detail versus noise.
For each example I've included four images: the original tiff, the tiff after cleaning by my Photoshop DNR (Digital Noise Reduction) actions, a jpeg of the original, and a jpeg of the cleaned image. If you just want to compare the images visually, the jpeg images will more than suffice. If you actually want to print them for comparison then you might want the tiff files, but they are very large and you'll only want to download them if you have a high bandwidth connection.
I've found that the greater the focal length, the more noise I see with this camera. For this photo I was trying to achieve some depth of field---with the exception of DSLRs, digital cameras are notoriously bad at this---and so had the camera at its maximum zoom. Download the jpeg of the original, zoom to 400%, and look closely at the face areas. The noise is among the highest I've seen, and I ended up using this as the demo image for this article.
My Canon G1 really puzzles me. Although this picture and the one above were taken with the same camera, this picture is virtually noise free. There is some minor chroma noise in the blue sky, but nothing approaching the extreme pixellation so obvious in the photo above. I used one of the most extreme DNR actions on this image, and the result is an incredibly crisp picture with virtually no loss of detail. If you want a true example of the power of cleaning digital noise, print both the original and the cleaned image at 8x10 on a high resolution printer and compare the results.
Shot with my Canon G1, the upper third of this image suffers from chroma noise, especially toward the left side of the picture. None of the tools I tried could clean up the noise in this image without harming the detail of the water. If you look at the result from "DNR Advanced" you'll see that the sky has been cleaned to perfection and the detail in the water, rocks, and trees is virtually untouched. Before you get too excited, though, if you look in the action set for the "DNR Advanced" action, there isn't one. Advanced means that I didn't use one of the "prepackaged" actions (Level 0 through Level 9), but instead I manually applied the actions to gain complete control over the masking and preservation of detail, and to vary the level of noise reduction in different areas of the picture. I'm planning a future post on advanced usage that will detail exactly how I removed the noise in this image.
Here is Version 1.0 of the Photoshop action set, licensed via the MIT License.
Though this first attempt is actually quite simple in its approach to noise rediction, I believe you'll find the performance to be as good or better than most existing tools across a wide range of images. I'm still experimenting with new test images and composite action sequences, and am already working on more sophisticated approaches for what will be my next version. Version 1.0 works well enough, though, that I thought I'd go ahead and release it and write this up so that those with an interest in this sort of thing could make use of it and provide possible feedback.
If you use the actions and find them useful, let me know how they performed for you. If you have ideas for enhancements, if you improve the actions in some way, or if you have test images that seem particularly troublesome, I'd love to hear from you. Whether you give me any feedback or not, though, my hope is that you learn something from all this, and, whether you end up using my solution or something else, that you end up being able to improve the quality of your digital photographs through digital noise reduction.
A Quick Guide to Using the Actions
Download the action set above, save it to your file system, and then load it into Photshop's action palette. To use one of the actions, load an image into Photoshop, make sure that the image is stored in the background layer (this will occur by default if you open any non-layered image), and then run one of the "DNR Level X" actions. For basic use, that's all there is to it. The actions make no attempt to load or save files, they simply operate on the current background image, so it's also easy to use the actions in a batch to process multiple images in a single run.
The action set currently contains ten different actions, DNR Level 0 through DNR Level 9, that attempt greater noise reduction the higher the level. Levels 0 and 1 do a good job removing most luminance noise, but will barely touch chroma noise. Starting at Level 2 the actions begin to reduce chroma distortions as well, from modest reduction at Levels 2 and 3, to absurd levels of reduction in Levels 8 and 9.
Starting at Level 2, the actions actually come in pairs in terms of how much noise reduction they attempt to do. That is, Level 2 and Level 3 attempt the same level of reduction, Level 4 and Level 5 attempt the same higher level of noise reduction, etc. The difference is that the even numbered level of each pair attempts a greater level of detail preservation. For example, Level 4 and Level 5 perform the same noise reduction, but Level 4 takes additional steps in attempt to preserve a finer level of detail than Level 5.
For images that will be used online, I've found that using Levels 0 through 2 on the original image, before resizing, do an excellent job. For images to be printed, Levels 4 through 9, depending on the amount of detail in the picture, followed by 100% to 150% unsharp mask using a pixel radius in the 0.5 to 1.5 range, produces superb quality prints, especially if you preinterpolate the image to the printer resolution (see Making High Quality Prints below).
Finally, I realize that being able to act only on the entire background image is a limitation. I am working on a second version that uses a bit more sophistication in its approaches to noise reduction, and I also plan to generalize the actions to operate on any layer or active selection. I have personally already run into situations where I wanted the actions to operate on selections or other layers, so I know it's a limitation as I myself have encountered it. If it turns out there is sufficient interest in the existing action set, I might take the time to create a Version 1.1 release that is basically these same actions but with the ability to act on arbitrary layers or selections. So, sit tight; it's coming!
Until a few months ago I naively assumed that more megapixels meant better images. I knew camera quality was also an issue, and I spent a long time researching cameras a couple years back before I finally chose the Canon G1, Canon's top consumer-level camera at that time. I will say that I am very pleased with the images it produces, and the level of detail it records compared to my older 1 megapixel camera is striking.
When I made prints of the images, however, I was very dissatisfied. They were not horrible, but neither were they anywhere near photo quality. The images from my 1 megapixel camera actually looked better! My expectation when moving up to a 3.3 megapixel camera was that the prints should be virtually indistinguishable from film-based prints when viewed casually from any distance greater than six inches. The problem was most noticeable in large areas of what should have been smooth color; those areas sometimes did look horrible. For a long time I blamed the issue on my 600 dpi printer, until I upgraded to a 2400 dpi printer and saw no improvement whatsoever. I finaly concluded that the problem must be with the images, not the printer, and that led me into the world of digital noise, specifically, luminance noise and chroma noise.
Because of the nature of how a digital camera records images using a Charge Coupled Device (CCD), all digital cameras introduce noise into the images they record. It turns out there are pros and cons to the CCD. The pros are that they are extremely sensitive, they can record images even in very low light without the degradation normally observed using film, they can record at extremely high resolution, and they can be increased in size almost without restriction. The cons are that their extreme sensitivity can produce luminosity abberations, and because they are sensitive only to light and not to color, color abberations are introduced by the methods used to change light intensities into color.
Luminosity abberations are commonly referred to as luminance noise, and color abberations as chroma noise. Luminance noise shows up as bright flecks at the pixel level or in small groups of pixels. Imagine a photo covered with tiny particles of irregularly shaped glowing dust and you begin to get the idea. Chroma noise shows up as irregular color variation across large areas of what should be flat or consistent color, blue being the most problematic. Newer digital cameras, especially the more expensive DSLRs (drool), are introducing hardware and software to combat luminance and chroma noise, so I'm personally hopeful that within the next couple generations of digital cameras, this whole problem will just go away, or at least will become reduced to the point that post-capture noise reduction will become unnecessary.
Conception of the RP DNR Actions
When I first went looking for tools to help reduce digital noise, I found several options ranging in price from about twenty dollars to several hundred dollars. What I learned from experimenting with them is that there is not, at present, any definitive solution to this problem. Sometimes a $200 Photoshop plugin might do a marvelous job, and other times it would be shown up by a $20 command line tool. Preferring not to spend several hundred dollars investing in different tools that may or may not work when I wanted them to, I decided to try tackling the problem myself. As an amatuer photographer for 25 years (anyone else out there whose first camera was the incredible Minolta SRT 201?), a software developer for 20 years, and a Photoshop user for almost 10 years, I thought that if I could get a grasp of the problem, I'd have a decent shot at producing something useful.
After researching and studying the issues of luminance and chroma noise for a while, I had an idea that I thought had merit. Most approaches to Digital Noise Reduction (DNR) appeared to try to find the noise and remove it. An obvious and sensible approach, but distinguishing the noise from everything else turns out to be very difficult indeed. My thought was to take the opposite approach. What if I could identify the details of the picture, preserve it, and then clean everything else as if it were noise. I had no idea whether it was a sensible approach or not, but a couple of hours later when I had my first working action that would reduce noise as well as some of the expensive tools I had tried, I was convinced the idea was worth pursuing.
From that point, it took about twenty to thirty hours of work, spread over a couple of weeks, to create and refine the action set that I have now. That the technique works so well with so little effort has motivated me to see if I can really make it shine. I am presently working on more sophisticated ways to mask and preserve detail, experimenting with better ways of removing noise that don't destroy detail that the masking process missed, looking at the individual color channels from lots of different photos and different cameras to see if there might be any benefit to cleaning individual channels, and several other things. I'm doing this in my spare time so it will probably be a while before I have something I'm willing to call Version 2.0, but I promise to try to make it worth the wait.
Version 1 of the RP DNR actions work on a very simple premise: identify and preserve the image detail, then clean everything else as if it were noisy. Most other approaches I've seen attempt to identify the noise and then remove it without damaing the detail, which seems a much more difficult problem. On the other hand, if the detail can be identified and preserved, severe noise reduction can be applied to the rest of the image, resulting in a very clean image with all the detail of the original. In practice this turns out to work rather well with even a simple implemenation. For RP DNR Version 1, the noise reduction is just Photoshop's own dust and scratches filter; the reall challenge, where most of the development time was spent, was the technique to identify the image detail.
In the published action set, identifying the image detail works like this. First, we make a copy of the image to be cleaned, and then clean it rather severly. If we now take the difference of the cleaned image and the original image, we should be looking at either detail or noise. Since the noise should be dispersed an generally noncohesive, we should be able to apply a somewhat fuzzy color selection filter to the difference image to do a good job selecting the detail and ignoring the noise, and this is, in fact, precisely what it is done.
Once the detail mask is created, the various levels of noise reduction actions do different things with it. At the simplest levels, Level 0 and Level 1, we simply mask out the detail and apply a small amount of noise reduction via the dust and scratches filter. For the higher levels, detail is actually extracted from the original image into one or two additional layers, the entire underlying image is noise reduced, and then the detail layers, one of which is also noise reduced, are overlayed on the original image.
This simple approach produces some remarkable results given the small amount of actual development time I've invested so far. I was impressed enough with the results that I'm convinced it's worth continue to explore the idea. For more information on where I'm going with this, read the last paragraph of the "Conception of the RP DNR Actions" section above.
A Comparison of Noise Reduction Methods
First, if you want a look at what already exists in the way of DNR tools, the best article I've found by far is the Noise Reduction Tool Comparison by Michael Almond. This will also give you a few images for a basis of comparison between the RP DNR actions and the tools reviewed in the article.
Below I've prepared an example of each of the RP DNR actions, along with three different levels of noise reduction the Nik Multimedia's Dfine tool. I used Dfine as a basis for comparison, because of all the tools I tried, I felt it did the best job on this particular image. The sample image itself is the first image from the example images above. I chose it because it contains areas of both high and moderate detail, areas of solid color, a high level of luminance noise, and a moderate level of chroma noise.
Since it wasn't practical to show the entire image for each action, I chose a 200x150 pixel region of the image that contains some high detail (around the eye), some low to moderate detail, some areas of smooth color, and some of the background. Clicking on one the images will open a 400% maginified version of the image in a separate window. You can open multiple magnified images simultaneously for comparison.
Original Image: The crop from the original image.
Photoshop Despeckle: Good reduction of luminance noise, no reduction of chroma noise, noticeable blurring and loss of detail.
Photoshop Dust & Scratches at 1 Pixel Radius: Good reduction of luminance noise, trivial reduction of chroma noise, noticeable blurring and loss of detail.
Photoshop Dust & Scratches at 2 Pixel Radius: Good reduction of luminance noise, minor reduction of chroma noise, excessive blurring and loss of detail.
Photoshop Median at 1 Pixel Radius: Good reduction of luminance noise, trivial reduction of chroma noise, noticeable blurring and loss of detail.
Photoshop Median at 2 Pixel Radius: Good reduction of luminance noise, minor reduction of chroma noise, excessive blurring and loss of detail.
Dfine Low Reduction Settings: Good reduction of chroma noise, poor reduction of luminance noise, little to no detail loss.
Dfine Medium Reduction Settings: Very good reduction chroma noise, poor reduction of luminance noise, minor blurring and loss of detail.
Dfine Maximum Reduction Settings: Excellent reduction of chroma noise, good reduction of luminance noise, very noticeable blurring and loss of detail, plastic feel to image.
RP DNR Level 0: Excellent reduction of luminance noise, trivial reduction of chroma noise, no noticeable blurring or loss of detail.
RP DNR Level 1: Excellent reduction of luminance noise, minor reduction of chroma noise, no noticeable blurring or loss of detail.
RP DNR Level 2: Excellent reduction of luminance noise, minor to moderate reduction of chroma noise, no noticeable blurring, trivial detail loss.
RP DNR Level 3: Excellent reduction of luminance noise, good reduction of chroma noise, minor blurring and detail loss.
RP DNR Level 4: Excellent reduction of luminance noise, very good reduction of chroma noise, minor blurring, trivial detail loss.
RP DNR Level 5: Excellent reduction of luminance noise, very good reduction of chroma noise, minor blurring, minor detail loss.
RP DNR Level 6: Excellent reduction of luminance noise, excellent reduction of chroma noise, noticeable blurring in low-to-moderate detail areas, very good preservation of fine detail.
RP DNR Level 7: Excellent reduction of luminance noise, excellent reduction of chroma noise, noticeable blurring in low-to-moderate detail areas, good preservation of fine detail.
RP DNR Level 8: Extreme reduction with high preservation of fine detail. Intended for very low detail images only. Detailed images become plasticy looking.
RP DNR Level 9: Extreme reduction with moderate preservation of fine detail. Intended for very low detail images only. Detailed images become plasticy looking.Advanced Use of the DNR Action Set
When Version 2 of the action set is ready I'm planning a detailed article on making advanced use of the actions, but even for the comparatively simple Version 1, sophisticated noise reduction is possible beyond the the "out-of-the-box" actions by controlling the "detail mask" for an image.
If you read the discussion above about how the RP DNR actions work, then you know that the premise is that if the detail areas of an image can be identified and preserved, we can then apply rather extreme noise reduction to the non-detail areas (e.g. flat colors, smooth transitions across large areas, etc.), and still achieve an excellent final image. The default method of identifying detail is the "Detail Mask" action. Each of the RP DNR actions calls this action as one of its initial steps. This action works by essentially comparing the original image to an extreme noise-reduced version of the image and identifying areas where there are differences between the two. In general this works amazingly well, but there are situations where it does only a modest job, or where manual selection of the detail gives better results.
Thus, there are two fundamental ways to gain more control over the noise-reduction process. The first is to manully intervene at certain steps of the Detail Mask action, specifically the "Dust & Scratches" action at the top, and the "Color Range" action. Keeping in mind that the goal is to select the areas of high detail, you'll have to experiment a bit to determine how these affect the creation of the detail mask, but with a little practice, you can get quite good at using these two steps to create excellent detail masks for a wide variety of images.
The second method for better control over the noise-reduction process is to manually create the detail mask. For tricky images this is by far the best approach, and can be mastered with only a bit of practice. Once you create a good detail mask, save it with the name "rpdnr-detail-mask" and then run any of the RP DNR actions with its first step disabled. Manually constructing a detail mask and then using Level 8 is how I noise-reduced the final sample picture above (the ocean picture). Some extremely good results are possible using this technique.
If you're interested not only in the best digital images but also the best prints from your printer, then you need to learn about interpolating to print resolution and color management, and ideally you should find a tool that will take care of these things for you. Personally I've settled on an inexpensive but excellent little utility called Qimage. A discussion of interpolation and color management is beyond my intent for this page, but you can read an excellent presentation of the issues at the Qimage web site.
Based on what I've learned over the past few months, I'll close with a quick list of my personal recommendations for tackling digital noise. Hope the information here has proven useful to you!
- First, if you find your images and prints acceptable, don't bother with digital noise issues. Many of my prints sucked---really---and so I took the time to find out what was going on and deal with it, but I have friends who's prints look great without any DNR processing at all. After looking at this problem for a while now, I'm convinced that nearly any image can probably be improved to some degree by noise reduction, but sometimes the minor improvement may not be worth the effort. In my case, however, I've found it to be well worth the effort for nearly all my images.
- Apply the minimal amount of noise reduction that gives good results. Too much noise reduction results in fake or plasticy looking images. The airbrushed look might be cool for special effects, but if you're going for realistic images, try to keep the noise reduction as minimal as possible. Personally I usually use Levels 0 through 3 from the action set presented here, and the results have been excellent.
- If you'll be printing images, don't overlook interpolation and color issues. See the section above on obtaining high quality prints if you haven't read it yet.
- Finally, I just hope you were able to learn something and perhaps find a useful tool. If you already use something else, or you prefer another tool to my action set here, by all means use it. I don't claim that the actions here are any more or less perfect than the many other things I've evaluated, but the idea I've implemented definitely seems to have merit so I'll personally be pursuing it.