There's a scene in the old movie Marathon Man where Dustin Hoffman wakes up strapped in a Dentist's chair, mouth forced open, while the evil dentist drills into the nerves of his teeth, repeating the same question over-and-over: "Is it safe"? Tortured past endurance, he would answer if he could, but the question has no meaning to him. Is what safe? He has no idea.
I have nightmares where I'm that guy, with someone just out of my field of view repeating over-and-over: "Is it sharp?" I'd answer if I could, but the question has no meaning. Is what sharp? The center at f/8? The edges at f/1.4? Sharp as in high acutance? Sharp as in great microcontrast? Is it still sharp if it has massive distortion but resolves well?
People who obsess about "is it sharp?" believe that things like microcontrast and how a lens 'draws' or 'renders' are subjective and can't really be assessed. I work with lenses all day and like to think I'm a pretty decent judge of what's good and what's not. I know MTF charts, construction diagrams, and specifications for way more lenses than I ought to, but there are still lenses that I know are great without being able to show on paper WHY they are great.
I also know that really great photographers usually comment on how a lens draws or renders rather than the absolute resolution or MTF chart. They didn't get to be great without learning something so I pay attention to what they say. Even I, a mediocre photographer, often find myself absolutely knowing that a certain lens makes images that I find much more pleasing than another lens that on paper has better numbers. I hate that. I try to be scientific about these things.
So Are you Going to Tell us About the Thumbtacks Yet?
Not yet. Patience, my friends. Let's save those who shouldn't be reading this at all some time first.
This is intended as a moderately advanced article, so I'm going to assume (dangerous I know) that you understand what acutance and microcontrast are, understand in general terms what aberrations are, and know the basics of lens testing. If not, just click on the topic of interest in the previous sentence and it will take you to a superbly written article about that subject. You'll probably want to read those before you tackle this.
This started because I was discussing three 35mm prime lenses that I really like. I stated without hesitation that one was my favorite even though a different 35mm lens for the same camera measures out to be a sharper. So, I thought I'd research ways of testing the lens to measure other things, like distortion and aberration.
That didn't take long: after reading about methods like "attenuated phase-shifting mask ratios compared with simulated light intensities"; "multiple field measurements by laser collimator" and so on I tried to let it drop. I really need to develop that skill: letting things drop. Instead I found myself doing all kinds of comparisons with those three lenses, trying to find out exactly why I liked pictures from that lens so much. Without comparing any attenuated phase-shifting mask ratios.
I'm going to show a bit of what I found out in this article, comparing three different 35mm prime lenses. I'm not going to mention which lenses they are (although I'm sure some of you will figure it out) because this isn't a lens review. I think they are all excellent lenses and recommend each often, but it will become apparent they have some differences. The other benefit of choosing that focal length is that by definition 35mm SLR lenses are all retrofocus designs, which makes the lens designer bring out his entire bag of tricks to make a good lens.
Lens A is a good consumer-grade lens that most people consider an excellent buy. Lenses B and C are expensive, professional-grade lenses that are both considered excellent. Some people prefer one, some the other, but everyone considers both to be excellent. Each lens I used for this article has been through the steps described in my Basic Lens Testing article so we know they're good copies. They've also been compared to other copies of the same lens to make sure they are functioning as well as they should.
When we use the vague and unscientific term 'sharpness' we're really thinking of resolution and acutance, so that's where we'll being our evaluation.
Below are center and corner crops of resolution chart images from the three 35mm lenses (all taken at f/2.0 under the same lighting and conditions).
First, everybody who immediately thought "Lens B is the sharpest" go to the back of the class. Lens B does have the highest acutance in the center: the transitions from black to white (well, light gray) around the numbers, large bars, and black box are sudden and complete. If we apply some unsharp mask in Photoshop, we can improve the acutance quite a bit for Lens A and slightly for the other two (they're already excellent). This may explain why some people might claim Lens A is nearly as good as the more expensive pair. Its acutance in the center nearly as good as the other two with a little postprocessing, and acutance is the primary concern for photographers making only small prints or web graphics.
But let's look at the resolution - the smallest objects the lens can actually resolve clearly. Examine the smaller bars in the center of the images. On the center crops, even with Lens A, you can make out the three separate bars, both horizontally and vertically in the smallest set under the "0". In lens B and C you can also make out the first set under the "1". That's the resolution and in this case B and C are even and A, the much cheaper lens, isn't too far behind. Unsharp mask isn't going to improve resolution and in fact may cause artifacts in areas of the image nearing the resolution limits of the lens.
In the corner crops, Lens A falls farther behind the other two lenses in resolution and acutance. Lens B has lost a little more ground in the corner compared to its center than Lens C has. Lens B clearly had better acutance in the center crop, but B and C are pretty even in the corner. (You may have to look carefully to decide that though: since lens C vignettes the most, its corner is much darker than its center.)
Our resolution charts do a pretty good job of explaining why many photographers own Lens A and feel they've gotten a bargain: in the center, when acutance is the main consideration (small prints and web images), it does just about as good a job as lenses costing 3 times as much. It also shows why some photographers are happy to pay 3 times the price for the more expensive primes: if the corners are important to you, you have to spend the money. But the resolution test doesn't show why some people absolutely prefer Lens C to Lens B.
Resolution charts aren't very good ways to examine lens aberrations, they're too complex. And as I mentioned above, scientific ways to assess such things are too expensive and too complex to be used by mere mortals.
Introducing the PhotoGeekathizer
However, you can make a perfectly serviceable test chart to look at these kinds of things for about $12. It's simply a black 4 X 6 foot posterboard with round white thumbtacks placed in the center, diagonals, and in straight lines around the edges. This one is incomplete but complete enough to give you the general idea. (And it has day-glo yellow dots, because yellow shows up well in web browsers. And Staples was out of white ones.)
Doesn't look like much, does it? Wait until you see what we can do with it! Perfectly round dots can show us things that all the resolution charts in the world can't. My original plan was to market Roger's Photogeek-a-Thizer for about $600 a copy because I knew every gear-head on the planet would want one -- and I want to retire to Tahiti. Unfortunately, my attorney says I can't really get a patent for putting thumbtacks in a posterboard so I dropped Plan A and wrote this article instead.
Let's take a look at those corners again
I've cropped out the near-corner group of 5 dots and the corner dot and pasted the images shot with each lens at various apertures together so we can examine how each lens handles the edges and corners as we decrease the aperture. We all know that decreasing aperture improves many types of aberrations and sharpens the corners and edges, right?
First let's take a look at Lens A:
Wide open, you can see some coma-like distortion and smearing of the dots, especially in the corner at f2.0. As we stop down we notice it's better by f/2.8 in the 'near corner' (5 dot pattern) but not entirely gone even by f/5.6 in the absolute corner. Of course this area is not even visible on a crop sensor camera and rarely of importance in any photograph. Overall, I'd say this lens has sharpened up nicely by f/4.
One thing I will note: I used the term coma-like aberration. Modern lenses are all designed to correct aberrations to at least some degree so you rarely see a classic demonstration of the aberrations. Rather you see partially corrected aberration, like this. And reflected dots won't show as spectacular a coma as a bright point of light would. But evaluating the dots certainly demonstrates why the resolution chart on this lens is weaker in the corners in a way the resolution chart itself can't.
Things get a bit more interesting with lens B.
Wide open (f/1.4 for this lens) we see a very similar pattern to Lens A, but as we stop this lens down to f/2.8 the smeary coma tails disappear. However, a new aberration becomes evident: our circles are becoming distorted and oval-shaped, with the long axis of the oval at right angles to the smearing tails seen wide open. This probably represents astigmatism of the lens. The reason for the change is that coma and spherical aberration are reduced exponentially by closing the aperture, but astigmatism is only reduced linearly. (That's a bit of mathematical approximation, but a pretty close approximation.)
Lens C shows quite a different picture:
There is some blurring wide open, but no coma-like tail and no spherical aberration. At f/2.8 there is less aberration than with either of the other lenses at f/5.6. You can also see that the vignetting, which is quite severe at f/2.0 is much improved at f/2.8.
Notice also the background of the posterboard, particularly around the groups of 5 dots and compare it to the other lenses. Even at f5.6 Lens B and Lens A don't look nearly as realistic and 3 dimensional as lens C does.
(I should mention here that none of the 3 lenses in question have significant field curvature and all of these images were shot with best focus on the group of 5 dots using Live View bracketing, tripod, yada, yada. Focus is as good as it could be made for the lower left corner.)
But, Wait! There's More!
Since we have this nice setup, it's very easy to flip the lens over to manual focus and move the focus in front of, and then behind, our posterboard. The bright dots on the dark background give us the opportunity to see what the out of focus highlights will look like. One thing you may find surprising is they can look quite different in front of the plane of focus than they do behind it.
LensB gives us an interesting picture:
The out-of-focus highlights are quite different in front of the plane of focus, where there is mostly chromatic aberration, than in back, where there is a large amount of coma distortion.
While I didn't have time to create real-world photographs trying to reproduce the highlights like this, I can certainly see why bokeh might be considered less than ideal, especially in images having both foreground and background out-of-focus highlights.
Lens C is quite uniform front and back:
Again, one would assume the bokeh being more uniform would probably give a subtly more pleasing image. At least in certain images.
It's not my intention to show one of these lenses is superior to the other: the tool is too blunt and my experience with it too insufficient for that. But I do think we've clearly shown they are different. That, to me, is enough to explain why some photographers would clearly prefer one to the other.
I mentioned earlier that retrofocus (AKA reverse telephoto) lenses are perhaps the most difficult to design. This is partly because the elements can't be made symmetric about the central stop and symmetry reduces aberrations markedly. In reducing aberrations without symmetry designers may sometimes use one aberration to partially cancel out another. That seems to be the case with Lens B, which exhibits different aberrations at different apertures and markedly different out of focus highlight patterns in front of and behind the plane of focus.
This would certainly explain why Lens B showed coma which disappeared as we decreased the aperture, but astigmatism remained until the lens was stopped down further -- coma has greater improvement with decreasing aperture than astigmatism.
Speaking of aberrations, we generalize that they are worse the farther from the center we go, but they don't all behave exactly alike:
- Uniform severity across image: Spherochromatism, spherical aberration
- Somewhat worse further from center: Coma, lateral chromatic aberration (purple fringing)
- Much worse further from center: Astigmatism, field curvature, distortion.
We also know aberrations are lessened when we close the aperture. However, all aberrations are not created equal in this regard, either:
- Improve greatly with decreasing aperture: Spherical aberration, coma
- Improve somewhat with decreasing aperture: Astigmatism, field curvature, spherochromatism, lateral chromatic aberration
- Does not improve with decreasing aperture: distortion
It's hard to put dozens of images in a blog post, but the dots-on-poster board does a great job of showing what effect the distance from center and of decreasing aperture has on a lens' aberrations. That's information really worth knowing in the field: it helps you determine how best to frame the shot and what aperture to shoot at.
Now, if some of you have some thumbtacks and black poster board handy, let me know what kind of things you find! I've got all kinds of questions myself: about wider lenses and longer lenses; if different camera bodies (which have different microlenses on their sensors) behave differently with the same lens; and even if certain cameras are using computer algorithms to correct some aberrations in-camera.