Why You Can’t Optically Test Your Lens with Autofocus

By far the most common email and PM I get basically says, “I think my lens is optically out of sorts. What do you think?” The second most common is, “So what’s the best way to optically test my lens without a ton of equipment.” Years ago I wrote an overly long post about testing lenses, but it was not specifically about optically testing. Plus, I’ve learned a bit since then and have been trying different things that anyone could do without a lot of equipment.

So I thought I’d write a series of posts about optically testing a lens. Not ‘determine MTF and write lab-test reviews’ kind of testing. But enough to tell if your lens is optically centered and a good copy.

We all know simply taking a few hundred images will tell you if you like the lens or not. But sometimes people don’t have that kind of time. They need to know if their lens is OK while they still have the opportunity to return it.

Other times they know they don’t like the lens, but they do want to know if that’s because they have a bad copy (especially if they’ve bought a used copy), or if that’s just how the lens is. There’s no sense trying 5 more copies of the lens if the one you tried at first is a good representation of how the other copies are going to be. But there’s also no sense giving up on a lens that everyone raves about just because you tried a bad copy. (For those who want to know, overall our experience is about 2% of lenses we buy are out of spec right out of the box.)

I’ll get into the setup I recommend for optical testing over the next couple of posts. But first, let’s talk about why I tell 90% of the people who ask me if their lens is decentered that I can’t tell anything from that image because it’s taken with autofocus.

Without Proper Focus, Testing is Useless

Critically accurate focus to test a lens is very different than acceptably accurate focus to take a picture. In order to make a nice print, everything within the depth of field is acceptably sharp. Notice the definition is acceptably, not maximally. The area of maximal sharpness is less than the entire depth of field.

Let me show you some examples using Imatest. You don’t need Imatest or any other expensive stuff to test your lens, but it does give us nice numerical data that makes a better demonstration than pictures could. Like all optical testing, Imatest is very sensitive to focus.

Here’s a simple demonstration. I set up a Canon 70-200 f/2.8 IS II on a 5D Mk II and obtained the best possible focus I could. Then I took 10 repeated shots changing nothing. This simply establishes how much variation in MTF 50 occurs when nothing about the camera or lens changes. Variables like minute lighting changes, mirror or shutter slap differences, perhaps even electrical interference or trucks driving by the building cause this slight variation.

 

 

Next I took 10 more shots, turning the focus ring to one extreme or the other and then carefully manually focusing each shot. I used a 17″ external monitor for manually focusing and there’s a specific focus target on the Imatest chart, so this is as good as I could ever hope to manually focus. Even under these ideal conditions, there’s a bit of variation compared to the 10 shots where I touched nothing between each shot.

 

 

Next I repeated the exercise, this time using the camera’s live view contrast detection AF system. The 5DII isn’t the absolute best contrast detection system I’ve worked with, but it’s not bad, either. Contrast actually is really close to my manual focus capabilities, but there is a bit more variation.

 

 

Finally, I repeated the exercise using the camera’s phase detection autofocus. Again, the 5DII isn’t the best phase detection AF system in the universe, but it’s certainly decent on a still target with center point AF and an f/2.8 lens. (And yes, we had checked microfocus adjustment and this camera and lens were spot on at this distance.)

 

 

To summarize my point, if I am checking a lens like our techs do all day, manual focus is simply much more accurate than phase-detection autofocus.

 

 

Let me again differentiate ‘testing quality focus’ from ‘picture taking focus’. If I was taking a 3-Dimensional picture, all of the images above the 850 line would be virtually indistinguishable. The two phase-detection shots that are lower would probably be acceptable; if you were comparing identical shots you might notice the two had slightly missed focus, but not horribly. But when we’re assessing a 2-dimensional test chart, that would be enough to make the lens look unacceptable.

Phase detection accuracy is different with different cameras and even with different lenses on the same camera. The pair used in this example are quite accurate; not the very best, but better than average. The very best (in these conditions) we’ve tested, the Canon 5DIII or 1Dx with certain lenses, are nearly as accurate as contrast detection AF.

Good microfocus adjustment lets the phase detection system focus at the proper location, but it can’t make the pattern of variation much smaller. (If MF adjustment is really off, the pattern can be bigger than this, but this is about as good as it gets.)

One other point that will become more apparent with the next two posts: some of the most sensitive indicators of a decentered lens are seen when it’s just barely out of focus. So the testing I’m going to describe will require evaluating the lens both in focus, and just barely out of focus in both directions. You can’t do that kind of evaluation with phase-detection AF.

So What’s Next?

This first post was to demonstrate that hand-held, autofocus optical testing isn’t very useful. If you want to test a lens using autofocus go take pictures and see if you like the lens, which is really what I recommend everyone do, anyway. But if you have a lens you don’t like, one that seems softer than it used to be, or bought a used lens that doesn’t seem nearly as good as it should, this series should be interesting.

In the next post I’ll describe a pretty simple testing setup that will you do a pretty good job of detecting a decentered or optically maladjusted lens in 10 minutes or so. It won’t be quite as good as a $150,000 optical bench, but it’s close. We not only use this setup for our basic lens testing in the intake room, we use it for some basic optical adjustments when lenses are decentered.

I expect many of you will have everything you need to set it up already. If not, you still won’t spend more than a couple of hundred dollars. Think of it as 90% of the function of an optical bench or Imatest lab for almost none of the cost.

In the third post we’ll give several examples of how to use the setup, showing what good and bad lenses look like. And if anyone is interested, we’ll bring a setup to WPPI in Las Vegas next month where you can play with it yourself.

 

Roger Cicala

Lensrentals.com

February, 2014