Technical Discussions

Autofocus Reality Part 1: Center-Point, Single-Shot Accuracy

Published July 18, 2012

Once upon a time I wrote an article attempting to explain phase-detection autofocus and its limitations. That was a lot like trying to explain what happened to the Mary Celeste or why Stonehenge was built.

Actually we probably have a better handle on the Stonehenge thing than we do on the Mary Celeste or phase-detection autofocus. So rather than rehashing and speculating, I thought I’d instead do some simple demonstrations and explorations.

Obviously there’s a lot of ground to cover, so rather than making a 162-page blog post, I thought I’d break this up a bit. This first part will be very simple. We’ll compare the accuracy of phase-detection AF, contrast-detection AF, and Roger-detection MF (that would be me manually focusing using 10X LiveView) on still targets using just center point AF. While we’re here, we’ll demonstrate just what AF Microadjustment does, and doesn’t, accomplish.

The Setup

We use Imatest to demonstrate AF variation using Canon full-frame cameras and 50mm f/1.4 and f/1.2L lenses. The reasons are pretty straightforward: Our setup involves shooting a still test target with star, vertical, and horizontal focusing aids in the center from a tripod, so we should be getting very accurate AF. The setup we’re using involves shooting at a 12-foot distance so the depth of field is narrow. If focus is off by just a bit, Imatest will detect the decrease in resolution.

Phase-Detection AF Accuracy with an Older Design

We started with a Canon 50mm f/1.4 lens mounted to a 5D Mk II camera. The first thing we did (deliberately a bit out of focus so it would show up easily) was set the lens on manual focus and take eight repeated images without touching anything. This gave us a baseline (represented by blue diamonds) of the variation in testing procedures if nothing else changed.

Then we took eight repeated shots using LiveView and manual focus (represented by red squares), spinning the focus ring between each shot to either infinity or absolute close up. The red squares demonstrate Roger Units (how well Roger can manually focus given all the time in the world on a tripod with a perfect test target.)

Finally we did the same thing but let the camera autofocus in LiveView (represented by green triangles), resulting in contrast-detection autofocus. In theory this should be as accurate as Roger is, perhaps more so.

The graph below shows the results of those shots. For those of you who are not familiar with our Imatest graphs, the numbers reflect the sharpness of the image in Line Pairs / Image Height. The sharpness in the center is shown on the X-axis and average sharpness on the Y. Higher is better and, in this test, better focus equals higher sharpness.

 

The graph shows that there’s a little bit of variation when we take several shots without changing anything (blue diamonds) which probably reflects tiny movements from mirror slap, minute changes in lighting, or possibly even tiny fluctuations in sensor performance. Remember the blue shots are purposefully a bit out of focus so they’d show up separately in the graph.

When we tried to manually focus accurately (red boxes) or let the camera do it with contrast detection there is a bit more shot-to-shot variation. But the variation is still small, as is the difference between the camera and Roger. To put some numbers to it, the standard deviation of the baseline shots was 5.5; for the Roger-focus shots 11.5; and for the LiveView AF shots 8.4.

Now let’s throw away the repeated shot results from above since we’ve made that point, and replace them with standard (phase-detection) autofocus shots. These are taken in exactly the same way as the live view AF shots: take the image, spin the focus ring to one extreme, let the camera refocus, save the image.

 Two things become obvious:

1) When phase detection AF gets it right, it’s every bit as in focus as contrast detection or Roger detection.

2) Four of these 10 shots aren’t quite as accurate as the other six. That sounds like a ridiculously high miss rate but let’s put the numbers in a bit of perspective.

If we believe in Subjective Quality Factor (SQF) then an SQF difference of about five is needed to see a difference in a reasonable print or pixel peeping at 50 percent, given that the two lowest blue diamonds (phase AF shots) are going to show up as a bit soft or out of focus if we’re really critical. The third lowest might or might not look different from the “good shots”–it’s right on the edge.

Still, that gives us a “missed focus” rate of 20 percent in this little test, and that’s using center point AF with a still target. That’s not a big sample size obviously, and we repeated it with several other 50mm f/1.4 lenses and got “missed focus” rates of between 10 and 20 percent for all of them.

That’s not horribly out of focus, but it’s definitely pixel-peeping out of focus. For those of you who like numbers, the standard deviation for phase-detection AF ranged from 25 to 44 on different runs (always greater than the 5 to 15 we got with LiveView AF.)

Demonstrating Microfocus Adjustment

Next on our trial list, we thought we’d compare with a newer designed lens. The Canon 50mm f/1.4 doesn’t have a real USM motor and is known to be a bit difficult to focus. Not wanting to make the test too easy on the camera (and because we were set up at 50mm), we grabbed a few Canon 50mm f/1.2 lenses. These lenses have a modern design but again are known to be a bit “focus challenged”, to use the politically correct term.

One of the first copies we shot gave us the opportunity to demonstrate microfocus adjustment. We took our usual LiveView autofocus and manual focus charts, followed by a set of phase detection autofocus shots. When we graphed the results, we had to change the axis range because the phase-detection AF shots were so out of focus. Some of the shots still don’t show up; they’re even worse than the expanded axis shows. Unlike the above example, every single one of this is obviously out of focus at a glance–no pixel peeping necessary.

 

What happened was readily obvious: This copy of the 50mm f/1.2 was backfocusing badly on this camera. We did a quick microfocus adjustment (12 points) and reshot the lens with the results below.

Here is a superb example of what microfocus adjustment accomplishes. After adjustment phase detection AF is now very accurate, although there still is going to be shot-to-shot variation. For the two or three of you who like to scream, “I don’t want to us microfocus adjustment! The lens should be perfect out of the box!“, we went ahead and put the same lens on a different body with no microadjustment.

On camera No. 2 the same lens autofocuses accurately. The other 50mm f/1.2 lenses we tested all autofocused accurately on the first camera. The lens is perfect, just not with the first camera. The first camera is fine with all the other lenses we tested. Sometimes a given lens that’s fine doesn’t match up with a given camera that’s fine. So it goes.

The conclusion is pretty obvious: If you want to shoot wide aperture prime lenses and you don’t want to use microfocus adjustment, you just refuse to cope with reality.

We did several runs with 50mm f/1.2 lenses and found similar results all the way through. Standard deviations for sets using LiveView AF were 11 to 20, while phase detection ran from 20 to 35. These results are similar to what we saw with the 50mm f/1.4.

Conclusions

This really wasn’t a “conclusion” post; our intention was to just demonstrate some autofocus basics that most people know already.

  • LiveView (contrast-detection) AF on a still target is more accurate than phase-detection AF. It should be so. Contrast detection is using the actual sensor to determine focus; phase detection is not. Overall we found about one shot in 10 was out of focus with phase detection.
  • LiveView AF is as accurate as Roger View MF. You may be better than this, or you may not.
  • Phase-detection AF has more shot-to-shot variation than contrast detection. It’s not huge, but it’s real. This shouldn’t surprise anyone. Phase detection was developed for fast AF speed and to detect subject movement. It wasn’t developed to be more accurate.
  • Microfocus adjustment pulls good phase-detection AF results up to a par with LiveView, but it doesn’t eliminate the small amount of shot-to-shot variation that phase-detection AF has.

I’ll be expanding on this first article in some subsequent posts. Next I will be comparing the “two beep” AF method (where you push the shutter halfway down to focus then repeat before taking the shot) to the “single beep” method we used here. (I think the “two beep” method is more accurate for no reason other than when I started people told me it was so.  We shall see.)

We’ll certainly do some comparisons with different lenses on the Canon cameras and try to see if newer designs, STM, or other lens changes reduce the AF variation. Obviously we’ll have to get a larger database to be able to detect subtle changes, but that’s just a matter of repetition. (What we showed today were just examples. We’ve done a larger number of these lenses already.)

We will also look at different camera systems’ AF to see if they seem to have less phase-detection variation. I had thought about comparing contrast-detection systems in other cameras, but it’s obvious we’ll struggle to find one that’s better than Canon’s.

I’m not saying Canon’s is the best, but it’s as good as what I can do, so it may be difficult to detect “better” in this situation. We might be able to detect worse, of course.

Finally, within limits, I’m asking for suggestions about other things we can look at.

Remember when you make suggestions, that this model requires a fairly narrow depth of field to detect differences. Very long focal lengths (over 200mm full-frame equivalent) test at longer distances which may increase depth of field too much for us to detect small differences. Narrow maximum apertures (f/5.6 or so) would have the same effect.

Given those limitations, though, I’m open to suggestions!

 

Roger Cicala

Lensrentals.com

July 2012

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Technical Discussions
  • Mike

    Hi,
    Ive seen the same results on a Canon 5d MK4 with a 35mm F1.4LII and a 50mm F1.2L.
    Unless I have a real brightly lit high contrast target, phase detect gets 90-100% hit rate that drops to as low a 60% in low light. LV focus nails focus every time. I have done micro adjustments in the body and even sent the body to Canon to be checked (returned as meeting specs, no trouble found). Phase detect does have limitations with wide apertures. Luckily we have LV as a work around and can take multiple shots using Phase detect when live view is not practical.
    -Mike.

  • rafa0453

    I believe the newer USM lenses employ motors with higher resolution (angular dpi, or whatever). I’m blown by the focus accuracy of my 100-400 II on the 6D. Previously I used the 200 f/2.8 II for years, and the AF, while generally good (better than 50 f/1.4), it wasn’t truly consistent, even in One Shot AF. Despite being a ring USM motor, it’s 1996 vintage.

  • Dirk S.

    Hello,
    I just want to give my Cents to the discussion.
    One problem is the focus shift.
    Normally the AF sensors see only an aperture of 4.0 or 5,6. If the plane of best focus differs between 4.0 and 1.4, for say an 1.4 lens, the AF phase sensor never can detect the right focus, even if everything is perfectly mechanically adjusted. The AF sensor, limited to see f/5.6, never can determine the focus at 1.4!!!

    See:
    http://www.the-digital-picture.com/Reviews/Zeiss-85mm-f-1.4-ZE-Planar-Lens-Review.aspx

    Comment from Carl Zeiss on Focus Shift:
    “Fast lenses of this optical design (without floating elements) shift the focus due to spherical aberration when the f-stop is changed. This phenomenon is especially visible on closer object distances and cannot be influenced. The AF system of most camera models does not respect those characteristics of a lens. The focus is measured and confirmed by the focus indicator as if the lens has been stopped down to f/5.6.
    While using the lens wide open, the correct focus lies usually a little bit in front of the point that has been expected. Beside this, the focus confirmation function of any AF-camera is a relative improper tool for accurate focusing of an f/1.4 lens on closer distances. Also, a standard focusing screen and viewfinder of an SLR camera shows the limitations of accurate manual focusing with an 85mm lens at f/1.4. For instance, it is not possible in practical use to focus more accurately than about +/- 2 cm at an object distance of 1m without additional magnification systems. For accurate focusing at full aperture or stopped down a little bit, we strongly recommend:
    -use a tripod if possible
    -use zoom-in function in live-view mode while the lens has been stopped down to the designated f-stop.
    -focus bracketing exposures (with small steps of rotating the focusing ring)”

    So if I have lens with this problem, the contrast AF will work, but the phase AF will have problems with larger apertures. This can only be corrected by correction factors in the camera and/or lens firmware.

    Regards

    Dirk

  • Andy

    How do the newer Dual Pixel autofocus sensors compare to these methods? I would love to see an update!

  • I’ve just been through hell getting Canon to fix the soft focus problem on my 600/4L IS (I) and while it was giving me soft focus due to a misaligned element I noticed that phase detect AF was very erratic, sometimes way in front or behind the subject, even ones well isolated from potential competition for focus. Contrast detection in LiveView did give the best possible focus, as poor as that was, under the same conditions where phase detect failed.

    It seems possible to me that, when the AF systems can’t get very sharp focus, they just guess as best they can and that’s not very good. So, very sharp lenses may give much better AF than somewhat softer ones.

  • G.cavitori

    From my 3 years test i have got the same identical results of the dear Roger, author of this realistic article. Simply the phase detect is something that should be strictly used ONLY in particular situation where speed is more important of accuracy. Otherwise, for still, landscape or slow scene it is simply a shit.
    3 travels i come back home angry due to lackness of tack sharp focus on lot of very stupid day shots…..with live view never had problems
    And all this happened in different bodies with different lenses….
    I sent back for calibration but no luck, everytime if you take the same identical still shot 10 times, you can bet 10000000 dollars 3 or 4 over 10 pictures will be badly focus
    Good bye phase detect, end of the story. I revert back to good old days of compact cameras contrast detect system.

  • Rick N

    Excellent. You have confirmed my observations with clear concise data.
    Someone needs to start a business verifying the performance characteristics of each lens copy to a measurable standard that would instil confidence in the product. A few extra hundred for a certified lens would be worth it. This service could also provide an independent evaluation of a lens when repair by the manufacturers service centre fails to resolve a perceived issue.

  • Carl

    Roger, another great series of articles! (I haven’t read parts 2 and 3 yet).

    I think my camera has issues with the shim spacing (or whatever) of the focusing screen, because I can almost never focus accurately via the viewfinder (no matter what lens I use…telephoto, wide angle, fast aperture, etc…and I do have the finder’s focus adjusted for my eye). It’s possible this can only be so accurate, at least with lesser-than-pro series “crop” bodies, but I can only guess. Mine does have the “pentaprism” and not “pentamirror”.

    On an unrelated front, have you thought of doing noise test comparisons, like what they did here? http://www.techradar.com/reviews/cameras-and-camcorders/cameras/digital-slrs-hybrids/canon-eos-1dx-1091200/review/page:4#articleContent

    I find this particular test even more interesting than what DXOmark has done so far. Especially surprising is that the 5D3 they tested, outperformed all others in a narrow range from ~ISO 25k to 50k…besting even the D4 and 1DX. I’m sure there are logical explanations and counter arguments, but still it’s nice to look at a graphed comparison of noise and dynamic range directly comparing the latest and “best” DSLR’s.

    If you’ve already done a similar test, I apologize for not seeing it. I’m less familiar with your new website’s layout.

  • Roger Cicala

    Paul,

    I don’t think you can make any conclusion just by that. PDAF isn’t going to be as accurate as contrast detection. But if you have a problem with consistent front or back focus then a microadjustment is in order.

    Roger

  • Paul

    Very nice post. Thanks.

    Now i have issues with my camera and AF all lenses behave differently. My short question is: if say Nikon 50 1.4 works perfectly on all distances and all focus points in LV, but has many AF issues in PDAF on the same camera as other lenses works well. Can I conclude that it’s bad lens or problems with camera?

    Thank you.

  • Steve Runyan

    coincidently, i’ve been trying to find a better Canon 50 1.4 than my old one which has been improved by 2 visits to Canon service but is still disappointing wide open. i’ve tested 2 additional copies using Imatest. One appeared to be slightly better both with micro-adjusted pdaf and cdaf, one is much better with cdaf than pdaf, and the old lens is MUCH better with pdaf than cdaf. one of the tested lenses has focus shift in the opposite direction of the old lens.

    am i wasting my time trying to find a copy of this lens that will autofocus wide open with decent resolution (all of the lenses clean up nicely between f2.8 and f4)- i’m looking for decent (if not great) resolution off center for subject isolation and autofocus wide open for street shooting in low light.

  • Joaquín

    Hi Roger,
    I’ve just come back from a trip in Vietnam and Camboya. I’ve shot more that 5000 frames in 3 weeks. I’ve used a 5DMKII.

    30% of my shoots are out of focus. A third of them (10%) are due to my wrong using the camera ( after checking exif data) but still 20% of my shoots are simply out of focus, even using the new 70-200 f/2.8L IS II

    Trying to find an explanation, this site crossed my path. Roger, this is the clearest and most interenting reading I’ve found by far. Thanks a lot

  • Kelvin

    Thank you Roger for the feedback 🙂 I have recently been looking at a pile of my old 35mm negatives from last century and scanning them into the computer at around 20 megapixel equivalent. In hindsight I am amazed at how many of my old shots haven’t achieved critical focus. Most of them were taken with a Nikon F4 or FM2.

    In comparison, a lot more negs and slides taken with a rangefinder (Leica etc) had pin sharp focus.

    My critical focus hit rate on the negs / slides taken with a Hasselblad 500-series was abysmal.

    I am suspecting that SLR focusing has always been a compromise thanks to mirror box, mirror, and focusing screen alignment tolerances, but that they haven’t really come to prominence until high resolution digital sensors have come to the fore.

  • Roger Cicala

    Kelvin,

    It was live view manual focusing on a star chart – just because that’s where I’m most accurate and I wanted a good baseline. I’m not sure how accurate pentaprism would be in comparison: in theory it should be as good, but if the focusing assembly is shimmed improperly it can front focus or back focus.

    Roger

  • Kelvin

    Was RogerView manual focusing done using the focusing screen or the TFT screen? Just wondering whether careful manual focusing with the pentaprism focusing screen can be trusted as well.

  • boels069

    There’s a list of some USM lenses with the number of steps of the AF-stepping mechanism:
    http://redmodz.com/red-birger-lens-mount-unofficial-mini-faq-and-thread-compilation/

  • Roger Cicala

    hi Wayne,

    It was all at completely wide open, hoping to avoid focus shift if possible. I agree with Thom (because Thom’s almost always right) – I’m a cynical person and wonder, since the manufacturer’s don’t seem to want us to know about this, if knowing people are looking may make them a bit more willing to discuss it.

    Roger

  • Wayne S.

    Thanks Roger for doing all this testing and posting it.
    I didn’t see you mention what apertures you were shooting at but I assume at or near wide open.
    I ask because as you know fast lenses 1.2 or 1.4 usually suffer from some amount of focus shift which will cause the normal PDAF focusing to be much more inaccurate vs. the stopped down LV mode focusing.
    Also, I wish Canon and Nikon would allow more adjustment variables which can be tweaked liked the Conurus converted Contax N lenses. Not only is there a base micro-adjust value for a lens that can be specified but also parameters which affect the focusing value as function of distance from subject. These adjustments allowed me to get accurate near and far focus with my Contax N 50/1.4 lens.
    As Thom Hogan mentions in his blog, these AF databases will be interesting between Canon and Nikon and will hopefully add more pressure for these manufacturers to improve their AF accuracy. Most of the time I use MF Zeiss lenses and mostly use LV to get accurate focus where I want it.

  • Roger Cicala

    5D bodies use contrast detection for Live View autofocus, Phase Detection for standard AF.

  • photodummy

    How do you get the same camera to use contrast detect vs. phase detect? Is it as simple as the camera uses contrast detection when in “one shot” focus, and uses phase detect when in “AI servo”?

  • Roger Cicala

    Lester,
    I’m definitely going to test the Oly and other cameras. I’m afraid that my test tool here is too blunt to tell any difference. I didn’t mean Canon is as good as it gets, I meant it’s as good as my test can detect.
    Roger

Follow on Feedly