Technical Discussions

Autofocus Reality Part 3A: Canon Lenses

Published July 27, 2012

 

Sometimes I get to write cool, new stuff. Sometimes there’s just grunt work to be done, usually of my own making. This is one of those times.

To bring you up to date if you’ve landed here first, this is how I got here. Autofocus Reality Part 1 showed that on center-point, single-shot autofocus, standard phase detection was less accurate than  manual focus and LiveView contrast-detection AF. No surprise there.

Autofocus Reality Part 2 showed that two of Canon’s newer lenses, the 24mm f/2.8 IS USM and 28mm f/2.8 IS USM, seemed to have much more accurate phase-detection AF when shot on 5D Mark III bodies but not on 5D Mark II bodies. That was pretty surprising. To me at least.

That made the next step very obvious and not particularly fun.

We need to find out what other Canon lenses have the more accurate AF. If there are any, we’ll see if we can determine when this change occurred. Then we need to compare the other Canon bodies.

This post will simply compare Canon lenses (shot on the 5D Mk III) to see if we can find which ones autofocus more accurately.

Assuming what we’re seeing is real, I made the assumption that recently released lenses would be more likely to autofocus accurately. (I have a background in science. Meaning, until someone besides me reproduces this finding, it’s just a blip. Perhaps there are other explanations to what I found. Maybe the Imatest charts I use are just more accurate at 24mm or something.)

So I started testing the most recent releases and then worked backwards. That, I should point out, makes a big assumption.

My method assumes that the lenses are designed and developed in the order they are released. That’s probably not 100 percent accurate, particularly given Canon’s habit of announcing things about a generation before they actually appear on the street.

Today’s Test Subjects

I set up the following lenses for testing on 5D Mk III cameras over several days, grouping them by year of release:

Previously I’ve been putting up graphs of the individual results. They’re pretty and all, but they take up a lot of space.

A quicker way to compare for this article will be to look at the standard deviation (SD), a measurement of how much the numbers vary. In all of the previous testing, LiveView AF groups with 10 test shots had SDs between 10 and 20, as did phase-detection autofocus on the Canon 24mm and 28mm f/2.8 IS lenses.

Phase-detection AF sample groups on all other lenses had SDs of well over 20, usually over 30. I added one older lens, the Canon 70-200 f/2.8L  NON IS, because I would be testing zooms for the first time and wanted a baseline. This is a rather blunt tool, so it is not going to be 100 percent accurate scientifically.

I’ll add that I don’t particularly have the time to perform (nor do I expect you have the inclination to read) a scientific statistical analysis of the data involved. I consider this just a screening test–I’m just looking to see if there might be something worth investigating further.

Results

For brevity I’ll simply place the results in a table instead of showing a bunch of graphs.

The standard deviations are the average SD of (a minimum of) three separate runs of each lens for each type of focus. I have started off with some of the lenses from the previous AF posts and continued on to today’s results. I’ve listed the lenses by year of release, since I assumed more recent lenses were more likely to have improved AF.

Lens LV SD AF SD year
Canon 50mm f/1.4 9 34 1993
Canon 70-200 f/2.8 15 31 1995
Canon 50mm f/1.2 15 27 2007
Canon 24mm f/1.4 12 29 2008
Canon 100mm f/2.8 IS L 13 34 2009
Canon 70-200 f/2.8 IS II 14 24 2010
Canon 70-300 IS L 11 13 2010
Canon 24mm f/2.8 IS 10 12 2012
Canon 40mm f/2.8 13 16 2012

 

At a glance if you look down the LV SD column, which is the standard deviation of the LiveView contrast-detection focus runs, you see the variation is pretty consistently in the teens for all of the lenses tested.

Looking next at the standard deviation of autofocus we see there are accurate focusing lenses (which I’ve marked in bold) and other lenses. The accurate focusing lenses are very, very close to LiveView results.

I didn’t show it in this graph, but the 28mm f/2.8 IS is also an accurate focusing lens. There’s a bit of a surprise in that the 70-300mm L appears to focus with the accurate group, while the 70-200mm f/2.8 IS is kind of on the border. Not in the teens certainly, but it’s the best of all the other lenses.

Both of these were released in 2010, and I’m not sure what this means.

It might be that the 70-300 seems more accurate because it’s being tested at f/4 rather than f/2.8, but we’ve tested other lenses at f/4 and they weren’t this accurate. The 70-200 f/2.8 IS L may be better than the major group of other lenses but not quite as good as the newer lenses. Or maybe not. Testing three copies probably isn’t sensitive enough to make that call.

 And the 300mm f/2.8L IS II

There was one last task to be done. The new Canon Supertelephoto II lenses are all recent releases: 2011 or 2012. They replace lenses designed in the 1990s, which were considered some of the best lenses Canon had made. So we have to check them.

We tested the 300mm f/2.8 IS II for a couple of reasons, but first among them was that our Imatest setup starts struggling at 400mm. I felt the 300mm should be representative of the group.

Because I felt supertelephotos were different lenses than the others we tested, we checked both the 300mm f/2.8 IS and IS II on both the Canon 5D Mk II and 5D Mk III.

As expected, neither lens proved to focus more accurately with the 5D II camera, so I won’t go into that further.

We will post graphs, like in the previous articles, of the results of a sample series (one copy of each lens on the same 5D III) as an illustration–they are pretty and all. But I repeated the tests with other copies and received the same results.

First is the 300mm f/2.8 IS. There was a bit of frontfocus (the difference in average between the two groups), which I could have helped fix with microfocus adjustment, but I didn’t have the time.

The standard deviation for manual focus was 20. For phase detection AF, it was 35. I’m not certain why there was difficulty with contrast-detection AF with this lens (most other lenses were in the teens as the table above shows), but it showed about the same number for all three copies tested. Clearly, though, phase detection AF was less accurate, as was the case with all other older designs.

The 300mm f/2.8 IS Mark II, though, did behave differently on the 5D Mk III.

As expected, the 300mm f/2.8 IS II has more accurate phase-detection AF than the older version 300mm f/2.8. In the example above, it is absolutely as good as the contrast-detection LiveView focus. If you prefer the standard deviation numbers, the 300mm f/2.8 IS II had an average SD of 17 with Live View and 17 with phase autofocus.

As an aside, in case you haven’t noticed, I had to change the scales of the resolution graph for the 300mm f/2.8 IS II. It’s an amazingly high resolution lens.

The older 300mm f/2.8 may look bad in comparison, but those numbers are very good. The original version is sharper, for example, than the 70-200 f/2.8 IS II. But the new version is amazingly good.

So What Did We Learn Today?

Well, we learned that on 5D Mk III bodies, the newest Canon lenses seem to autofocus more accurately, nearly as accurately as LiveView contrast-detection focus. And of course, standard phase-detection AF is much faster and more convenient than Live View.

As best as I can tell, the three lenses released in 2012 (24mm, 28mm, and 40mm f/2.8), the 70-300 IS L lens released in late 2010, and the new Mk II supertelephoto lenses are more accurate.

Maybe the 70-200 f2.8 IS II is, but probably it isn’t. I’m not able to make the call since it seems to be somewhere in between the “best” group and the other lenses.

Oh, yeah: we learned the 300 f/2.8 IS II is amazingly, bitingly sharp.

Next up, obviously, will be comparing with other Canon bodies.

I think we can safely assume the 1Dx will be at least as good as the 5D Mk III. We can also assume the 5D Mk II and previous cameras with the same AF system are not. It will be interesting to see where the 1D Mk IV, 1Ds Mk III and 7D fit in this picture.

Roger Cicala

Lensrentals.com

July 2012

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Technical Discussions
  • Eric Bowles

    Roger – This series on AF continues to be a good reference on the issues we encounter with focus. The idea that we have a distribution of AF errors with all AF techniques is very important to lens evaluation and fine tuning. It’s also important to understand technologies are constantly evolving – both with the lens and the camera body.

    Is there any chance you will update these articles comparing some of the newer mirrorless technologies as well as state of the art DSLR AF? With all the comments about AF on the Nikon Z cameras, it would be great to see how the Nikon D850 and Nikon Z7 compare – particularly in terms of standard deviation of AF errors..

  • Klaas Groothuis

    Where is Part 3 B and so on ?

  • Tom

    Hi, What do the graph axes and SD value represent?

  • Phil

    I know I am reading this two years on, but sterling work and very interesting.

    Considering phase autofocus, one could surmise that the feedback loop with the newer lens/camera combination has a greater bandwidth, allowing a greater accuracy/repeatability to be attained within a given time. That being the case, even older lens/camera combinations should be able to achieve the same accuracy/repeatability as the new, it would just take longer.

    If this is true, it could explain why LV focus accuracy/repeatability does not change much from camera/lens to camera/lens, given LV seems to allow whatever time is required.

    Again, assuming the above is true, it is a shame a custom function was not provided to trade accuracy/repeatability against focus time for phase focus.

    Phil.

  • Roger Cicala

    Per,
    I’m not the person to comment on it, really, there are others with far better skill sets in that area than I. But for me, it is very hit or miss and I just stopped using it.

    Roger

  • Per Inge Oestmoen

    Roger, when you say “The 7D in AI Servo mode is a place I just never, ever go” what do you mean? Are you pointing to any shortcomings with the 7D’s AF? I use the 1D IV and 1D X for all forms of photography, and I know the 7D to have a less capable AF – but is it truly inferior to a degree where it cannot be recommended?

Follow on Feedly