LensRentals.com

Autofocus Reality Part 3B: Canon Cameras

Posted by

Some days it’s good to be a geeky gear-head. This is one of them because a) I finished testing autofocus on all of the Canon camera bodies we had, b) I actually found out some interesting stuff, and c) I got worked up about camera marketers while doing it, so I have my next post in the works already.

Because I’ve been writing this series on the fly (telling you what we found as we found it), I’ll keep going in that fashion and keep the conclusions for the end of the article.

What We’ve Found So Far

In the first Autofocus Reality article, we demonstrated two things:

1. Phase-detection autofocus (even using still targets and center-point only) wasn’t nearly as accurate as contrast detection.

2. The contrast-detection autofocus was about as accurate as the most careful manual focusing.

Part two of the series showed that a few newer lenses did focus as accurately as contrast detection on 5D Mark III cameras but not on 5D Mark II cameras. The third article (part 3A) showed that the newest Canon lenses (40mm f/2.8, 24mm f/2.8 IS, 28mm f/2.8 IS, 70-300mm L IS and 300mm f/2.8 IS II) focus more accurately when mounted to 5D Mark III camera but not on 5D Mark II cameras.

The Canon 70-200 f2.8 IS II, oddly enough, seemed not quite as good as the above lenses, but more accurate than the older ones. Whether this was oddness in my measurements or a real finding, I wasn’t sure.

Since we knew that the newest lenses autofocused accurately on 5DIII but not on 5DII cameras, the next step was obviously to compare an accurate AF lens on different camera bodies to see with which ones it was capable of accuracy.

We expected the 1Dx (which has the same AF system as the 5DIII) would be accurate. We weren’t sure about the others.

Today’s Contestants on The Focus is Right!

We know the Canon 28mm f/2.8 IS II had accurate autofocus on the Canon 5DIII so we chose one copy to be our test lens.

We AF microadjusted each camera to the lens prior to shooting. If the camera did not offer AF microadjustment, we checked the body with the test lens and exchanged it for another copy if the lens back or frontfocused at the test distance.

(Microfocus adjustment makes absolutely no difference in the shot-to-shot variation in AF–it only improves the average value of the group. But I got tired of explaining that to people in the previous articles. It was easier to just do it than to answer another 50 emails.)

We then tested it in our Imatest lab using one copy of each Canon camera we carry. To save you from running amok on the internet, finding out which cameras are how old and have what kind of autofocus, I’ve listed that information in the table below.

 


Camera Year released Year firmware AF description
1DsIII Dec-07 12/16/09 45 point, 19 cross, 26 assist, f/4 at center, dedicated AF processor
5D II Dec-08 2/28/12 9 point, f/2.8 cross center
50D Dec-08 2/28/12 9 point cross, dual diagonal center cross
7D Aug-09 4/25/11 19 point, all cross, center dual diagonal cross (advanced algorithm), first “zone AF” and “spot AF”
1DIV Dec-09 3/29/12 45 point, 39 cross (f/2.8-f/5.6, f4 at center), first AF expansion camera
60D Aug-10 6/19/12 9 point, f/2.8 cross center
T3i Jan-11 1/30/12 9 point, f/2.8 cross center
1Dx Mar-12 2012 61 point reticular, 41 cross type, 5 dual diagonal cross, including center
5D III Mar-12 2012 61 point reticular, 41 cross type, 5 dual diagonal cross, including center
T4i Apr-12 2012 9 point, f/2.8 dual cross center, hybrid CMOS AF Live View

 

I suspected that the autofocus improvement we’ve seen had more to do with hardware than firmware, but I listed both year of release and year of latest firmware upgrade for completeness. I also listed the basics of the camera’s AF system, as well as any marketing comments made about it at release like the 7D having “the most advanced AF algorithms.”

My thought going in was the difference would be in hardware not firmware. I don’t think any amount of firmware is going to make the AF sensor on the lower left behave like the one on the lower right, even in center-point, single-shot mode.

 

The Results

In the lens variation article, we used the standard deviation (SD) to measure how accurately the camera focused: Softer focus results in lower Imatest values. As an example, we’ve put up a graph of Imatest values for the 5D Mk II versus the 5D Mk III below.

 As you can see, the 5D Mk III shots (red square) are all very similar. The 5D Mk II shots (blue diamonds) are more spread out: The variation in shot-to-shot focus is greater.

In this example, the SD of the 5D Mk III samples was 17 lp/ih, while the SD of the 5D Mk II was 38.5. Those are similar to the numbers we’ve seen over and over—accurate focusing combinations have SDs in the teens while less accurate ones have SDs in the 30s.

Rather than clog up the post with a lot more graphs, I’ll list the SDs of the various cameras with the 28mm f/2.8 IS lens in the table below.


Camera SD
1DsIII 29
5D II 38.5
50D 34
7D 41
1DIV 22
60D 34
T3i 41
1Dx 17
5D III 17
T4i 29

 

It’s a little confusing. There’s a range of variation, of course. But clearly the 5DIII and 1Dx do better than the other cameras, while the 1D Mk IV seems to be a bit between those two and the rest of the pack.

It seems a little clearer to me if we graph the standard deviations and separate the cameras by type (more expensive at the top, less expensive at the bottom.) The oldest cameras in both groups are on the left, while the newest are on the right.

 

A couple of points are worth making.

The graph of the more expensive cameras seems to show a pretty logical progression. The 1Ds III is by far the oldest, having been around since 2007. But it included every possible AF technology of the day, including a separate AF processing chip. The 5D Mk II, even on its release, was known to have “consumer-grade” autofocus.

Despite my well-recognized modesty, I will also point out that when the 5D Mk III was first released, and Canon fanboys were dropping off cliffs right and left, I said “the 5D III is no minor-upgrade camera; it’s an entirely new camera using the old camera’s name”. Its autofocus system is certainly not a minor upgrade–it’s moved over to the big-boy camera side.

I had hoped the T4i might be more accurate than it was, at least with new lenses. It does seem more accurate than the other consumer / prosumer cameras in phase detection, but it’s not nearly as good as the 1Dx or 5D III.

I assume that it’s new hybrid LiveView system does not carry over to create phase-detection AF. I will say, in it’s defense, that when focusing in LiveView it is obviously faster than and just as accurate as any of the other cameras, including the 5DIII and 1Dx.

So Why Could This Be?

All this autofocus stuff 1) gave me a headache and 2) made me rather curious and uncertain.

I started doing a simple demonstration of what I already knew: Phase-detection AF isn’t as accurate as contrast-detection AF. But then I got results that indicated sometimes it is just as accurate as LiveView. But you have to have a certain camera and a certain lens or it doesn’t happen.

This didn’t make much sense to me.

I would have understood if each generation of newer cameras and lenses got a little better. Or if a new camera or lens was dramatically better. But why a rather sudden change, and why did you need both a new camera and a new lens?

I spent a fair amount of time emailing with Dave Etchells of The Imaging Resource  and SLRGear.com who was kind enough to bounce ideas back and forth with me. His thoughts sent me on the right track for figuring this out.

I started off by trying to find out how long it took for an autofocus improvement to go from idea to released-to-the-public. I found one answer in a Canon patent from 2003 describing combined phase / contrast-detection AF in the camera mated to a lens using a stepper motor.

Which we saw  . . .  oh, yeah. Now. With the Canon T4i / EOS-M sensors and new STM motored lenses.

One thing of interest in this patent application, tucked away in the background section, is the following statement:

. . . though the prior art indicates a method for realizing high speed and high precision of autofocusing at the same time, it does not accompany a lens drive control for realizing this and thus does not adequately realize [. . .] high precision of autofocusing.

In other words, back in 2003, Canon recognized that a high-accuracy AF system in the camera required a more accurate lens drive to yield precise autofocus. You can’t have one without the other.

Suddenly the need for both a new camera and a new lens to get accurate autofocus began to make sense. But wait, the patent was talking about using a stepper motor to achieve accurate AF. We did find the 40mm pancake (stepper motor) was more accurate. But we also found that the new 24mm, 28mm f/2.8 IS and 300mm f/2.8 IS II lenses were more accurate. Yet they don’t have stepper motors.

Then I read on a bit further in the patent:

In order to achieve this objective, this invention provides a camera system comprising: a first focus detection unit, a second focus detection unit, a stepping motor that drives a focusing lens, . . . or a rotation detector, which detects the rotation . . . of the motor. . . The control circuit performs closed-loop control, based on the output of the rotation detector to control the motor.

The wording of this patent, back in 2003, suggests that closed-loop was not how AF worked at that time. It was largely open loop. The camera took a measurement and told the lens where it should go. Done.

This is backed up by a lot of other information, including statements in Canon’s “EF Lens Work III” and quotes made by a lot of people who would have known.

It’s clear that later AF systems were closed loop (the camera double checked where the lens had gone), although exactly when and where that change occurred I don’t know. But this probably explains the lengthy, and now largely meaningless, debates about whether systems were open or closed-loop: They used to be open. Now some (probably most) are closed.

More to the point, though, is the comment that a rotation detector would be needed in lenses driven by ultrasonic motors to increase accuracy of the focusing movement.

If this is the case, then the newer Canon lenses should definitely have a rotation detector built into them. We know there are rotation detectors in many lenses released after 2000, but if they are  in older lenses we can’t identify them, so this fits too. (As an aside, I am particularly skilled in finding them because usually if you touch them with your fingers the lens won’t focus anymore and the unit has to be replaced.)

 

From Left: A Canon and Nikon magnetic rotation detector.

 

Just to be certain, I asked Aaron to find an excuse to take apart one of the new lenses. Or failing that, I told Aaron to take one apart and not tell Tyler or Drew, who never really believe we can put them back together correctly.

The result: Aaron checked and the newer lenses did have rotation detectors.

There was one other big hint hiding in plain sight in Canon’s resource article on precision cross-type AF sensors. The article states that the farther apart each pair of AF sensors are (each pair compares phase from opposite sides of the lens) the more accurate the sensor. It also states only the 5D Mk III and 1Dx have them. I italicized some of the quote for emphasis:

[...]simply by reading which pixels on each sensor line are being struck by light, the AF system can tell instantly what direction to move the lens in for proper focus, and by how much to move it. . . Canon EOS SLRs with high-precision AF sensors simply move the pairs of sensors much farther apart, and accordingly, the AF information can be more finely broken-down and reacted to.

The 5DIII and 1DX have f/5.6 cross-type sensors (most previous cross-type required f/2.8 or faster lenses). The central AF sensors in these cameras also have diagonal cross sensors. The article continues:

For pros who prefer to use the center AF point, simply manually selecting it [. . . ] gives the user the highest precision AF possible with these cameras (assuming an f/2.8 or faster lens is being used). And, with their unique diagonal cross-type layout, the AF points are much more likely to latch-on to typical horizontal or vertical subject details.

In Summary

As it turns out, my little tests just confirm things that were already published (if obscurely.)

The two newest Canon cameras have more accurate phase-detection sensors than their previous cameras. The newest lenses have more accurate focus movement (or provide more accurate focus movement feedback, or both) that takes advantage of those sensors.

Older cameras don’t have accurate enough AF sensors to take advantage of the new lenses’ capabilities.  Older lenses can’t move their focusing elements with enough accuracy to take advantage of the new cameras’ accurate sensors.

It’s rather sad (given the amount of other work that’s piled up during this little series) that I didn’t find this information until after I’d done all this testing, but it’s rather a testament to our times. There’s so much marketing drivel and useless verbiage thrown about that the marketing noise drowns out the actual useful information the camera makers offer us.

But that’s the topic of another blog post in which I will take personal offense at the marketing crap that’s been shoveled our way at an ever-increasing rate.

I don’t believe we photographers are nearly as stupid as the marketers seem to think we are. In the meantime, while I’m on this rant, I highly recommend Thom Hogan’s very funny look at camera icons “A Different Kind of Focus.” Thom writes better stuff sitting on the tarmac waiting for takeoff than I do at my desk.

And I guess, given all the marketing fluff, I wouldn’t have really believed “better autofocus” if I hadn’t seen it with my own tests.

I’ve heard it before and it wasn’t so. Not to mention I doubt seriously we’d have seen the marketers say, “better autofocus…but not with the lenses you already have.” So maybe this is worthwhile after all.

Otherwise I might have put my trusty old 85mm f/1.8 on a 5D III and thought “not better at all, they’re lying to me again.”

 

Roger Cicala (with thanks to Aaron Closz and Dave Etchells)

Lensrentals.com

August 2012

169 Responses to “Autofocus Reality Part 3B: Canon Cameras”

LensRentals Employee

Roger Cicala said:

Wilba, I don’t, really don’t, want to debate semantics. If by closed you mean there is at least one follow up reading, with extrapolation of data gained from secondary readings then that’s correct.

If you mean there’s is some confirmation reading that says “this is the correct point”, then I disagree. Phase detection by principle is less accurate the closer it gets to the focus point. Unlike contrast detection, it does not choose a ‘best reading’ point.

There are at least 3 control loops inside a lens, and at least 1 inside a camera. But the overall behavior depends on settings, camera, lens, and a host of other factors including light temperature, strength, others. I’ll add that there’s also a differentiation in lens feedback in different systems and even within system in different lenses. Nikon uses at least 4 different types of position sensors. Canon at least 3. Making a blanket statement that is supposed to cover a gear driven lens with perhaps 700 steps and an older magnetic sensor and also cover a USM driven lens with 3500 steps and a new optical position sensor is making a very, very broad statement indeed.

I don’t absolutely disagree – you may well be correct. But I fear you’re trying to paint a gray picture black and white and I’m hesitant to do that. I’ve seen to many conflicting patents and too much conflicting data cloaked in too much corporate secrecy to be comfortable with that.

But in the interest of not arguing let’s just assume you’re correct and there is a closed feedback loop. If so it’s a fairly inaccurate closed feedback loop, and an inaccurate closed feedback loop isn’t all that different from a semi closed loop in behavior.

Roger

Dave Sucsy said:

Roger,

You’re a gem and a genius! You make all this geeky stuff fun, funny, and even useful!!!
I can’t thank you enough!

If you ever tire of being a lens geek and businessman, there is undoubtedly a great future for you in technical writing, science fiction, or comedy!

Oh. One other thing. If you get bored and need something to do, I’d like to see equivalent information on the Nikon D800/e system and lenses. And the reason for Stonehenge.

Thanks!

Dave

LensRentals Employee

Roger Cicala said:

Dave,

My previous career (well, one of them) was in technical writing: 5 books and uncounted chapters and articles. You can probably tell by my grammer how much I miss having a full-time editor.
Unfortunately, my writing style was 1) set a deadline 3 months from now, and 2) realize 2.5 months from now that I hadn’t done doodly on the article due in two weeks. I need more structure :-)

Roger

Bill said:

I’m a long-time Canon owner and i refer myself to an advanced hobbyist. How can i know which bodies perform closed-loop focus operations, which bodies have the ability to work with the rotation detectors, and which lenses actually have the sensors to support this finer focusing method? I have some older lenses that I would assume will not have this capability, and some newer ones that I hope do. Further (and this may need to be addressed separately), what about third-party lenses? I happen to prefer Sigma when I’m not finding what I need from Canon.
Thank You.

TTMartin said:

Canon stated that firmware 2 for the 7D was based on lessons learned in developing the 1D X. It would be interesting to see if firmware 2 for the Canon 7D would change the results at all for that camera. i.e. is it a hardware improvement in the newer cameras, a software improvement, or a combination of the two.

CyberDyneSystems said:

Thanks for another fantastic insightful article Roger.
Your hands on dig out the guts and see what they tell us approach is much needed.

I am another that feel the 5DIII is/was a game changer, the fabled “EOS 3D” by a more familiar name. I’ve been enjoying it’s superior AF in a more affordable (and full frame to boot) body very much!

Alex said:

Roger,
I am not trying to be a hole of an ass, but weren’t you referring to a grammar when you mentioned “grammer”?
I know, I can come across as a prick, but believe me I am not…(smiley face)(wink, wink).

LensRentals Employee

Roger Cicala said:

Alex, you are correct. I have major grammar failings – for all of my life before now I’ve had an editor. I never realized how much that encouraged sloppiness in my writing until I didn’t have an editor anymore.

John Bickford said:

What about the Canon 6D? What about the 6D specifically with the 24-105 1.4mm L zoom lens? What about zoom lenses in general?

Tony Webster said:

Sir, whenever you is nex in Barbados, I kin offer you free scenic drive; cupple cold beers; and lessons in how to spik and onderstand bajan language. Tip of my hat to you Zir, on velly amazin articul. Thanks mil for exercizin my brains , or residue thereof. No yoke, when in “Bim”, please advise advace arrival. Would be honor. Camera talk not allowed; jes relax; smell the flying-fish frying; Atlantic breezes will cool over-heated brain. Terif article.

Wilba said:

Hello again. By closed-loop I mean exactly and only what it has always meant – feedback about the state of the thing being controlled is used in working out how to control it while it’s being controlled.

Test any EOS camera with a half-press in One-Shot mode and it will show you that focus is only confirmed when the AF sensor sees an in-focus subject. Only a closed-loop using feedback from the AF sensor can do that. Here are the tests – http://www.dpreview.com/articles/5402438893/busted-the-myth-of-open-loop-phase-detection-autofocus. Don’t assume I’m correct, prove or disprove it for yourself. I’d love to hear how you go with them.

JB said:

Roger, will you have the opportunity to see how the 70D AF stacks up? There seems to be quite a controversy as to whether the 70D has the same phase detect AF system as the 7D, or perhaps even a bit better – unfortunately with all the hubbub about the new LiveView capabilities the phase detect AF system of the 70D has been largely ignored.

Daniel Low said:

Roger,
What a fantastic technical article on Canon autofocus! Mighty thanks for shedding some (a lot) light on accuracy of autofocus of dslr

Peter L said:

Thank you for your excellent experiments and articles on phase detection. I often have problems with autofocus misses on my cheap Canon 1000D, particularly with the old EF 35 mm f/2.

Looking for answers, I stumbled upon another site, which offers some results on autofocus acuuracy:

http://www.optyczne.pl.

It is in Polish, but you can get a reasonably good translation by Google. Some articles have been translated on another site, e.g. on the new EOS 70D:

http://reviews.1001noisycameras.com/canon-70d-at-optyczne-pl/

Keep up the good work!

Anders said:

Note that a closed loop control of the lens’ movement, after determining where to go by a measurement of the AF system, isn’t the same things as a closed loop control of the whole focusing process. Controlling the lens doesn’t close the loop all the way back to once again measuring focus, to figure out if the movement done by the lens not only did move the lens to the desired point but also actually achieved correct focus.
High precision AF points, spread far enough apart to require f/2.8 lenses, have been around for a long time. The diagonal points available in the 5D Mark III and 1DX (pioneered by the 40D) only add high-precision in two perpendicular directions at the same time. Older cameras like the 1D Mark III and 400D also support high precision, but only in one direction.
A novelty, though, within the 1DX is that all points have dual elements in zig-zag configuration for each orientation. That effectively increase the resolution by a factor of two, without sacrificing signal to noise ratio. The AF sensor in the 7D only has three such elements, to compare with one simpler camera.

Igor said:

Thanks Roger, very interesting. I would suggest to test the effect of the light spectrum on the PDAF accuracy. To start with, my 600D/Sigma 17-50f/2.8 OS work perfectly in sunlight, but in skylight there is a considerable backfocus (or may be front, can not remember now). Are there any cameras that are good in this respect there?

Lynn Allan said:

Old article … but you might check with the gurus at MagicLantern to see what they think and may have observed.

Phil said:

Another excellent article.

It would be interesting to hear your views on the EOS 70D controversy. The web has been alight with complaints about this camera’s phase detection and Canon has been silent. I for one had to dial in +15 MFA for a 50mm 1.8 for 2.5m distance, only to find that a figure of +10 was more applicable for much longer distances. The CF point is supposed to be more precise than outer ones, but I did not find this.

The biggest issue was with repeatability and indeed your results for the 7D (assuming the 70D has the same AF sensor) do indicate this camera is worse than others. Even my old 400D was more repeatable!

I could surmise that Canon, with the 70D, has tried to reduce production costs by removing/reducing AF sensor calibration, instead relying upon improved manufacturing tolerances, but has not entirely succeeded. I noticed that German purchasers report far less satisfaction than US purchasers, from looking at Amazon feedback, which is specific to the 70D (i.e not 7D, 60D) so perhaps Germany has been fed from a dodgy production line.

On the other hand, are we just expecting too much from cameras these days? If so, Canon ought to come out and tell us. Your thoughts would be welcomed. Indeed, if you have a chance, if you could have a go at measuring some 70Ds the results would be interesting to read.

Phil.

Derek said:

Roger

A fascinating and valuable article, certainly seems to be the best on the net.

I do however want to raise one point that impacts the conclusion, slightly.

I work in electronics R&D, and amongst other things have to deal with measuring and interpreting the results from multiple samples of the chips we develop. Having been caught out by quoting a certain spread of expected results only to be presented by a wider spread in production I wish to share the culmination of this experience.

“Any measurement of sigma has it’s own sigma, and it’s surprisingly large”

If I measure only 3 samples, I might get a sigma of say 10.

If I measure another 3 samples I could get a sigma of 75.

really.. THAT different, it’s all down to sample size. (the average also moves about)

I finally found that in order to be sure of my measurements I needed to measure 30 devices, calculate the sigma and average, then work out the +/-4 sigma limits, and quote those in documentation as +/-3 sigma limits. Even then I could only be 95% certain that my quoted 3 sigma limits were in fact not set outside the true 3 sigma limits. If I just calculated +/-3 sigma and quoted them as such I’d have a 50% chance for each limit that it would turn out to be worse than quoted.

So what sort of errors are we talking about with sample size?

with 10 items I got an “uncertainty-sigma” or a sigma of my sigma, of 27% of the measurement sigma.

with 30 items this reduced to 12% of measured sigma.

What does this mean?

well in your AF tests.. you had 10 data points per test.

that means where you had a measured sigma of say 30, you would have an uncertainly sigma in that reading of about 8. If you want to be 95% certain that another camera has a better autofocus, then you need at least 2 sigma gap to that other measurement.. i.e. you need a gap of around 16.

Now there is a gap of roughly this amount for the 5DIII and 1Dx vs the rest (with the exception of the 1DIV)

However the other cameras are all sufficiently closely spaced that they could all be producing the exact same accuracy, or even that the 7D could be outperforming the 60D in reality. In order to split that difference you really need something like 30 data points or more.

I just thought you should have some quantifiable way of determining your measurement accuracy and therefore confidence in your results.. I’m sure this AF accuracy question will reappear with the 7DII.

Please take this as constructive criticism of an excellent article. Many many thanks for all your hard work.

Derek

Leave a Reply