LensRentals.com

Autofocus Reality Part 3B: Canon Cameras

Posted by

Some days it's good to be a geeky gear-head. This is one of them because a) I finished testing autofocus on all of the Canon camera bodies we had, b) I actually found out some interesting stuff, and c) I got worked up about camera marketers while doing it, so I have my next post in the works already.

Because I've been writing this series on the fly (telling you what we found as we found it), I'll keep going in that fashion and keep the conclusions for the end of the article.

What We've Found So Far

In the first Autofocus Reality article, we demonstrated two things:

1. Phase-detection autofocus (even using still targets and center-point only) wasn't nearly as accurate as contrast detection.

2. The contrast-detection autofocus was about as accurate as the most careful manual focusing.

Part two of the series showed that a few newer lenses did focus as accurately as contrast detection on 5D Mark III cameras but not on 5D Mark II cameras. The third article (part 3A) showed that the newest Canon lenses (40mm f/2.8, 24mm f/2.8 IS, 28mm f/2.8 IS, 70-300mm L IS and 300mm f/2.8 IS II) focus more accurately when mounted to 5D Mark III camera but not on 5D Mark II cameras.

The Canon 70-200 f2.8 IS II, oddly enough, seemed not quite as good as the above lenses, but more accurate than the older ones. Whether this was oddness in my measurements or a real finding, I wasn't sure.

Since we knew that the newest lenses autofocused accurately on 5DIII but not on 5DII cameras, the next step was obviously to compare an accurate AF lens on different camera bodies to see with which ones it was capable of accuracy.

We expected the 1Dx (which has the same AF system as the 5DIII) would be accurate. We weren't sure about the others.

Today's Contestants on The Focus is Right!

We know the Canon 28mm f/2.8 IS II had accurate autofocus on the Canon 5DIII so we chose one copy to be our test lens.

We AF microadjusted each camera to the lens prior to shooting. If the camera did not offer AF microadjustment, we checked the body with the test lens and exchanged it for another copy if the lens back or frontfocused at the test distance.

(Microfocus adjustment makes absolutely no difference in the shot-to-shot variation in AF--it only improves the average value of the group. But I got tired of explaining that to people in the previous articles. It was easier to just do it than to answer another 50 emails.)

We then tested it in our Imatest lab using one copy of each Canon camera we carry. To save you from running amok on the internet, finding out which cameras are how old and have what kind of autofocus, I've listed that information in the table below.

 


Camera Year released Year firmware AF description
1DsIII Dec-07 12/16/09 45 point, 19 cross, 26 assist, f/4 at center, dedicated AF processor
5D II Dec-08 2/28/12 9 point, f/2.8 cross center
50D Dec-08 2/28/12 9 point cross, dual diagonal center cross
7D Aug-09 4/25/11 19 point, all cross, center dual diagonal cross (advanced algorithm), first "zone AF" and "spot AF"
1DIV Dec-09 3/29/12 45 point, 39 cross (f/2.8-f/5.6, f4 at center), first AF expansion camera
60D Aug-10 6/19/12 9 point, f/2.8 cross center
T3i Jan-11 1/30/12 9 point, f/2.8 cross center
1Dx Mar-12 2012 61 point reticular, 41 cross type, 5 dual diagonal cross, including center
5D III Mar-12 2012 61 point reticular, 41 cross type, 5 dual diagonal cross, including center
T4i Apr-12 2012 9 point, f/2.8 dual cross center, hybrid CMOS AF Live View

 

I suspected that the autofocus improvement we've seen had more to do with hardware than firmware, but I listed both year of release and year of latest firmware upgrade for completeness. I also listed the basics of the camera's AF system, as well as any marketing comments made about it at release like the 7D having "the most advanced AF algorithms."

My thought going in was the difference would be in hardware not firmware. I don't think any amount of firmware is going to make the AF sensor on the lower left behave like the one on the lower right, even in center-point, single-shot mode.

 

The Results

In the lens variation article, we used the standard deviation (SD) to measure how accurately the camera focused: Softer focus results in lower Imatest values. As an example, we've put up a graph of Imatest values for the 5D Mk II versus the 5D Mk III below.

 As you can see, the 5D Mk III shots (red square) are all very similar. The 5D Mk II shots (blue diamonds) are more spread out: The variation in shot-to-shot focus is greater.

In this example, the SD of the 5D Mk III samples was 17 lp/ih, while the SD of the 5D Mk II was 38.5. Those are similar to the numbers we've seen over and over---accurate focusing combinations have SDs in the teens while less accurate ones have SDs in the 30s.

Rather than clog up the post with a lot more graphs, I'll list the SDs of the various cameras with the 28mm f/2.8 IS lens in the table below.


Camera SD
1DsIII 29
5D II 38.5
50D 34
7D 41
1DIV 22
60D 34
T3i 41
1Dx 17
5D III 17
T4i 29

 

It's a little confusing. There's a range of variation, of course. But clearly the 5DIII and 1Dx do better than the other cameras, while the 1D Mk IV seems to be a bit between those two and the rest of the pack.

It seems a little clearer to me if we graph the standard deviations and separate the cameras by type (more expensive at the top, less expensive at the bottom.) The oldest cameras in both groups are on the left, while the newest are on the right.

 

A couple of points are worth making.

The graph of the more expensive cameras seems to show a pretty logical progression. The 1Ds III is by far the oldest, having been around since 2007. But it included every possible AF technology of the day, including a separate AF processing chip. The 5D Mk II, even on its release, was known to have "consumer-grade" autofocus.

Despite my well-recognized modesty, I will also point out that when the 5D Mk III was first released, and Canon fanboys were dropping off cliffs right and left, I said "the 5D III is no minor-upgrade camera; it’s an entirely new camera using the old camera’s name". Its autofocus system is certainly not a minor upgrade--it's moved over to the big-boy camera side.

I had hoped the T4i might be more accurate than it was, at least with new lenses. It does seem more accurate than the other consumer / prosumer cameras in phase detection, but it's not nearly as good as the 1Dx or 5D III.

I assume that it's new hybrid LiveView system does not carry over to create phase-detection AF. I will say, in it's defense, that when focusing in LiveView it is obviously faster than and just as accurate as any of the other cameras, including the 5DIII and 1Dx.

So Why Could This Be?

All this autofocus stuff 1) gave me a headache and 2) made me rather curious and uncertain.

I started doing a simple demonstration of what I already knew: Phase-detection AF isn't as accurate as contrast-detection AF. But then I got results that indicated sometimes it is just as accurate as LiveView. But you have to have a certain camera and a certain lens or it doesn't happen.

This didn't make much sense to me.

I would have understood if each generation of newer cameras and lenses got a little better. Or if a new camera or lens was dramatically better. But why a rather sudden change, and why did you need both a new camera and a new lens?

I spent a fair amount of time emailing with Dave Etchells of The Imaging Resource  and SLRGear.com who was kind enough to bounce ideas back and forth with me. His thoughts sent me on the right track for figuring this out.

I started off by trying to find out how long it took for an autofocus improvement to go from idea to released-to-the-public. I found one answer in a Canon patent from 2003 describing combined phase / contrast-detection AF in the camera mated to a lens using a stepper motor.

Which we saw  . . .  oh, yeah. Now. With the Canon T4i / EOS-M sensors and new STM motored lenses.

One thing of interest in this patent application, tucked away in the background section, is the following statement:

. . . though the prior art indicates a method for realizing high speed and high precision of autofocusing at the same time, it does not accompany a lens drive control for realizing this and thus does not adequately realize [. . .] high precision of autofocusing.

In other words, back in 2003, Canon recognized that a high-accuracy AF system in the camera required a more accurate lens drive to yield precise autofocus. You can't have one without the other.

Suddenly the need for both a new camera and a new lens to get accurate autofocus began to make sense. But wait, the patent was talking about using a stepper motor to achieve accurate AF. We did find the 40mm pancake (stepper motor) was more accurate. But we also found that the new 24mm, 28mm f/2.8 IS and 300mm f/2.8 IS II lenses were more accurate. Yet they don't have stepper motors.

Then I read on a bit further in the patent:

In order to achieve this objective, this invention provides a camera system comprising: a first focus detection unit, a second focus detection unit, a stepping motor that drives a focusing lens, . . . or a rotation detector, which detects the rotation . . . of the motor. . . The control circuit performs closed-loop control, based on the output of the rotation detector to control the motor.

The wording of this patent, back in 2003, suggests that closed-loop was not how AF worked at that time. It was largely open loop. The camera took a measurement and told the lens where it should go. Done.

This is backed up by a lot of other information, including statements in Canon's "EF Lens Work III" and quotes made by a lot of people who would have known.

It's clear that later AF systems were closed loop (the camera double checked where the lens had gone), although exactly when and where that change occurred I don't know. But this probably explains the lengthy, and now largely meaningless, debates about whether systems were open or closed-loop: They used to be open. Now some (probably most) are closed.

More to the point, though, is the comment that a rotation detector would be needed in lenses driven by ultrasonic motors to increase accuracy of the focusing movement.

If this is the case, then the newer Canon lenses should definitely have a rotation detector built into them. We know there are rotation detectors in many lenses released after 2000, but if they are  in older lenses we can't identify them, so this fits too. (As an aside, I am particularly skilled in finding them because usually if you touch them with your fingers the lens won't focus anymore and the unit has to be replaced.)

 

From Left: A Canon and Nikon magnetic rotation detector.

 

Just to be certain, I asked Aaron to find an excuse to take apart one of the new lenses. Or failing that, I told Aaron to take one apart and not tell Tyler or Drew, who never really believe we can put them back together correctly.

The result: Aaron checked and the newer lenses did have rotation detectors.

There was one other big hint hiding in plain sight in Canon's resource article on precision cross-type AF sensors. The article states that the farther apart each pair of AF sensors are (each pair compares phase from opposite sides of the lens) the more accurate the sensor. It also states only the 5D Mk III and 1Dx have them. I italicized some of the quote for emphasis:

[...]simply by reading which pixels on each sensor line are being struck by light, the AF system can tell instantly what direction to move the lens in for proper focus, and by how much to move it. . . Canon EOS SLRs with high-precision AF sensors simply move the pairs of sensors much farther apart, and accordingly, the AF information can be more finely broken-down and reacted to.

The 5DIII and 1DX have f/5.6 cross-type sensors (most previous cross-type required f/2.8 or faster lenses). The central AF sensors in these cameras also have diagonal cross sensors. The article continues:

For pros who prefer to use the center AF point, simply manually selecting it [. . . ] gives the user the highest precision AF possible with these cameras (assuming an f/2.8 or faster lens is being used). And, with their unique diagonal cross-type layout, the AF points are much more likely to latch-on to typical horizontal or vertical subject details.

In Summary

As it turns out, my little tests just confirm things that were already published (if obscurely.)

The two newest Canon cameras have more accurate phase-detection sensors than their previous cameras. The newest lenses have more accurate focus movement (or provide more accurate focus movement feedback, or both) that takes advantage of those sensors.

Older cameras don't have accurate enough AF sensors to take advantage of the new lenses' capabilities.  Older lenses can't move their focusing elements with enough accuracy to take advantage of the new cameras' accurate sensors.

It's rather sad (given the amount of other work that's piled up during this little series) that I didn't find this information until after I'd done all this testing, but it's rather a testament to our times. There's so much marketing drivel and useless verbiage thrown about that the marketing noise drowns out the actual useful information the camera makers offer us.

But that's the topic of another blog post in which I will take personal offense at the marketing crap that's been shoveled our way at an ever-increasing rate.

I don't believe we photographers are nearly as stupid as the marketers seem to think we are. In the meantime, while I'm on this rant, I highly recommend Thom Hogan's very funny look at camera icons "A Different Kind of Focus." Thom writes better stuff sitting on the tarmac waiting for takeoff than I do at my desk.

And I guess, given all the marketing fluff, I wouldn't have really believed "better autofocus" if I hadn't seen it with my own tests.

I've heard it before and it wasn't so. Not to mention I doubt seriously we'd have seen the marketers say, "better autofocus...but not with the lenses you already have." So maybe this is worthwhile after all.

Otherwise I might have put my trusty old 85mm f/1.8 on a 5D III and thought "not better at all, they're lying to me again."

 

Roger Cicala (with thanks to Aaron Closz and Dave Etchells)

Lensrentals.com

August 2012

169 Responses to “Autofocus Reality Part 3B: Canon Cameras”

Mike Mullen said:

Roger, thanks for all the info - very useful.

I assume you used Imatest charts as the AF target?
I am curious under what light levels your testing was perfomed? My testing of the 7D and 40D (using various fabrics as AF targets) suggested that center point phase detection AF consistency and accuracy took a dramatic hit as light levels fell below EV 6 or so when testing lenses of f/2.8 or faster. However, this fall-off was not noticed when using an AF point off center (non-HP AF point) or a lens slower than f/2.8. The difference was so dramatic it appears the high precision AF point causes the lens to make an erroneous correction under light levels that are not typically thought of as causing AF issues. I imagine using a test chart with more contrast than typical, real world AF targets could mask this phenomenon. That is why I use fabrics to test AF (because I do not use high contrast test charts as my AF targets under actual shooting conditions).

It would be interesting if you could duplicate these results. I know testing is time consuming but I think this is an area that needs further exploration because photographers often use the center (HP) AF point under low light conditions with apertures f/2.8 and larger and with AF subjects that are not high contrast black and white test images. They do this expecting higher AF precision. If this is not the case, then they would be better off selecting one of the regular precision AF points (surrounding the center AF point).

Thanks again for sharing your results so generously.

LensRentals Employee

Roger Cicala said:

Hi Mike,

We are using Imatest targets with 80% 20% gray target to background variation, lit with diffused halogen lights to keep our exposures at ISO 400, f/2.8 to between 1/250 and 1/800. I think your points are completely valid. Given the time involved, and my desire to limit variables, I just want to do one thing at a time. I see this as a year's worth of effort, at least.

Roger

Zak said:

Roger,
I'm sure that 3rd party lenses will have a disadvantage but it would be interesting to see whether there are models that will provide the same type of feedback to the body. I imagine it'd be difficult for the manufacturer to include such a system-specific thing in a lens. But if they manage to do it anyway, it would be a shame if it went unnoticed.
Regarding the open-loop/closed-loop story: I'd agree that the AF procedure _must_ include a feedback of some form if you want reproducible results with different lenses (i.e. the body must re-check the focus and correct the lens until it's "good" (for whatever "good" means). But with a sensor in the lens and/or a step motor, the feedback contains a lot more information so the autofocus loop will need fewer iterations because the result of a signal given to the lens can be predicted better, and therefore Focus can be achieved quicker.
Only quicker, not more accurately? Well, depends. In control engineering you can always trade one for the other: If you move quickly, you arrive sooner, but might miss the target a bit. If you want to be extra-precise, you need to move slower, or check back more often, and in general apply smaller inputs in between getting feedback. I've seen this in the completely unrelated example of a hydraulic robot arm once. In one extreme it's lightning fast but can't really sit still, in the other it is rock-steady and precise but sloooooow. The difference is made by how the control circuit is designed.
Now if you get more feedback from the lens, you can determine the required dose of lens movement quicker (in the case of step motors you can also execute more precisely), and that gives you an advantage for either speed or accuracy, or a bit of both, whichever the engineer designing the system prefers.

Kai said:

Roger,

Thanks for the effort. Any comments as to why the 7D actually has a larger standard deviation (41) than both the 50D and 60D (34)?
Bashing SD's can be fun, but what about the average resolutions? If the 7D is 1000 +/- 41, I'll take it over a 700 +/- 34 50D any day thank you ;-)

Also, picking some nits: the 50D was not released in December 08, but September or October (mine is from mid October). DPR lists the 50D as announced on August 26, 2008. Likewise, the 5D2 is listed as announced on September 17, 2008.

josh bornstein said:

Very helpful review. One spelling tip (that I used to give to my 3rd and 5th grade students): If you don't know the difference between 'its' and "it's", always fully write out "it is," and that means you NEVER need to use "it's."

Sorry for being so pedantic, but it's the English teacher in me . . . it just hurts to see really basic spelling errors. :-)

Greg said:

Roger, thanks again for some terrific research and write-up. The data is very interesting and the info about rotation detection in the lenses very pertinent. But I think you muffed your reading of Rudy Winston's article and the conclusions you drew from it, though it still provides insights.

Rudy employs an historical approach in explaining cross-type AF points and "High-precision" AF points, not just strictly info about the 5D Mk III and 1D X. He does NOT state that ONLY the 5D Mk III and 1D X have "precision cross-type AF sensors" but that Canon has offered "high-precision" AF sensors since the 1990s. Indeed, we see a cross-type sensor in the EOS 1, marketed 9/1989, and a high-precision sensor in the EOS 1N, marketed 11/1994 (a hybrid cross -- f/5.6 horizontal and f/2.8 vertical). Rudy also says (re: 5D Mk III and 1D X) "These aren't the first EOS cameras to use a diagonal cross-type, high-precision arrangement, but they are the first to offer it with multiple points in the central area." The EOS 40D (marketed 9/2007) introduced a sensor with a dual cross-type center point -- a f/5.6 "+" and f/2.8 "X", like the 5 in the center column of the 5D Mk III and 1D X.

Instead, allow me to speculate on clues in Rudy's article. The traditional "High-precision" (f/2.8) sensors use longer lines, with 2x-3x more sensor pixels, spaced further apart than standard-precision (f/5.6) sensors. While the lens aperture and the size of its cone of light remains a constraint, the AF optics could be improved to focus on smaller, more densely packed sensor pixels.

While this is just speculation, it could also be applied beyond the 5 "High-precision" diagonal cross-type sensors to improving the precision of all 61 points. This would help explain the improvement of the EF 70-300mm f/4-5.6L IS USM which, tested at f/4, would have used the "standard" f/5.6 "+" sensor.

Returning to the AF sensor introduced in the 40D -- according to Canon, it has also been used in the 50D, 60D, and T4i. You tested the last 3 and found 50D and 60D equal but the T4i improved by a value of 5. To me this indicates that even without a new, higher precision sensor design, more modest improvements can be made via the AF algorithms (which are usually coded in firmware). I hope you re-test the 7D with the new firmware -- the test results with the existing firmware are appalling.

So there are my speculations: a) substantial design precision improvements in the new AF sensor; and b) modest improvements via the algorithms (that could be applied to older AF sensors).

I hope someone can pin a Canon person down at Photokina for some real answers!

In investigating all this I also noted: 1) The 5D Mk III and 1D X both have separate dedicated AF processors; and 2) The exposure light metering sensor on the T4i is the image sensor.

Thanks again,
Greg H.

LensRentals Employee

Roger Cicala said:

Thank you, Greg: as always, a well thought out point well made. I always appreciate your contributions and input.

Best,

Roger

Jerry said:

Are there any data or planned tests of 3rd party lenses with the newer EOS's? I'd like to see how the Tamron 18-270 PZD compares when used on the 4Ti.

LensRentals Employee

Roger Cicala said:

Jerry,
Not anytime soon. It's extremely unlikely the third-party lenses could have whatever the new algorithms or hardware that the new Canon lenses do. We may look to see if they're significantly less accurate after we do the Nikon tests. It's my suspicion that third party lenses are more or less accurate on different bodies and we'll look into that. Certainly anecdotal evidence seems to suggest that.

Roger

Mike S said:

Roger,
Thank you for bringing together your experimental design skills, your statistical analysis background, and your long time interest in photography, to define Canon's current AF outcomes - in the context of variational data. It has been a while since I could go through some of your writings, and, on vacation this week returned to your website to see what is new. Wow, the development of results in terms of averages around variation is a great way to define focus outcomes for lenses and cameras, and, your data and analysis are excellent. The reverse engineering of the lens rotation sensor, and publishing it, is probably causing Canon some consternation (although their competitors probably do read, carefully, all of their patents). The patent examination, and correlation to the new 40mm lens design was also outstanding. In addition to your business enabling mere mortals to test lenses we cannot afford, we have access to data that no other site publishes. Nice job. Thanks for starting the analysis with Canon.

greg thurtle said:

I was just going to ask if you were going to test nikon.. and you answered it.
I've been using some software (FoCal) to automatically set my AF muicroadjust values, and i see the variability in the AF, i've wondered why, and you've answered that very well.
I'd be interested if there was a way to select improved precision via a custom function, sort of a firmware double check, without relying on rotation detection, this would open up more accurate AF for older lenses too.
I'm expecting this may be an even bigger issue with older nikon lenses and cameras with high resolution such as the Nikon D800.
I look forward to reading your results and conclusions with eagerness.
Thanks again for the article(s)
Greg

BengtS said:

Having just updated my 7d to firmware version 2.0, it seems to me that AF is now working more consistently than before.

Would it be possible to rerun your tests with the new firmware to verify if that is actually the case?

Thanks for your very thorough analysis.

Denny said:

In the US, the 7D, 60D, T4i has a dual cross center AF point, but in Hong Kong, the 7D, 60D, 650D only has a single cross center AF point. Strange! Why would Canon do that?

Greg said:

@Denny -- That would be exceptionally strange, especially since Canon's Hong Kong website says the 7D, 60D, and 650D have dual cross center AF points:

From http://www.canon.com.hk/ Home > Product Catalog > EOS Digital SLR Camera > EOS Digital > EOS 7D, EOS 60D, and EOS 650D Features:

7D) The newly developed AF System features all cross-type19-point AF sensors which detect both horizontal and vertical lines for better focusing performance. They evenly spread out across the viewfinder, allows easier focus on off-center subject. All 19 points are f/5.6-sensitive for f/5.6 or larger aperture lenses, while the center AF point adds high-precision diagonal cross-type sensitivity for f/2.8 and larger aperture lenses for enhanced accuracy. Inherited from the EOS-1D series, a dedicated microcomputer for AF calculations is featured in EOS 7D, not only achieve high-speed continuous shooting at approximately 8 fps, but also improve AF accuracy and reliability.

60D) The 9-point AF system on the new EOS 60D features f/5.6-sensitive cross-type focusing on all 9 focusing points. The focusing points are widely spread out across the image plane and clearly displayed through the viewfinder, making it easier to focus off-center subject. The center AF point is a hybrid of standard cross-type and special diagonally-shaped "X" cross, with high-precision sensitivity for f/2.8 and larger aperture lenses. EOS 60D also adopts the AI Servo AF II algorithm as found in EOS-1D Mark IV and EOS 7D. It offers enhanced focus tracking accuracy by fast and correct prediction of movements, free from the short-term effect caused by intervening objects. AI Servo AF II also optimize auto focus performance in macro photography, to further enhance the ratio to obtain sharp macro images with auto focus.

650D) The 9-point AF system on the new EOS 650D features f/5.6-sensitive cross-type focusing on all 9 focusing points. The focusing points are widely spread out across the image plane and clearly displayed through the viewfinder, making it easier to focus off-center subject. The center AF point is a hybrid of standard cross-type and special diagonally-shaped "X" cross, with high-precision sensitivity for f/2.8 and larger aperture lenses.

Greg H.

Denny said:

Greg,

Thanks for your update. The previous reply from Canon HK stated these models do not support dual-cross center AF point with the quotes seems to be from the manual. Let me check with them again and let you know the result when I got the reply from them.

Denny said:

The first two written reply from Canon HK stated the models have a f/5.6 "+" cross-type AF and a f/2.8 "+" cross-type AF. But the third verbal reply confirmed they have f/5.6 "+" cross-type AF and a f/2.8 "X" cross-type AF. She mentioned that they are different from the those on 5D3/1DX which features an Offset Array Sensor (with staggering AF point arrangement)

Mattias said:

Roger,
great job finding out that the 5DM3 needs the new types of lenses to show it's full AF capabilities.
That leads me to two questions:
1 - What lenses of this "new type" do exist currently? I understand that the 24-70 2.8 M2 is one of those, but do there exist others today?
2 - What about lenses of older design, but newly maufactured, like e.g. a 50 1.4 or a 24-105 L bought today, are they still of the "old type" or did they have some kind of unpublished "face lift" to fit the 5DM3 AF capabilities?

Regards,
Mattias

LensRentals Employee

Roger Cicala said:

Mattias,

As best I can tell, the 24 and 28mm f/2.8 IS, 40mm f/2.8, and all of the Mk II supertelephoto lenses, as well as the 70-300 IS L are in the 'new lens' category. I'm not certain about the 70-200 f/2.8 IS II. New copies of older designs should not be different than the older copies. They aren't redesigned. But I haven't tested them all to be certain.

Roger

Daniel said:

Thanks, Roger, for a *very* interesting article. Would you expect all new EOS cameras released from now on to have the new AF system? Or would you think the new AF won't "trickle down" to cameras like the "entry level" FF rumored to be coming soon (not to mention future "Rebels")? Thanks a lot for all insights!

Daniel

Rodrigo said:

Roger,

It was very interesting and enlightening to read your posts on this subject. It must have been quite a job to run all those tests.

I disagree with one of your statements however: "I would have understood if each generation of newer cameras and lenses got a little better"

If yopu bear in mind Canon has different camera segments, you can see clearly a trend to a better accuracy in your graphs:
1st segment is the flagship 1D series, each generation gets better.
2nd segment the 5DII / 5DIII and the 7D (5DII and 7D were released with only some months separation and approx geared to the same semi-pro users but for the APS-C/Full frame feature), again a betterment.
3rd segment EOS 50D and 60D, this time approx the same, would be interesting to see the 40D performing
and last, 4th segment wich is the entry level T3i and T4i, showing again a betterment

With the exception of 2nd and 3rd segs which are quite similar, you see a trend that each segment is better than the following which also corresponds to the price tags of the cameras.
Regards,

BozillaNZ said:

The rotation detector in Canon's USM lens is optical, not magnetic. There are laser itched slits in the half circle transparent plastic stripe shown in your photo. If you touch it the grease on your finger gets filled in the slits hence the movement detection fails.

And this mechanism has been in the ring time USM for a long time. I recently took apart the 17-35 2.8L lens, which was introduced in 1996 and it had the exactly same optical detector.

jay said:

Hi Roger, my 70-200 IS II sometimes has the Image stabilizer being louder than usual when it engages and disengages. During the stabilizing process it is quiet. Most of the time it just clicks when IS starts/stops which i know is normal, but sometimes it will give out a louder chirrp when IS starts/stops and it only happens occassionally, more so in Al servo mode when shooting sports. Is there something wrong with my Image stabilizer? Or is it newer bodies like 5 D mark 3 makes IS lenses noisier? thanks for your help :)

LensRentals Employee

Roger Cicala said:

Hi Jay,,
IS noise on the 70-200 II is really variable. We see that variable IS noise all the time. It's just normal. Weird. But normal.

Roger

Daniel said:

May I —now that the 6D is officially announced— ask again: do you expect its seemingly completely new AF-system to be be as precise as the 5D MkIII's with the new lenses? Are you going to test it? Thanks a lot for answering!

Daniel

LensRentals Employee

Roger Cicala said:

Daniel,

We'll test it, of course, but from the specs it seems to use a less robust AF system. On the other hand, we're just looking at center point, still focus, so it's possible it does well there.

Roger

Pete said:

Hi there, I don't really understand all that much about camera testing, but I know a small amount about stats. Hasn't Kai (above - August 6th) made an important point? Isn't standard deviation a measure of variability rather than average performance? Please could you respond to this? Many thanks!

LensRentals Employee

Roger Cicala said:

Hi Pete,

That is exactly correct -- I am trying to measure the variability of the set of shots. 10 contrast detection shots have less variability and thus a lower SD. What I wanted to know is how reproducible is AF, shot to shot. How reproducible is it.

Others have instead looked at the frequency of absolutely missed shots, but that becomes something of a judgement call: what is missed? How far off must you be? Imatest gives us accurate numbers on the focus, but is too time consuming to do series of hundreds to determine # of missed shots.

The mean of the group of shots would differ a bit because of the increased variability of phase detection, but there is a greater difference in mean depending upon how well or poorly the camera is microfocus adjusted.

Pete said:

Thanks ever so much for your answer; as I understand you, in the graph comparing the 5D ii and 5D iii, the mark iii has far less spread, which is (naturally) desirable. But the mark iii also seems to exhibit higher values on the axes. I am not sure what these axes represent, but I assume that this is a good thing? I have a 7D which, according to your tests, seems to have a high degree of variability. I've only had this camera (my first DSLR) for a year, but I love it. I was wondering if this is because, although its performance may be varied, focusing tends to be pretty good on average? I understand your point about microfocus and wonder whether I got lucky (as I wouldn't have a clue how to adjust this) - and this being the case, could your results differ from camera to camera? Apologies if my questions seem naive - I am genuinely interested but relatively new to all this...

Rob said:

Very good test, thanks its very usefull for me!
For several year I worked with a workaround, use AI Servo even for one shot focus.
Program one button for autofocus, use it to focus then take your finger of the button, a voila, a perfect in focus picture.

I still use it with my 5d mark 1 and my eos 7d!

Rob said:

Very good test, thanks its very usefull for me!
For several year I worked with a workaround, use AI Servo even for one shot focus.
Program one button for autofocus, use it to focus then take your finger of the button, a voila, a perfect in focus picture.

I still use it with my 5d mark 1 and my eos 7d!

Pete said:

Hi Roger, in conducting further research in an attempt to answer to my earlier question, I came across your take on the autofocus capability of the 7D from your own (current) website: 'Initially I was nervous over the whole “Canon introduces a new autofocus system” idea, but a very short time with the camera eased that. My impression is the autofocus is clearly improved — better than anything this side of a 1D body that I’ve experienced'. Is this comment simply out of date?

LensRentals Employee

Roger Cicala said:

Hi Pete,

No, but remember we're only looking at center point AF on still shots. There's a lot more variables going on with every day use.

Jan I said:

Hi Roger,

I really enjoyed this article when you first published it, and it still is one of my favourite AF performance articles out there.

Have you had a chance to test the performance of other STM lenses than the 40mm pancake?

Paolo Smeraldi said:

Hi Roger,

I really enjoyed these posts.
I'm a Panasonic GH2 user and recently I have shot some dance/sports events in low light, using manual FD canon lens.
I usually use the old trust canon fd 85 mm 1,8 or the 135 f 2,8.
I use these lenses because panasonic's own lenses are darker, but I have problems focusing when the lenses are wide open because of the shallow depth of field.
I do use this camera at 1600-3200 isos, 1/250 - 1/500, and usually f4.
Now as many micro 4/3 users I wonder if using a full frame camera I would be better off (less noise).

What I would like to ask you, after reading this article, is how considering the crop factor my real life picture taken under such settings would compare with a canon 5d mark 2 picture taken with a 170-260 mm lens.
Would the 5d mark 2 be able to focus at such a long range, considering that the depth of field is much shallower?

If I calculate depth of field with the canon fd 135 mm at f/4 at 10 meters, I get 0,65 meters. To get the same DOF on a canon 5d mark 2 with a 270 lens, I must step it down to f/8.
At this point, do I really have an advantage with the bigger sensor in terms of noise?
On the contrary, if for want of a brighter image I would set the aperture at f/4, would the canon camera get sharp images?

Thanks

Paolo

LensRentals Employee

Roger Cicala said:

Paolo,

That's a complex situation with a lot of answers, but rather than write a book I'll simplify.

The 5D2 would certainly give you less noise at ISO 1600 and up. It would have no problem autofocusing at the distance described, especially with an f/2.8 lens. BUT the phase detection AF on the Canon is not going to be as accurate as the contrast detection on your current camera. This would be compounded somewhat by the shallower depth of field. I would add there is some variation in focus accuracy depending on the lens used - the two that you mention are older designs and less accurate than, say, the 70-200 f/2.8 IS II lens would be.

A 5D III may be another matter. It's autofocus is more accurate, particularly in low light, and particularly with a newer lens like the 70-200. While I use my m4/3 set ups more and more, if I was shooting what you are describing I would absolutely grab a 5D III and a 70-200 2.8 IS II.

Roger

Paolo Smeraldi said:

Thanks Roger;

If I got the 5dmark III and the 70-200 lens, I would not get the same magnification as the one I can get with the canon fd 135 on the panasonic gh2; considering the 2x crop factor, on the canon I would need a 270mm lens, something like the canon 70-300 f4-5,6.
At this point, if I set the panasonic at f/4, I have to set the canon at f/8 to get the same DOF.
In this case, the canon loses exactly one stop; I think that the sensor is much better on the canon, so it will probably make up for the difference.
It the canon autofocus is good enough and I do not need depth of field, I can even open up the lens and it gets even better for canon.
I think that if panasonic could make a sensor with a better signal to noise ratio for the gh3 it would have made a killer camera.
I hoped that they could do as well as fuji.
I very much like the panasonic as an almost ideal camcorder and camera combination.
I think that mirrorless is the future, but probably with bigger or better sensors.
Thanks Roger, you blog is very nice and your articles interesting and well written.

Paolo

Dave Seeley said:

Thanks Roger!!!

Do you know if the 70-200mm f2.8 IS II was NOT optimized to be (future) compatible with the latest autofocus? - and that is why is performs at a lesser standard than the newest lenses? I was contemplating selling my IS v1, and upgrading to an IS II, but if the answer to my question is that is was NOT optimized, then I'd think that Canon might introduce a vIII sometime soon..... and I'd wait.

Dave

LensRentals Employee

Roger Cicala said:

HI Dave,

I'm working on the next AF article, but I've gotten educated. It appears there's not a yes-no answer but rather an increasing series of capabilities built into lenses and cameras. But with this in mind, the 70-200 is fairly optimized, although perhaps not quite as accurate as the newer primes (but then neither is the 24-70 Mk II). More in a week or so.

Roger

Martin said:

Wow! I foumd your findings just amazing. Thatt is one of the best or the the best analysis seen in internet. Unfortunately i have some serious issues with my 5d3 AF. It was serviced several tmes and calibrated but 135
L is not working properly on longer distance. Also I have tested 8 samples of 50 1.4 and found that it can't focus properly on short distance. All samples were backfocusing on short distance up to 1,5 meter. Have no idea if thats a camera problem or lens itself. After my experience with Canon i am considering a d800 as i did not have eny issues with other nikon bodies. I would be so grateful if you test d800 vs 5d3 in AF area. I am really looking forward to seeing such an artcle. Thank you once more for your great work and effort. Priceless.

LensRentals Employee

Roger Cicala said:

Martin,

It's been done, although not directly. The Nikon AF article shows the D800 and D4 are better than previous Nikon cameras, but not quite as good as the 5D III.

There will be another article in a bit, BTW, that mentions the lens part in this equation. The Canon 50mm f/1.4 is the least accurate autofocusing lens in the lineup, partly because it has so few 'steps' in it's movements.

Roger

Martin said:

Thanks for reply Roger. I am not sure I understand you correctly regarding D800, however I could not find any articles about D800 AF in the blog.

If I understood you properly it means that 50's motor does not move smoothly but in some kind of steps which are more "distanced" compared to more sophisticated lens.
I have heard a lot about 50 1.4 AF inaccuracy, however I have no idea why there is such a problem in my camera with that lens or... it is not?. I checked a few new copies, and found the same behaviour. On distance up to ~1 meter, all the copies (one was adjusted with service) were back focusing @ 1.4 and when I set aperture @4 the back focus was even bigger. I never heard about shift focus in that lens from many user so I was pretty sure that there is a problem with my camera (Canon service exchanged the mirror box after purchase as there was a significant misalignment of AF sensor so I though something went wrong). The 50 1.4 spec. shows that that lens focus from 0,45m but all tested copies did not focus correctly up to specific distance. I never heard from any users or review about this issue in 50 1.4 so I wonder if that is a general behaviour or camera specific problem.

I have also an idea for other test. I am wondering what are differences in camera AF precision/tendency when focusing in different light situations (day vs incandescent). just in case you have some free time...

Once more-great work!

Florian Uhlemann said:

Hello Roger,

I'm very, very amazed by the work you do. I have completed a statistics class in college and that type of stuff is so interesting. So I understand what your trying to say.

The more interesting thing is that certain lenses now have the rotation detectors or STM.

Now what I am really curious is If the new 35mm f2.0 IS follows the same principle as the 28 1.8 IS, and if the 24-70 mkii is also better at focusing with 5d3 1dx. But I think you mentioned above that it won't be.

Keep up this excellent work as you are one of the few that does it passionately. :)

Thanks
Florian

John Leslie said:

Roger - I wonder if you have read Reikan's work on Contrast AF, which seems to disagree with your conclusion on how good contrast AF is? I'm using his software to try auto-calibrating my lenses on a 5DmkII, although I haven't come to a conclusion on how much I like it yet - have you tried it?

Anyway, here's the link:
http://www.reikan.co.uk/focalweb/index.php/2012/12/af-consistency-comparison-nikon-canon-phase-detect-contrast-detect/

Cheers
John

LensRentals Employee

Roger Cicala said:

Hi John,

I did, and we've corresponded a bit about it. Plus I've spent about, oh, way too much time educating myself on autofocus these last 2 months (which is why the posts stopped). One thing I really learned is I had underestimated the complexity (and so has Rich). Doing things the way I was doing I realized (seriously) I'd need many thousand experimental runs to figure out what I want to know.

As to contrast being less accurate, I've learned contrast accuracy deteriorates as ISO goes up and lighting goes down much more than phase. So we found contrast very, very accurate -- but we were in the lab with 500 watt halogen lights and ISO 200 or 400. I'm not sure about their testing.

The other part is comparing two very different lenses - I have no disagreement with what they saw within Canon or within Nikon, but people are taking the results and comparing the brands. Inappropriate when using a Macro lens at long focusing distances and comparing to a standard range lens meant to work at longer focusing distances.

Anyway, Reikan's data will be a welcome addition. Obviously I'm not going to figure this stuff out by myself and although a couple of people have volunteered some information, most who know stuff are under nondisclosure agreements not to reveal stuff.

Roger

Panurus said:

Have you a list of the lens with rotation detector?

David said:

Really appreciate this thank you. I'm about to replace our 5Dmk2s with 5DMk3s in our photography studio (for Tv lenses we still use humans ;)
I'm personally interested in focus speed because I want to upgrade from Sony's new Nex6 with "hybrid autofocus" to a Nikon D5200 or Canon T4i in the hope that a cheap camera and decent lens will give quicker AF.. Do you have a similar test including Nikon cameras? I'm just not sure what to get in this instance. Thank you!

LensRentals Employee

Roger Cicala said:

David, part IV has Nikon cameras. I think the bottom line is the Canon 5D III and 1D4 have the most accurate AF, but other than that I don't think there's a difference between Canon and Nikon. And, of course, that's just still AF, I don't have any information at all about servo type AF.

David said:

Thanks!

Todd Shaner said:

I recently purchased a Canon 8-15mm F4L fisheye lens and 85mm F1.8 lens for use with my 5D MKII body. I spent quite a bit of time adjusting the autofocus microadjustment with both lenses using various techniques, followed by actual in-field tests for real-world focusing accuracy. What I discovered is that both of these lenses achieve better SD by first manually focusing the lens to a slightly closer focus setting, and then letting the camera autofocus. This is also the case with my Canon 17-40mm F4L lens across the zoom range, which I just received from the Canon Repair depot after an extensive rebuild.

SD results with these lens manually set past infinity before autofocus showed far more inconsistency, sometimes front focusing, and then sometimes and back focusing. The amount of microadjustment required using this “closer” pre-autofocus manual adjustment is much less as follows:

8-15mm F4L -2
85mm F1.8 +4
17-40mm F4L +3

Using this procedure I can achieve very good accuracy and repeatability with the 85mm @ F1.8, which has a very small depth of field. There are some misses for sure, but it’s general due to the phase detection system failing to properly detect and lock on the subject area I’m focusing on due low contrast area and/or edge definition.

I thought I would pass this on to get your thoughts and see if this is something you’ve run across during your testing.

Wilba said:

Since I left a message in August 2012 clarifying that the Ishikawa patent is talking about the control loop _inside_the_lens_ (not the control loop inside the camera), I've tested the EOS 650 (the first EOS, from 1987), and found that it behaves exactly the same as contemporary cameras. So it's simply false that "They used to be open." EOS has always been a closed-loop AF system, as are Nikon's, Pentax's, and Sony's similar systems. See http://www.dpreview.com/articles/5402438893/busted-the-myth-of-open-loop-phase-detection-autofocus.

Leave a Reply