Canon 24-70 Mk II Variation

A while back we ran Imatest resolution numbers on a few copies of the Canon 24-70 Mk II. That was right at release time and the resolution was most impressive, but we only had 5 copies available to test. Over the next couple of months a number of reviews came out. Most seemed to consider the resolution stellar, but a few found it average and several people seemed to have bad copies.

Of course all of this made me interested in checking the amount of copy-to-copy variation in this lens. We finally got caught up enough to run a reasonable sample size through Imatest evaluation, so I thought I’d share the amount of variation we’re seeing. Since most of the complaints I’ve seen online were at the 70mm end of the zoom we tested there first.

 

 

A word before we begin: if you aren’t familiar with the degree of resolution variation in zoom lenses don’t read any further. It will make you go insane and make a fool of yourself online. If you want some background, I would impartially and modestly suggest reading this first.

Resolution Variation

If you aren’t familiar with our Imatest results, the graph measures MTF 50 at the center (horizontal axis) and the average over the entire lens (vertical axis) reported in line pairs / image height. Each dot represents the best image of a given copy of the lens from a bracketed set.

The first graph shows the results of 70 copies of the 24-70 f/2.8 Mk II at 70mm (blue dots) and a set of 125 copies of the Canon 24-70 original version (red dots) we tested a couple of months ago. (Note: the axes do not go to zero to give a more expanded view of the points.)

 

 

A look at the graph shows a couple of things. Overall, the range of variation of the Mk II lens is about the same as the Mk I version. (And these are recent Mk Is after we learned how to keep the 70mm variation to a minimum. It used to be much greater.)

You can also see that three of the tested Mk II lenses seem to be out-of-sorts.  Even with 70 copies, though, it’s hard to be absolutely certain where the ‘acceptable’ cutoff should be by just looking at the graph.

For that reason we use SQF numbers to help determine where the acceptable cut-off should be. (If you don’t want to read the entire link, SQF basically tells us how big of a difference in resolution would be apparent in an 8 X 10 print.) Basically we read the SQF on the highest few lenses, and then put a cutoff where the lenses become 5 SQF units below the worst lenses. An SQF difference of 5 should be barely noticeable in an 8 X 10, high quality print.

SQF suggests that the lowest 3 copies of the 24-70 Mk II are noticeably less sharp than the best. In addition, the two lenses above those lowest three are right on the SQF 5 difference.

The bottom line is that 3 lenses out of the 70 copies were not up to expectations. That is a bit higher than the 2% unacceptable rate we usually see ‘out of the box’.

One other thing demonstrates the difficulty we have with one of the less-than-expected copies; they’re still better than the best of the 24-70 Mk I versions. They just aren’t as much better as they should be.

Comparing to the 70-200 f/2.8 IS II at 70

When I tested that first small batch of 24-70 Mk II lenses I said they were a bit sharper at 70mm than even the legendary Canon 70-200 f/2.8 IS II lens was. The 24-70 resolved 950 Line pairs / Image height at the center, and 810 overall, compared to the 70-200’s 885 / 765.

Of course, I wanted to repeat that comparison now that we have a reasonably large sample size (the original group was just 5 lenses). The average (mean) resolution of the70 samples of the 24-70 II is a bit lower at 940 / 800.

Since then, I can’t tell you the number of people who’ve commented, “but Bill tested a 24-70 against a 70-200 and the 70-200 was sharper.”  Yeah, I’m not surprised. Showing you the actual data makes it pretty obvious why I’m not surprised.

In the graph below the same 24-70 data points are compared to a run of 85 copies of the 70-200 f/2.8 IS II (red crosses) shot at 70mm.

 

 

If you pick one of our 24-70s and one of our 70-200s at random, there’s almost a 40% chance the 70-200 will have the same, or better, resolution.

I think most people realize there is sample variation. But this provides a nice illustration showing just how random a test report comparing just one copy of each lens can be. Don’t get me wrong; those reports are totally worthwhile.

But we have to be really careful splitting hairs with camera lenses. With 70 copies tested, I’m comfortable the average (mean) resolution of the 24-70 Mk II is slightly higher than the 70-200 f/2.8 IS II at 70mm. But that difference is much less than the sample variation

Addendum

I think it might help to demonstrate how big number differences above are fairly small image differences in the real world. Below are 100% crops of one of the highest rated ten copies of 24-70 IIs and one of the lowest 10 (not counting the bottom 3 that I called not OK). These are 100% crops of an ISO 12233 chart, something a pixel peeper could shoot at home. If you look really carefully I think you can barely tell the best lens. But it’s close.

 

 

For the record, there is a 110 point center and 80 point average difference in the two lenses, so they’re near the extremes of difference. You can see it if you pixel peep like this at 100% in side to side comparisons. But I think you’d agree in real photographs it will probably be impossible to see the difference, as SQF suggests.

BTW – if you feel the need to ask which was the highest and which was the lowest, then I’ve made my point.

 

Roger Cicala

Lensrentals.com

November 2012