fbpx
Resolution Tests

Canon Wide-Angle Zoom Comparison

Published July 30, 2014

 

Left to right: Canon 16-35 f/2.8 II, 16-35 f/4 IS, 17-40 f/4. Can you spot the one with the wrong hood? The intern obviously couldn’t. 

 

As is so often the case, I bit off more than I wanted to chew when I came back from vacation. The Canon 16-35mm f/4 IS lens had just been released, a few copies were in stock, and I thought I’d do a nice quick test. But one of the reasons I’d wanted an optical bench was because I don’t trust Imatest results with wide-angle lenses. At 16mm, even with the very largest test charts, we’re testing at about 4 feet shooting distance.

So I after I did our standard Imatest on the 16-35 f/4 IS, I wanted to repeat the results on our optical bench. Of course, I don’t have a big database of optical bench results to compare against like I do with Imatest. So I had to do optical bench tests on some other wide zooms for comparison purposes. Then I had to do some more comparison with other lenses to see if the variations we were seeing on the optical bench were simply a new, higher resolution testing method, or if they were telling me something about variation with wide-angle zoom lenses. (Both things were true.) Anyway, the testing I thought would take a week has taken three.

I realize some of you just want to see the usual Imatest results on a group of these lenses since that’s what you’re used to seeing. Others are also interested in the optical bench results showing how the lenses resolve at infinity, rather than just close up. And of course there are a few of you who want all the gory details of Geekiness that the optical bench reveals. So I’ll try to present this in three parts: the Imatest results first, the optical bench optical test results second, and the geeky stuff third. It’s a buffet; just grab what appeals to you.

Imatest Results

We tested 10 copies each of the Canon 16-35 f/4 IS, 17-40 f/4, and 16-35 f/2.8 Mk II in our Imatest lab. The table below presents the average values of those copies. Center is the MTF50 at center of the lens. Average is the average MTF50 taken at 32 points across the surface of the lens. Corner average is the average of eight measurements, two (one vertical and one horizontal) in each corner.

Those of you who read a lot of our articles might notice the numbers given for the 17-40 f/4 L and 16-35 f/2.8 L lens are slightly higher than those we’ve presented in some older articles. This is because we now use larger, higher resolution test charts and different lighting. (And a good example of why comparing Imatest measurements between testers is an iffy thing to do.)

We tested the 16-35 f/2.8 Mk II at both f/2.8 and f/4 so it would have a level playing field. You can sort the table by clicking on the top row of whatever column you’re interested in.

  Center Average Average Corner
17-40 f/4 L 17mm1240875590
16-35 f/2.8 L 16mm at f/2.8890730440
16-35 f/2.8 L 16mm at f/41360970615
16-35 f/4 IS L 16mm14451185795
17-40 f/4 L 40mm1150810460
16-35 f/2.8 L 35mm at f/2.8995820460
16-35 f/2.8 L 35mm at f/413151095795
16-35 f/4 IS L 35mm13601190895

The conclusion is pretty simple: the new 16-35 f/4 IS has the best resolution. At 35mm, the 16-35 f/2.8 shot at f/4 is nearly as good, particularly in the center. The new f/4 IS is slightly better in the center, but it’s close enough that copy variation would have some overlap. The difference is a bit larger away from center. The 17-40, at the long end is clearly not quite up to the other two. (For completeness, the 17-40 f/4 is clearly better than the 16-35mm shot at f/2.8, but that’s a pretty unfair comparison.)

The difference is more pronounced at the wide end, where the 16-35 f/4 IS is clearly better than either of the other two lenses. As most of us are aware, the 17-40 does better at the wide end than the telephoto end, but it still can’t keep up with the 16-35 f/4 IS.

I know a lot of you want to see f/8 results, too, and we’ll try to do those in the near future, but for this test I decided I wanted more copies at each aperture rather than fewer copies at more apertures.

Semi-Conclusion

Obviously there’s a lot more to choosing a wide-angle lens than simply MTF50. Flare resistance is critical, off-axis aberrations are very important, etc. But from a purely resolution standpoint, the 16-35 f/4 IS is definitely better than either of the other two Canon wide-angle zooms.

There’s a value difference, too. The 17-40, at $839, is the least expensive of the bunch and is a pretty good lens. Personally, for $250 more, I find the 16-35 f/4 IS the better value both for its resolution and for the IS system, which I love on a wide lens. Many of you never use IS on a wide-angle zoom, though, so you may find the 17-40 a less expensive option. The 16-35 f/2.8 II lens has had a price drop to $1,560 with rebates. If you need f/2.8, you need f/2.8 and that’s your choice. You still benefit, though, because Canon dropped its price after releasing the f/4 IS lens.

I’ll add a bit of editorializing. I think Canon priced this new lens aggressively and I appreciate that. If it had come out at $1,500 it would still have sold. It’s that good.

This ends the quick summary portion of our program. If you just wanted to see the Imatest test results and get my opinion, you may go now. The rest of this post gets into the geeky MTF part of things. If you’re into that, I think you’ll enjoy it, and I’ve left you a couple of fun comparisons at the end. None of the information below is going to change the summary above, but it may add some interesting detail.

Those of you who are forging ahead, let me also warn you that the cool graphics we discussed in the graphics contest post aren’t quite ready yet. There’s way more work involved in that than I realized and it’s going to take some time to implement. So consider this MTF data presentation V1.1.

MTF Bench Results

Basic Methods

Imatest gives us MTF50 results, which are somewhat a reflection of how well the lens resolves.  Imatest is also measured at fairly short focusing distances for a wide-angle zoom: a little over 4 feet for 16mm and just under 9 feet at 35mm using our largest charts. The optical bench tests at infinity focusing distance and gives us some additional information that isn’t easily available with Imatest.

One isn’t better than the other. They give complimentary data.

For bench testing we measured 9 copies of each lens. Each lens was tested 4 times, at 0, 45, 90, and 135 degrees of rotation. We use the distance scale mark on the lens as the reference for “0” degrees, so if you were facing the lens, the measurements would be made like the diagram below. If we were measuring on a camera, rather than the optical bench, that would give us top-to-bottom, side-to-side, and two roughly corner-to-corner measurements.

At each position we measure 20mm on either side of the center point (the absolute corner is about 21.5mm from the center, the horizontal sides are 18mm) in 2mm increments. I won’t bore all of you with how we handle the measurement data, but for both of those who want to know I’ve included it in an appendix.

MTF Charts

Just for whatever reasons, I should mention actual focal lengths. The 17-40mm measures 17.5mm to 39mm, while both the 16-35s measure 16.75 to 34mm, so there were no focal-length surprises with this group of lenses.

As was suggested after previous posts, we are going to show MTF readings up to 50 line pairs/mm. The graphs below are the average (mean) of all readings at each point.

You don’t need to be an MTF chart wizard to compare the results. Higher MTF is better. Dotted and solid lines of the same color close together is better than when they’re far apart. “0” is the center of the lens and 20mm is the far edge.

Legend for MTF graphs
MTF charts for the Canon 16-35 f/4 L IS at 16mm and 35mm

 

MTF graphs for the Canon 17-40 at 17mm and 40mm

 

MTF graphs for the Canon 16/35 f/2.8 test at f/4. 

 

The MTF bench results are slightly different than Imatest, or perhaps just more complete. At the wide end, the three lenses are very similar right at the center, but once you move away from the center the 16-35 f/4 IS pulls away from the other two. In the outer 1/2 of the image (the 10-20mm distance from center) the 16-35 f/4 IS is clearly better and the further out you go the more difference there is. At the edges of the image (18-20mm) the difference is striking.

At the telephoto end of the zoom the 17-40 f/4 just isn’t as good as the other two lenses, even in the center. In the outer half of the image, the 16-35 f/4 IS is again clearly better than the others.

I’ll rearrange the wide-end graphs a bit to make the comparison easier, since most of us are interested in the wide end of wide-angle zooms.

 

A couple of points are worth mentioning here. The first is that the 17-40 is better at the wide end than it is at 40mm, which most of us who shoot with it have always said. The other, which is also often mentioned, is that the 17-40 f/4 is better at the wide end than the 16-35 f/2.8. That is true if both are shot wide open, but if both are shot at f/4, as we did in this test, there’s not really a significant difference. On the other hand, if you’re going to shoot at f/4, no significant difference means the cheaper lens is a better value.

The second point is that the results are a little different when testing these three lenses with Imatest versus the optical bench. How much of that is testing distance (infinity for the bench, 4 to 9 feet for Imatest) and how much is different methods I can’t say — yet. That will have to wait a few weeks until the close focus adapter for our optical bench is available. For now, I just consider it two different ways to look at the same thing, and more data is always better.

Astigmatism

The MTF graphs above show the astigmatism, of course, but it’s sometimes helpful to look at the astigmatism separately. I’ll limit these graphs to 10, 20, and 30 line pairs/mm —  the 40 and 50 line pair astigmatisms are very similar to the 30 line pairs/mm plot for all of these lenses. Again, these are multiple-copy averages. The astigmatism was very consistent between copies, though, with no outliers that were dramatically worse or better than other copies.

As you would expect, the astigmatism is more pronounced at the wide end and generally worsens the further you go from center. The 17-40 and 16-35 f/2.8 have less astigmatism further out, while the 16-35 f/4 IS seems to develop more astigmatism near the edges. But if you look back up at the MTF graphs above, you’ll see that what’s actually happening is that both the sagittal and tangential readings are dropping rapidly with the two older lenses, while with the new 16-35 the sagittal readings stay higher at the edge and only the tangential fall off. This increases the astigmatism number but gives better resolution.

 

There’s far less astigmatism for all 3 lenses at the long end of the zoom range. I’d call the 17-40 and 16-35 f/2.8 (shot at f/4) very good performers, while the 16-35 f/4 IS is really amazing.

 

Field Curvature

The MTF bench gives us a nice printout of the shape of a lens’s field curvature, which is always of interest with a wide-angle lens. As you can see, the 16-35 f/4 IS and 17-40 f/4 have little field curvature at the wide end. At the telephoto end, the 17-40 f/4 has the most curvature and the 16-35 f/4 IS the least.

One thing to notice, though, is the lens designers really do know their stuff. If you look at even the most curved fields, you’ll see that the edges generally still fall in the sharpest area (lightest red color), although sometimes just barely in that range.

 

I also want to point out that these graphs aren’t average of all the lenses of each type tested; they are examples from a single copy. The software that does these calculations doesn’t allow us to average multiple lenses, but we did test all of them and they were all similar. The most common difference was just a bit of tilt of the field one way or another.

When we first started testing these lenses this surprised me a bit, but apparently this is just the nature of wide-angle zooms. The two most tilted copies we found (out of 30) are shown below. The tilt is obvious on the MTF machine, but again, notice that even with these both edges are still within the sharpest range. That’s why we don’t see this on Imatest — the flat field of an Imatest chart would still be in the sharpest focus area at both sides.

You’d be very unlikely to notice it in a photograph with lenses of this focal length and aperture because the depth of field is usually quite large. But if you took a 35mm image focusing, say, 8 feet away you might notice that on one side the sharp area went from 6 to 8 feet and on the other from 8 to 13 feet.

 

Distortion

All wide-angle zooms are going to have some distortion and these are no different. Distortion measurements differ with different testing methods, so I’ll provide both Imatest measurements and optical bench measurements. Remember, Imatest is measured at about 4 feet for the wide end, about 9 feet at the long end, while optical bench results are at infinity.

  Wide (Imatest) Tele (Imatest)  Wide (Bench) Tele (Bench)
16-35mm f/2.82.65%-1.7%0.6%-0.65%
17-40mm f/42.6%-1.1%0.7-0.6%
16-35mm f/4 IS2.95%-1.05%0.8%-0.75%

There are some differences between the 3 lenses, but not anything significant enough to affect one’s choice of lens, I think.

Sample Variation

When using Imatest, I showed sample variation as a scattergram with each copy represented as a dot placed on a grid showing peak and average MTF50.  We’ve used it for years and it works fairly well, but we’re aware it’s a blunt tool. For example, when we’re testing our own lenses there are a number that pass the ‘peak-average’ resolution test, but are clearly not OK. Usually they have one bad corner or one bad side that while clearly unacceptable, isn’t enough to drop the overall numbers to failing.

This is particularly common with wide-angle zooms, so I was eager to look at the amount of variation we saw on the optical bench.

Within-Copy Variation

First, let’s look at how much each copy varies from one area to another. Remember we test each copy at 4 different angles. For each lens we recorded the difference between the best and worst of the 4 readings at each of 20 locations from edge to edge. The graph below shows the average variance of all readings at each point (sagittal, tangential, at 10, 20, and 30 line pairs /mm).

I should make one point right away. Notice that at “0”, the center, there is a slight bit of variation. In theory, this shouldn’t be, the center should always be the same. In reality, rotating the lens around the center makes a tiny difference in location of the center on the machine’s table; there are slight differences when the same measure is repeated; etc. So this ‘center variation’ really is a good demonstration of the accuracy of the tests.

All three lenses, at both the wide and telephoto end, have roughly the same amount of variation. On average, any of the thirty lenses have one corner that is 10 to 15 MTF points lower than the best corner.

For you ‘best possible copy’ chasers out there, the deviation was pretty low. None of the 30 lenses were less than 9 or more than 20 MTF points difference from best to worst corner.

You Want Some Pictures of That Corner Variation, Don’t You?

I know what you’re thinking. “Roger, take some amazing real life photographs to show us this difference of which you speak!” Well, OK! Here are some amazing real life photographs of a 5-micron pinhole. Let’s face it, no matter what I took a photograph of, you were going to tell me the composition was bad and the lighting wasn’t ideal, either. (Both of which are usually true in my photographs.) So I might as well make them useful. A 5-micron pinhole is about the right size to fill up one pixel on a 5D Mk III (about 6 microns square).

I picked a typical copy (average amount of corner variation) for each lens, put it on OLAF, and took images of the pinhole at a 40 degree angle of view, which is just about in the corner of the lens at 16mm. I should add each corner pinhole image is at best focus for that image, so this isn’t about field curvature or tilt. This is the best possible image the lens can make in that corner.

We’ll start with the 17-40 f/4,

Four corner pinhole images showing typical variation for a 17-40 f/4 lens at 17mm.

then the 16-35 f/2.8,

Four corner pinhole images for a typical Canon 16-35 f/2.8 II at 16mm.

 

and finally the 16-35 f/4 IS.

Four corner pinhole images showing typical variation for a 16-35 f/4 IS lens at 16mm.

Do these pictures mean anything? Well, it does show that what the optical bench sees correlates with reality. For each set of four images above I think you can identify one corner or side that’s a bit better than the other side, just like the optical bench says you could.

But could you actually see a difference in a photograph? Perhaps, but we couldn’t in any of these three lenses, even photographing high-resolution test charts under testing conditions. I’ll add, for what it’s worth, that we use this pinhole view to do most of our optical adjustments and from long experience I wouldn’t try to improve any of these. I’d be far more likely to make them worse than better.

Why then, are some people testing their new 16-35 f/4 IS lenses saying they can see a softer corner or side? Well, a few of them probably have an out-of-spec lens. That happens occasionally with every zoom I’ve ever seen. But I’ll also suggest that some of it has to do with the fact that the 16-35 f/4 IS lens is sharper in the corners than the older lenses.

Scroll back up to the side-by-side MTF charts of the 16-35 f/4 vs. 17-40 and 16-35 f/2.8 II. Look at the edge area (18-20mm from center) and consider the MTF50 line (MTF 0.50 on the graph). For the 17-40 and 16-35 f/2.8 II, only the 10 and 20 lp/mm sagittal lines are at MTF50 or above at 18mm. With the 16-35 f/4 even 40 lp/mm is above the MTF50 line.

When photographing sharpness charts that test resolution (and therefore things like MTF50), you can see a difference in corners much more clearly on the 16-35 f/4 IS. To put it simply, in testing on-camera it’s easy to tell the difference between sharp and not sharp. The difference between not sharp and pretty soft is hard to see. So it’s less likely we would notice this variation in the 17-40 or 16-35 f/2.8 II lens, since they’re already softer in the corners.

Variation Between Copies

Next we’ll look at variation between different copies of the same lens. For this we took the standard deviation of all copies of each lens at every measured point.

First the standard deviation between copies at the wide end.

 

Now the SD between copies at the telephoto end.

I was surprised at how little variation the 16-35 f/4 IS had at the long end. I won’t say I’m surprised by how much variation there was in the 16-35 f/2.8 II. Regular readers of this blog know we have more trouble with the optics on that lens than any other Canon lens. Nevertheless, I wanted to check and recheck these numbers and look at them several ways. Then we even did some optical retesting.

But everything seems to agree – there’s more sample variation in the 16-35 f/2.8 II lenses and less in the 16-35 f/4 IS at 35mm. At the wide end the difference isn’t as striking.

Remember, this isn’t sample variation caused by one or two badly decentered copies; we’d already screened those out using methods we’re pretty comfortable with. This is ‘how much variation do we see between the acceptable copies’. For completeness, though, let’s remember the 16-35 f/2.8 and 17-40 lenses were off the Lensrentals shelves while the 16-35 f/4 IS had been rented once at most. Even if we assume the older two lenses would be better out of the box (I don’t, but I can see where others might) it still is a very nice performance by the 16-35 f/4 IS.

One Last Comparison

I’m completely aware that a lot of people love their 16-35 f/2.8 II and 17-40 f/4 lenses. I’ve never been a huge fan of them, and usually shoot a wide prime lens (Canon 17 TS-E or 14mm f/2.8) on Canon bodies. If I know I’m going to shoot a lot of wide-angle images, I’ll often check out Nikon D800 and 14-24 f/2.8. So for purely selfish reasons I wanted to see how well the new 16-35 f/4 IS compares with these lenses.

In the following graphs the newly introduced lenses are NOT averages of a dozen copies, they’re test results from single copies. I’ll do complete comparisons in the future with multiple copies of them. This was just to give myself a quick estimate as to whether further testing was worthwhile, and I thought I’d share it with you.

Canon 16-35 f/4 IS versus Canon 17mm f/4 TS-E

This is done with both lenses at widest zoom. I don’t know if the results shock you, but they did shock me.  The MTF charts clearly show the TS-E is better at . . . tilting and shifting. That’s huge for me, because I often used it as a straight lens, never intending to use the tilt-shift features, simply because of its resolution. Of course, the TS-E has less distortion than the zoom does, and that can also be important even if you don’t plan to use the tilt-shift features.

 

Canon 16-35 f/4 IS versus Nikon 14-24 f/2.8

I hesitated to put this up, because I know it will start fanboy flame wars. So first and foremost let me state clearly: This compares the Canon at f/4 to the Nikon at f/2.8. (Nikon’s electrically set mechanical aperture is a real pain to set to f/4 for testing off of the camera body. It can be done, but it’s time consuming. It will have to wait until I have time to do a full comparison.)

The Nikon will definitely improve at f/4; probably enough to be as good as, or even slightly better than the 16-35 f/4 IS. But they’ll be hair-splitting close, I expect. So for the people who shoot the Nikon 14-24 f/2.8 on a Canon camera via an adapter, the 16-35 f/4 IS will definitely give them a new option.

Conclusion

I set a high bar for new Canon lenses. I expect them to be excellent and generally their recent releases have been. Since this was a wide-angle zoom, though, my expectations were lowered a bit. Canon has always struggled with wide-angle zooms. The 17-40 is a good, not great lens. The 16-35 f/2.8 II is better than the Mk I replaced, but I’d consider it, at best, adequate considering its price.

The 16-35 f/4 IS changes that. It’s a superb optic — as good as anything else available. Of course, a lot of people want an f/2.8 zoom. But for many, like me, f/4 with IS is just fine for wide-angle shooting.

I have to add that I think the price represents an excellent value, which is not something I’ve said about many Canon lenses lately. The 24-70 f/2.8 Mk II and 70-200 f/2.8 IS II are world-class optics, but I’ve never heard them called bargains. The 16-35 f/4 IS isn’t cheap, but I think it’s an excellent value. If it were the same price as the 16-35 f/2.8 II, I’d still buy one.

 

Roger Cicala and Aaron Closz

Lensrentals.com

July, 2014

Appendix – Data collection

For each lens MTF is measured 4 times, once each at 0, 45, 90, and 135 degrees rotation. At each rotation MTF is measured at 20 points, equivalent to every 2mm across the image surface. At each of those points, 10 measurements are recorded (MTF in sagittal and tangential plane for 10, 20, 30, 40, and 50 line pairs/mm. So a single rotation measurement looks like this:

The average (mean) and standard deviation of the 4 MTF readings at each position and for each line pair frequency is then calculated as a summary table for that copy. The astigmatism (difference between sagittal and tangential readings for each position and frequency) is also calculated for each frequency and position.

A zero variance check of all the original tables (there would be 80 in a 10-copy test) is done to be a reading was not accidently transposed twice.  Additionally, readings are compared to verify some aspects of the testing. For example if all lenses read slightly lower on one side or at one rotation we might have an issue with proper leveling of the rotation table or calibration of the collimator.

Next all of the copies for each lens are summed and compared. The final MTF graphs are the average of all readings at each position, with the positive and negative points also averaged. (In other words, the MTF, standard deviation, and astigmatism readings at position ’20’ on the graph is the average of +20mm and -20mm from the tables.)

Although we don’t show it in these articles, we then check each copy’s averages against the mean of all copies to identify any outliers. In this particular series one lens was removed from testing early in the process (during the Imatest phase) because it was identified in this manner and further testing showed it to be decentered.

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Resolution Tests
  • Roger Cicala

    Francis, the min-max spread between copies is about 1.5 times the within copy spread for the 16-35 f/4 IS, which is really quite good. The 17-40 and 16-35 f/2.8 had about 2 times the within copy spread.

  • Steven

    Roger,

    On my copy of the 16-35 f/4 IS, I have noticed the left side is usually a bit more “tricky” to get as sharp as the right. It looks like it may have to do with the field curvature as you had mentioned in your article, as sometimes it looks just as good as the right, other times less-so. To me, it appears that the left side is sharper up closer than the right side.

    If this is the case, is this something that is corrected with adjusting the centering or tilt? Or is this something that is somewhat “fixed” in the specific elements inside each copy of the lens?

    Great article! Really enjoyed reading through.

  • derek

    as usual you are the best. extremely thorough and precise, it always helps us nothing really comes close to your level.

    Thanks Roger.

  • Simon

    Hi Roger,

    thanks for your detailed answer about sensor stack. My thought are somehow not directly related to this post but I still find hard to understand that some big manufacturer offer the same lens in several different mounts and therefore dedicated to different sensor stack thickness. For example there are the very expensive Zeiss compact prime range that are offered for m43 mount as well as other other mount with thinner sensor stack. Can we really expect very suboptimal sharpness or even poor performances with these lenses depending on the mount there are used? If I pay between 4000 to 25000 $ for these lenses, I expect them to be very sharp even wide open.
    I would love to know sour thoughts on this point.

  • François

    Roger,

    Thanks a lot for this detailed review which reinforce my interest for this new Canon lens.
    As you were mentioning your tests were made with 10 copies of each, have you noticed gaps in performance between the copies ? what would be the “min-max” for each ?

    Regards
    François

  • Roger Cicala

    Lynn, they can’t really tell us much about dynamic range, that’s a system, rather than just a lens, thing.

  • Roger Cicala

    Simon,

    Since that article we’ve set a protocol of first testing all lenses with a variety of sensor stacks to see if there is a difference, then using the thickness that gave best results for the test. You are correct in your thinking: with these lenses we found that 2mm and 0mm were identical, but 4mm did cause a deterioration.

    Roger

  • KimH

    Roger,

    I was wondering if Vacation was the reason for not having seen the 16-35 on LR. Believe me, I was looking almost daily.

    Now I know why :o)

    This must be your (the) best writeup on Lenses I have seen to date – very well done and full of yummy information – I hope you are able to share F8 in the not too distant future. Not that it matters too much, I puchased the lens on the day it got announced based on the Computed MTF.

    I find the lens MUCH better than my 17-40. The addition of IS – as you suggest – makes a difference as well. Especially when i don’t have a tripod-option for inside photo, the IS makes it possible to shoot without leaning on pillar/pole/rail.

    Thanks!!

  • Jeff Robinson

    Thanks for the excellent testing and write up Roger. I trust your multi-copy tests more than anyone else I have found, and have been eagerly looking forward to your test of the 16-35mm f/4 IS.

    I have tried multiple copies of both the 17-40 and the 16-35 f/2.8 II in the past, and have been disappointed enough by both to stick with an ancient Sigma 15-30mm I own. The Canons were clearly sharper in the center, but were notably worse about 1/2 to 2/3 of the way out from the center–probably corresponding to that dip I see in your test charts 10-13mm from center.

    Based on your encouraging test results, it looks like I’ll be renting one of these 16-35mm f/4 IS lenses from you in the future to see if Canon has finally made a wide-angle zoom I’m willing to pay for.

    Keep up the testing and writing!

  • Simon

    Hi there,

    these are very interesting results and your analysis is even more interesting as it balances the results very well. Thank you so much for that. I am really disappointed to be located in Europe and not be able to rent material from you.
    Roger, I have just one question for you: Am I right in saying that the results for the optical bench are measured without sensor stack (or any correction for these)? If I understand well your older posts on this topic, for f-number at f/2.8 and smaller (f/4), the results with and without sensor stack should be very similar even with ultra-wide angle lenses. The thickness of the sensor stack becomes really critical for lenses faster than f2.8 though? Am I more or less correct in my thinking?

    Thanks

  • > “Others are also interested in the optical bench results showing how the lenses resolve at infinity, rather than just close up.”

    Here’s other reasons for interest:

    * How well should Canon lenses be expected to perform with next-generations Canon sensors that are anticipated to have as least as many pixels as the Sony Exmor?

    * Are these lenses “future proof”?

    * Anticipated IQ on the Sony A7r with high quality adapter, especially compared to the Sigma Art lenses? On the Sony, a more “apples vs. apples” comparison can be made with the optical bench results removing the factor of the lower resolution Canon sensor.

    * Can these tests provide evidence of how much DR they are capable of, with a Sony Exmor-like sensor, and/or MagicLantern’s Dual-ISO? Or does that not apply?

  • Avo

    Love your lens comparison articles – straight to the meat, no nonsense, hard numbers, complete with multiple copy sample variation statistics and a dose of geeky humor.

    I find myself refreshing your blog daily when an important lens like this has been announced.

    One small “selfish-ish” request though – some of us Canon users have purchased the Zeiss 21 f/2.8 Distagon ZE lens since Canon did not have a good UWA before this. It would be nice to see how this legendary lens stack up against the modern Canon.

  • You, sir, absolutely rock.

  • Roger Cicala

    HI Aaron,

    I definitely will but I haven’t gotten that far yet. We’re still working right now overcomeing the bottleneck from machine to database or machine to spreadsheet to database.

  • virgil

    great work!

    I have a question concerning the pentax lens lineup.

    there is some “limited” lenses , who appears to have a special optical design.but i did not see any test who show any kind of “limited” effect.
    so , is it purely marketing?

  • A

    Thank you for another interesting and useful test. I’ve never been particularly smitten by my 17-40 f4/L, so it’s great to know that the 16-35 f4/L IS is better optically – I already wanted it because of the IS, so that it’s sharper as well is a sale as far as I’m concerned!

  • Roger Cicala

    Raoul, I think it’s the combination of everyone’s tests; mine, yours, dozens of others, that let people get a handle on what they should expect when they put down their hard-earned to get a new lens. And that’s a worthwhile purpose for all of us.

  • Aaron

    Glad to hear you’re still progressing with the data presentation after the contents. You wouldn’t by any chance need any help with the database portion of it? Or did you find someone else to take care of the whole thing for you?

  • Roger Cicala

    Mike, this is one of the reasons for the bench – now we can do direct tests on all brands. And you know me – I’ll be back shooting Nikon in another few months 🙂 I’ve already gone from the Canon to Pentax to SR7 since that article.

  • RaoulJ

    Of course, my ‘tests’ (rather ‘Empirical experiments’) have little to be compared with your real tests…
    😉

    Did I say I appreciate your work ?

    Have a good day !
    Raoul

  • RaoulJ

    I’ve done some tests with my Canon zooms on Sony A7R recently.
    Test @24mm f/8.0 on an ‘urban landscape’ at about 40m.

    The Canon EF 24mm f/14L II and Canon EF 24-70mm f/2.8L II come to a draw.
    The 1.4 has more CA. But somehow more sharpness from place to place…
    The Canon EF 16-35mm f/4.0L IS comes close, as a good outsider.

    The Canon EF 24-105mm f/4.0L IS is not as good.
    So are the old Canon FD 24mm f/2.0 and f/2.8 and even the excellent Canon FD 20-35mm f/3.5L.

    Just my 2€c… and that’s on A7R.
    Extreme corners on A7R look less sharp than on 1DX. But, to be fair, when you size the A7R images down to 1DX image size, this is about equal.
    With more details remaining in the center.

    Raoul

  • Roger Cicala

    Hi Raoul,

    Not yet but we will soon.

  • Mike

    Roger, it’s posts like this one that make me mourn anew that you chose Canon over Nikon in your great system search. ;~(

    Hope to see similar tests on Nikon lens choices.

  • RaoulJ

    By the way, the hood of the new Canon EF 16-35 f/4.0L IS is the EW-82.
    The EW-83L is for the Canon EF 24-70mm f/4L IS.
    Some vigneting, probably 😉

    Raoul

  • RaoulJ

    Hi Roger !

    Nice review !
    I’m afraid you’ll have to do the test with the Nikon 14-24 @16mm/f4.0 😉

    Just another question :
    Do you have data that could be compared between the 16-35 f/4.0L IS and the 24-70 f/2.8L II (at comparable FL) ?
    In my experience, the 24-70 is better.

    Have a good day !

    Raoul

  • Ron

    Roger, thanks for taking this into consideration. If it helps any, I find your posts very, very informative and interesting. No doubt this also has had a huge influence on how the LR brand is viewed by many.

    It’s much appreciated that you’re willing to share this information from what obviously is very costly (capital and manpower) to implement.

    Meanwhile other photo bloggers (myself included) don’t hesitate to offer opinions without nearly as much rigor and technical supporting information.

    For me, the benefit of LR as a lens testing source would be as a reliable third party alternative to most manufacturers’ calculated MTFs and a deeper look into performance characteristics, especially since you’re using an actual optical bench rather than ‘just’ Imatest. I see it as a reality check from a source with integrity against which manufacturer marketing materials (hype) can be compared. And as mentioned, some like Voigtlander, don’t even publish sufficient info, leaving photographers under informed.

  • Roger Cicala

    Thank you, Curby and Ron. It is something I think about a lot, I’d really like to write full time. But there’s a big line I draw that kind of says, “I don’t ever want to know that my income depends on telling people to buy stuff”. I don’t trust myself that much 🙂 We’ve thought about it, though, and actually I do expect that in the next year or so I’ll be giving more time to doing these comparisons.

  • Ron

    Thanks for this Roger and Aaron!

    Such a wealth of technical information that is very welcome to see offered.

    It’s also somewhat reassuring to see your results for the 16-35/2.8 II illustrate what I’ve long seen in my images with it, starting back when I got it at initial release. Back then, on an APS-H sensor, there was noticeable quality drop at the image edges at 16mm and field curvature was pretty obvious, too. Great to see the new 16-35 is a better performer, and is very tempting.

    But as a recent full frame convert, I’ve been finding 16-35 less useful than the ~20-45mm equivalent range that lens had on APS-H and have been pondering a 24-70/105 alternative. Considering the new 16-35’s strong showing at both ends, would it be safe to expect it’s also competitive with Canon’s 24-70/105 zooms at 24mm? It would be a tempting lens for just the price alone, compared to the 24-70s.

    Canon’s relatively lackluster wide angle lens performance (zooms and primes) was a factor that got me interested in Leica M for the excellent glass options available from Leica, Zeiss ZM and Voigtlander. In this respect, with lenses such as the Leica 21/3.4, I haven’t been disappointed.

    Along the lines of your 50mm comparison, I’d be extremely interested to see something similar for FF UWA lenses (up to 21mm), especially between Canon, Leica and Zeiss (both ZE and ZM).

    Perhaps an even more niche request would be a Voigtlander lens MTF resource. Maybe I haven’t looked hard enough, but haven’t been able to find any MTF information on Cosina’s website, which makes it difficult to get an overview of what to expect from their lenses without actually using them. For example, the recent 21/1.8 and 35/1.2 II are highly regarded and somewhat buck the impression of Voigtlander lenses as cheap ‘get what you pay for’ alternatives to Leica and Zeiss.

    Not that you don’t have anything else to do… 🙂

  • Minor typo:

    So I after I did

  • Awesome job! Your reviews are so detailed, with high-end test equipment, spanning several copies of each lens, and including comparisons between competing models. This is why I wish you guys did more lens reviews, even though I totally understand why you can’t do a lot of them.

    While you’re not a lens review shop, your expertise, test equipment, and access to multiple copies of each model uniquely positions you to get the most technical insights about lens performance. Is there any way you could expand this side of your operation? You’d obviously need more people, which is only feasible if you get more income from performing these tests. I’m sure you’ve asked yourselves the same things, but I really hope you can find some way of doing more of these. I don’t have any actual suggestions for making it work, but I really hope it does work out some way.

    Thanks for the great content!

Follow on Feedly