Lenses and Optics

Wide-Angle Zoom MTF and Variations

Published August 10, 2015

We recently published the MTF and variation curves of standard range lenses and now are going to tackle wide-angle zooms.

As we’ve seen, zooms have more variation than primes and wide-angle primes have more variation than standard range primes. Also, the MTF curves of wide-angle primes are weaker off-axis than standard range primes. So my expectations for wide-angle zooms were pretty low. I expected to see a lot of copy-to-copy variation and some pretty weak MTF curves away from the center. As with most zooms, I expected the wide-angle zooms would be distinctly better at one end than the other.

Before we get started, let me address the inevitable emails that are coming, asking if I’ll run out and test your favorite zoom lens tomorrow. The answer is no. This takes an incredible amount of time. Each wide zoom report you see below took a week of testing full time; the four lenses we’ve got here are nearly a month’s work. These were chosen on a pretty simple basis; they are the 4 most popular rental lenses in this category. Someday I’ll add the Canon 17-40mm (trust me, 17-40 fanboy, you’re not going to like the variance), the Nikon 16-35mm f/4 VR, and the Tokina 16-28mm f/2.8. But it won’t be until next month, as I have ADD and I’m really bored with wide zooms right now. Not to mention a large backup of lenses needing tested and adjusted for Lensrentals.

MTF Curves

Let’s start with the not-so-incredibly wide, Canon wide-angle lenses: the 16-35 f/2.8 L and the 16-35 f/4 IS L.
Roger Cicala and Brandon Dube, Lensrentals.com, 2015
Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

The optical bench confirms a couple of things that most of us already knew. For example, the 16-35mm f/2.8 is weakest in the middle of its zoom range. At the 16mm end, it’s at it’s best in the center, but gets pretty weak in the corners. At 35mm it’s not nearly as good in the center as it was at 16mm, but it holds up better in the corners. There’s no question, though, that if you have a 24-70 f/2.8 lens in your bag you’ll want to take it out for all your 24mm and longer shots.

The newer 16-35 f/4 IS L lens is clearly better than the f/2.8 (although remember it’s being tested at f/4). It is, like the f/2.8 lens, at it’s best at the wide end, but it holds up very well at 24mm and 35mm. Overall the 16-35mm f/4 IS gives very impressive performance. But I know what you’re thinking. Would the 16-35 f/2.8 lens be just as good if we were testing it at f/4? Well, I’ll show you.

 

Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

Stopped down to f/4, the 16-35 f/2.8 L dramatically improves in the center at both ends of the zoom range. It’s now about as good as the 16-35 f/4 IS at both 16mm and 35mm, although it is a bit different. At 24mm, though, it’s still not nearly as good as the IS lens. (BTW – those of you who notice the astigmatism jump at 16mm off axis when stopped down — this is a real change and not an artifact. This lens has some higher order aberrations that don’t improve much with the aperture reduced. Basically the sagittal plane gets much better and the tangential plane doesn’t.)

OK, let’s compare a couple of the even wider zooms: the Nikon 14-24mm f/2.8 and the Canon 11-24mm f/4. The Nikon is the gold standard for a wide-angle, wide-aperture f/2.8 zoom. The Canon has broken new ground for just how wide you can go with a zoom. I do want to point out that due to limitations of our optical bench we can’t completely measure the Canon lens at 11mm from edge to edge. Instead of 20mm off axis (nearly to the corner) we can only measure to 17.5mm off axis. So the ‘corners’ on the graph below at 11mm look better than they really are because it’s not measuring as far to the corners as the other lenses.

 

Roger Cicala and Brandon Dube, Lensrentals.com, 2015
Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

These lenses are both just awesome. They are both at their best at the wide end, but both stay really quite good to the long end of the zoom range. Even at f/2.8 the Nikon is as good as the 16-35 Canon is at f/4, except right in the center. And the Canon 11-24, in the center, is the best of all the wide-angle zooms. In summary, though, I’d have to say that all of the wide-angle zooms look better than I would have expected. Sure, the Canon 16-35 f/2.8 L isn’t as good as the other lenses, but at the wide end it’s still very good, and that’s where many of us shoot these lenses most of the time.

But the MTF curves are the averages for 10 copies of each lens. Let’s take a look at how much copy-to-copy variation these lenses have; at what the chance is that the lens you buy performs similarly to these numbers.

Consistency Graphs

Last week we posted the Consistency graphs for the 24-70 f/2.8 zooms and saw they ranged from 4.3 to 6.6 at various focal lengths. As zooms tend to do, the 24-70s all had more variability at one end of the zoom range. All of those zooms had Consistency Scores of less than 5 at the 70mm end, meaning there was a fair amount of copy-to-copy variation. My expectation was that the wide zooms would be about the same as the 24-70 zooms, or perhaps even worse.

Roger Cicala and Brandon Dube, Lensrentals.com, 2015
Roger Cicala and Brandon Dube, Lensrentals.com, 2015
Roger Cicala and Brandon Dube, Lensrentals.com, 2015
Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

So, as is so often the case, the results surprised me a bit. The wide-angle zooms had higher Consistency Scores than the standard range zooms did. The Canon 16-35 f/2.8 L clearly had more variation than the other wide zooms, which wasn’t shocking; it’s an older design and we’ve been commenting on the copy-to-copy variation with that lens for quite a while. The 16-35 f/4 IS was clearly more consistent, and the Nikon 14-24 f/2.8 was excellent.

The Canon 11-24 f/4 though was just shockingly good. We did not expect anything like those numbers. It’s by far the most consistent zoom we’ve tested; far better than many prime lenses. I have no idea what Canon is doing or how they are doing it, but the copy-to-copy variation in most of their new lenses is minimal. I know people love to think that better inspection or QA procedures would accomplish this, but that’s not really the case. A lot of it has to do with designing the lens so that tolerances are not so critical. Put another way, that means that a tiny movement of an element doesn’t cause a huge change in the optics.

The Different Look

I know some of you prefer the 2-D interpolated graphs rather than the MTF curves we used above so I’ll add those here. Remember, while this gives a nice, simplified, means of comparison, like any simplification you lose some data. Remember, too, that for these graphs we average out any astigmatism so I strongly suggest looking at the MTF charts for astigmatism (how far apart the dotted and solid lines of each color are) if nothing else.

 

Roger Cicala and Brandon Dube, Lensrentals.com, 2015
Roger Cicala and Brandon Dube, Lensrentals.com, 2015
Roger Cicala and Brandon Dube, Lensrentals.com, 2015
Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

Leaving out some data to make graphs like theses does reduce accuracy, but it gives a nice, easy comparison that shows which lenses are strongest where. As always, I suggest you take a trip over to The Digital Picture where our MTF data is posted in their lens comparison tool, letting you compare any two lenses side-by-side. They also post the 3-D graphs that I don’t have the space to post here.

 

Roger Cicala and Brandon Dube

Lensrentals.com

August, 2015

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Lenses and Optics
  • Brandon

    KimH,

    Most manufactures use the computed charts from the design software (typically Zemax or Code V) and some custom plotting setup parameters.

    I think Canon uses the toleranced design instead of the nominal design, I am not sure about others. I am not sure of Zeiss’ measurement procedures, but they do measure their lenses on their own K8 MTF bench for their charts.

    -Brandon

  • KimH

    Duuuh – thanks Flemming/Brandon!

    Interesting with the change is method – i wonder what the vendors do in the “ideal & calculated case”. which rotation point do they publish… Apples, Oranges & Pears – here we go :o)

    still VERY pleased with my choice(s)

  • Brandon

    KimH,

    The tangential and sagittal planes have switched formats/line styles because when I wrote the software that generates these plots, I chose the standard used in the design software used by Code V.

    The difference in the actual measurements are likely down to methodology; Roger and Aaron were previously standardizing to the axis of rotation crossing through the front element. I am standardizing to rotating about the entrance pupil for a properly gaussed measurement setup.

    You may see some info on how to best measure MTF from Optikos here:
    http://www.optikos.com/wp-content/uploads/2013/11/How-to-Measure-MTF.pdf

    All the best,
    Brandon

  • Roger Cicala
  • Flemming

    Kim,

    Sorry for replying for someone else, but the sagittal and meridional lines have switched places. The reason is in April they used the dashed line for the tangential curve while they used the dashed line for for sagittal in this article.

    Here is the legend from the article in April: http://wordpress.lensrentals.com/media/2015/02/legend1.jpg

  • Jeff Allen

    Im in the eviable position to be able to test my Canon lenses like Lens Rentals because we rent film equipment and have an MTF machine as well as projectors and f/t stop machine. We buy lenses for rental either to use as they are or re-mount into PL lenses and therefor reject weaker lenses. Personally Im able to make sure my own lenses are selected and whilst not in the market for the Canon 11-24mm f4L lens it did influance my choice of the 16-35mm f4L over the 16-35mm f2.8L and for landscape my main love f4 is not an issue.
    The reason cinematography lenses are more expensive is because they are individually optimised, individually engraved at every distance and the designs need to be weather sealed and operable over a wide tempreture range. They also are driven by motors which means mechanics need to be more robust as pulls can be quick. Good assessment well done guys.

  • Jack

    Great job. Any recommendation of a good article that covers the basics of MTF charts well and doesn’t require a degree in optics?

  • KimH

    Roger,

    you have done it again – this is exciting, interesting AND useful, no question!

    i have been staring at the test you did Feb 27 and compared MTFs with this article and i am stuck – i have to ask.

    With the consistency you give for the 11-24, the measured MTF from Feb 27th seems to be odd/off.

    it is like the Sagittal and Meridional have switched places and lost some umpf

    Looking at the 11 & 24mm MTFs compared, your most recent measure seem to be VERY different

    is there a reason for this or am I reading it all wrong.

    I have the 11-24 (and the 16-35 f4) and i am VERY pleased, both are amazing pieces of glass

  • Jack

    Thanks and great job. For those not well versed in MTF charts etc. can you recommend an article or two that does a thorough job with the basics?

  • KSGal

    This blog provides information we can find no where else, and it is all so incredibly interesting! Thanks Brandon and Roger for taking the time to find real world answers to our photography gear questions!

  • Brandon

    Andrew,

    The spot from the 11-24 is exceptionally clean. It is not loosely toleranced, just very precisely built and aligned from the factory.

  • Andrew

    Thanks again guys. How is canon doing this? I already commented but have been thinking a bit and it might be an idea to take a new Canon Lens (For fun make it the 11-24!), measure it, take it apart and then reassemble and test again. You could even do it twice: first tine you just reassemble, and the second time you try to calibrate as well as possible.

    Could be interesting to see the results – are they really designing for inaccurate assembly? That might be a world first I would have thought and the precision manufacturing world would like to know about it.

  • Jim Thomson

    Adam
    Roger’s take on the Tokina 16-28 f2.8 was very positive. Photozone.de also got good results, but had some issues with centering. They rated it optically better than the Canon 16-35 f2.8. I don’t think the lens will do very well in the consistency department.

  • required

    you are not made to write your own reference when the image belongs to you, simply a Fig. 1, and so on, in case you want to talk about them in the text. Also me

  • required

    and you rather make your text more dumb proof, because some people here can’t understand a graph not even a cartoon one 🙁

  • Brandon

    James,

    Funny you should mention that – just today I began work on an extension of the current program to handle just that =)

    It is in its very early stages, but it allows averaging copies, among other things the software Trioptics has prepared for this feature of the bench does not allow.

    It is going to get kicked back a little bit, since the last part for the franken-E-mount came in today, hopefully enabling us to put Sony to the bench and that takes priority to me.

    Here’s a plot: http://i.imgur.com/kOUZreS.png

    We will be able to provide any angle/view of the data.

    All the best,
    Brandon

  • Adam

    Thanks for the comparison, love this data. However, for the life of me I can’t figure out why you’d test the Tokina 16-28 over the Tamron 15-30. I can’t say I’ve ever heard anyone recommend getting the Tokina over the Nikon or the Canon, or even put them in the same class, yet the Tamron comes up as an alternative all the time since its release. The Nikon 16-35 makes sense, and I guess a lot of people own the canon 17-40, though we already know it probably won’t hold a candle to the 16-35 f/4.

  • James Scholz

    Thanks again Roger and Brandon for all your fine work. The information is very useful.

    As an architectural photographer flatness of field is very important to me, especially in wide angle optics. I realize that this series deals with sample variation, but hopefully going forward can include the field curvature graphs like you have published in the past.

    I am very interested to see how the new Nikon 24-70 compares in this regard to the current model.

Follow on Feedly