Lenses and Optics

Supertelephoto MTF Curves

Published July 25, 2015

Brandon has accomplished much during his summer with us, including several things I never thought we’d be able to do. One of those things was measuring the MTF of supertelephoto lenses on our optical bench. A vertical bench just isn’t designed to handle the mass of those big lenses and technically isn’t supposed to be able to test anything over 200mm focal length. Brandon found some workarounds that allowed us to get accurate MTF readings on a number of supertelephoto lenses.

 

 

Some of the workarounds are simple. For example, there’s a lot more vibration with a big, long lens. So after the machine moves to each measurement point we had to program in a long delay before doing the measurements. There is also a bench limitation that lightly affects the results. Our largest collimator provides up to a 100mm diameter beam of light for tests, so the 300mm f/2.8 lenses are at about f/3 and the 400mm lenses are at about f/4. There’s nothing we can do about that, Trioptics doesn’t even make a bigger collimator and if they did it would cost $100,000 by itself.

We weren’t able to do our full set of 10 copies of each lens there just weren’t enough in stock. There are at least 5 copies (Nikon 400 f/2.8), though, and most have 7 to 9 copies tested. But because there are less than 10 we won’t be putting out consistency numbers today.

Before we present these results to you, let me get one thing out of the way: we aren’t going to be testing many more of these. Brandon’s going back to school soon. Measuring these is too time-consuming, meticulous, and repetitious for me to want to do it. To put it in perspective, I could test 5 or 6 standard prime lenses in the time it takes to do one super telephoto run. So what I share today is likely to be all that’s going to be shared when it comes to measuring MTF curves on super telephoto lenses.

One other thing I should mention, you’ll notice some of these real MTF curves are fairly different from the manufacturer’s released MTF curves. (This is true for other lenses, too, but readily apparent for these lenses because the manufacturer’s MTF curves are so nearly perfect.) That’s the difference between a computer generated ideal MTF curve of the design, and an actual measurement of the manufactured product. The reality of the manufactured lens accounts for some of the difference, but also the reality of actual measurement (versus computer simulation) accounts for some of it too. In this case particularly, I want to emphasize that we’re using our MTF bench outside of its intended purpose, so I can’t be certain, especially at 400mm, that we aren’t introducing some other factors that might be affecting the measured MTF curves.

200mm Lenses

There is a lot of ways to get to 200mm. Most commonly, it’s a 70-200mm zoom shot at 200mm (we’ll release those MTF curves when we start releasing zoom lenses). Canon also has a relatively inexpensive 200mm f/2.8 prime lens. Both Canon and Nikon have very expensive 200mm f/2.0 primes. As always, when you look at these graphs, remember different apertures are comparing apples and oranges. A lens tested at f/2 would have higher MTF if tested at f/2.8, etc.

Just to give you a bit of orientation, I’m going to start with the MTF curves of two excellent lenses: the Canon 70-200 f/2.8 IS II at 200mm and the Canon 200mm f/2.8 L lens. These are both really good, although slightly different. The older 200mm f/2.8 L prime isn’t quite as good in the center at higher frequencies, but clearly does better off axis, which is what we would expect from a prime lens compared to a zoom.

 

Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

Those were very good lenses, but the f/2.0 prime lenses are just spectacularly good.

Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

Even at f/2.0, their MTF is clearly better than either of the f/2.8 lenses. Any fanboys who want to claim one is better than the other are being pretty silly. They’re different, sure, with the Canon 200mm f/2.0 IS L is a tiny bit better at higher frequencies, but having more off-axis astigmatism than the Nikon 200mm f/2.0 VR II. But both are totally amazing.

One other lens that I get asked about a lot is the Sigma 120-300mm f/2.8 HSM A1 zoom. I put its curve beside the Canon 70-200 f/2.8 IS II lens just for comparison purposes.  I’ll be the first to admit I’m surprised at just how good the Sigma 120-300 f/2.8 Sport lens is.  I put the Canon 70-200 f/2.8 IS II graph beside it to make the comparison easier and show just how excellent the Sigma is.

Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

The overall the 200mm summary is pretty easy: they’re all damn good. And yes, I’m aware that everyone wants to know more about the 70-200 f/2.8 zooms. That post will be out in a week or two; I just put this one example up for a comparison point today.

Longer Focal Lengths

I started off the 200mm section using some examples that a lot of people were likely to be familiar with, and I’ll do the same thing here. Below is the MTF curve for the Canon 300mm f/4 IS lens. It’s an older design, but one that’s pretty popular. I’ll put it beside the Canon 200mm f/2.8 we used above to give you a comparison point. The 300 f/4 IS is a pretty good lens, but the absolute resolution isn’t quite as good as the 200mm f/2.8. While we aren’t presenting consistency numbers with these lenses, we did do 10 copies of the 300 f/4 IS and it did show significant variation (Consistency 4.2 for the 300 f/4 IS, compared to 8.9 for the 200 f/2.8).

Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

The 300mm f/2.8 lenses are pretty expensive, and the MTF curves show some of the reason why; they are amazingly good. Like the 200mm primes, the Canon 300 f/2.8 IS II and Nikon 300 f/2.8 VR II they are slightly different, although the differences are minimal. The Canon, again, has a bit better center and sagittal resolution, the Nikon has less astigmatism away from center giving it better tangential resolution, but these are nit-picking, pixel-peeping differences. They’re both incredible.

Roger Cicala and Brandon Dube, Lensrentals.com, 2015
Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

The Sigma 120-300mm f/2.8 Sport at 300mm surprised me again. I knew it was very good, but in the center it’s every bit as good as the two name-brand prime lenses. It drops off away from center, obviously, but even there it’s still very good. That’s amazing performance for a zoom lens. Obviously, there is a lot more that goes into choosing a 300mm lens than just MTF curves, but the Sigma puts in an amazing performance.

400mm f/2.8

The 400m f/2.8 prime lenses are generally considered to be the absolute best lenses made. The MTF curves, which are at f/4, in my mind, suggest that they’re maybe a teeny tiny bit better than the 300 f/2.8s, but I’d be hard pressed to say I think you’ll see the difference in a picture. I do need to mention here that the aperture of the collimator in the MTF bench means these lenses aren’t being actually tested at f/2.8; the effective aperture is close to f/4.

 

Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

So What Did We Learn Today?

Not much, really. I learned the Sigma 120-300 f/2.8 HSM OS is really good; better than I thought it was. A lot of you already knew that. You may have learned that manufacturer’s MTF charts are sometimes better than any actual lens is (no copy of the supertelephoto lenses we tested approached the published MTF, period). Other than that, this post just documents what we already know. Super telephoto lenses set the high end of the MTF bar. They are as good as photo lenses get.

 

Roger Cicala and Brandon Dube

Lensrentals.com

July 2015

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Lenses and Optics
  • Roger Cicala

    Helder, I won’t say never, but probably never. It’s way, way too time consuming and difficult.

  • Helder

    Roger,

    Could you test some different super zoom primes? Like the sigma 300 2.8 or other budget primes.
    I’m really curious.

  • Brandon

    Ilkka,

    Lots of questions here that can’t be separated perfectly.

    I do not know how new samples compare to old. The range of ages for e.g the Canon 300/2.8 (12 copies) is 1 month / 3 rentals to 1.5yr / 41 rentals. The oldest copy and the youngest one are not more different to each other than other copies in the sample in terms of MTF.

    For a time I thought it was the drop in filters, since mechanically that is a very imprecise mechanism, or it was the mount being tilted under stress. But removing the filter altogether did not make a significant change to the astigmatism of a worse-off copy, and across the sample the tilts are completely random so it is unlikely to be the mount(which is itself supported on a 1/4″ thick steel plate).

    I will say that even the worst corner of the worst copy is still far and away better than what even the highest resolution cameras can see, e.g the 5DsR or the 7D mk II or the D810.

    Roger is better capable of answering how robust they are in comparison to each other, but they are all quite well made and I have no significant gripes with any of them, personally. The 300mm f/2.8 Nikkor does extremely well – it is actually the best lens I have tested for consistency. In absolute terms is variance is similar to the 135mm f/2 Apo-Sonnar, with significantly higher resolution which is quite a feat.

    I do not know if the lighter weight models are more susceptible to damage than the heavier ones. The actual lens element mounting methods should be quite similar, but the barrels may be more deformable; or they may not be.

    I think most of the tilt in these may come from the IS units, but it is impossible to tell.

    Best regards,
    Brandon

  • Ilkka Nissilä

    If the astigmatism (which to some extent is also present in the simulated curves for many lenses, though not so much for the fast supertele primes) is caused by tilted elements during shipping, how do the brand new lens samples perform compared to ones that have been rented many times? If there are large differences between a lens when new and after that same sample has been used in a while, the considerable financial investment involved in getting these lenses would not seem to be matched by expected long term performance. Expensive lenses like these should last with virtually unchanged performance for many years of use. Any comment? Also it seems that some lenses do not show much astigmatism e.g. the 300/2.8 Nikkor. Does this mean it is constructed more robustly (in terms of mechanical and optical tolerances and long term durability) or just not yet beat up as much as some of the others? Does the ultra light weight contstruction of some newer superteles lead to greater sensitivity to impacts? This is an important question as the lighter weight versions in many cases are substantially more expensive than their predecessors.

  • Brandon

    Christian,

    The tilt in these lenses is real – the astigmatism is seen both on the bench’s target as well as on OLAF, our pinhole collimator. A significant factor at play is the fact that these are rental lenses and get shipped around far more than a lens fresh from the factory has been. The optics themselves are quite massive and it seems likely that something has shifted a bit in all that bouncing around.

    An MTF bench performs a pretty simple function internally. It images a known object, adjusts the captured image based on some calibrations (for magnification, the microscope, the collimator, and background subtraction notably) and performs a FFT of the result.

    For this to be in error one must deform the spot or break a calibrations. The calibrations remain correct for these focal lengths, all that is left is the spot. Short of clipping the field of view of the microscope / Region of Interest of the calculation, any error is down to the sample lens. All of these lenses were profiled for distortion and closely monitored during testing, with extra care to ensure measurement accuracy.

    Best regards,
    Brandon

  • I was made aware of this thread and just want to comment on the procedure of the MTF measurements we use for making lens reviews at our website, http://www.lensfreaks.com.

    We measure MTF about the same way as you guys, that is with best focus adjusted for the center of the lens throughout the whole measurement. We do not adjust focus for every point to obtain a best possible MTF value. That indeed would be a scenario not very well correlated to the actual photographic use of the lens.
    We also have MTF at higher frequencies than 20 cycles/mm and they show about the same sagital and tangential behaviour. That is a rather flat curve all the way across the image field. I have uttermost respect for your work and knowledge but perhaps Alan F is correct. You may have some trouble with your equipment.

    Best regards

    Christian Nilsson
    author at lensfreaks.com (and swedish site objektivtest.se)

Follow on Feedly