Lenses and Optics

Supertelephoto MTF Curves

Published July 25, 2015

Brandon has accomplished much during his summer with us, including several things I never thought we’d be able to do. One of those things was measuring the MTF of supertelephoto lenses on our optical bench. A vertical bench just isn’t designed to handle the mass of those big lenses and technically isn’t supposed to be able to test anything over 200mm focal length. Brandon found some workarounds that allowed us to get accurate MTF readings on a number of supertelephoto lenses.

 

 

Some of the workarounds are simple. For example, there’s a lot more vibration with a big, long lens. So after the machine moves to each measurement point we had to program in a long delay before doing the measurements. There is also a bench limitation that lightly affects the results. Our largest collimator provides up to a 100mm diameter beam of light for tests, so the 300mm f/2.8 lenses are at about f/3 and the 400mm lenses are at about f/4. There’s nothing we can do about that, Trioptics doesn’t even make a bigger collimator and if they did it would cost $100,000 by itself.

We weren’t able to do our full set of 10 copies of each lens there just weren’t enough in stock. There are at least 5 copies (Nikon 400 f/2.8), though, and most have 7 to 9 copies tested. But because there are less than 10 we won’t be putting out consistency numbers today.

Before we present these results to you, let me get one thing out of the way: we aren’t going to be testing many more of these. Brandon’s going back to school soon. Measuring these is too time-consuming, meticulous, and repetitious for me to want to do it. To put it in perspective, I could test 5 or 6 standard prime lenses in the time it takes to do one super telephoto run. So what I share today is likely to be all that’s going to be shared when it comes to measuring MTF curves on super telephoto lenses.

One other thing I should mention, you’ll notice some of these real MTF curves are fairly different from the manufacturer’s released MTF curves. (This is true for other lenses, too, but readily apparent for these lenses because the manufacturer’s MTF curves are so nearly perfect.) That’s the difference between a computer generated ideal MTF curve of the design, and an actual measurement of the manufactured product. The reality of the manufactured lens accounts for some of the difference, but also the reality of actual measurement (versus computer simulation) accounts for some of it too. In this case particularly, I want to emphasize that we’re using our MTF bench outside of its intended purpose, so I can’t be certain, especially at 400mm, that we aren’t introducing some other factors that might be affecting the measured MTF curves.

200mm Lenses

There is a lot of ways to get to 200mm. Most commonly, it’s a 70-200mm zoom shot at 200mm (we’ll release those MTF curves when we start releasing zoom lenses). Canon also has a relatively inexpensive 200mm f/2.8 prime lens. Both Canon and Nikon have very expensive 200mm f/2.0 primes. As always, when you look at these graphs, remember different apertures are comparing apples and oranges. A lens tested at f/2 would have higher MTF if tested at f/2.8, etc.

Just to give you a bit of orientation, I’m going to start with the MTF curves of two excellent lenses: the Canon 70-200 f/2.8 IS II at 200mm and the Canon 200mm f/2.8 L lens. These are both really good, although slightly different. The older 200mm f/2.8 L prime isn’t quite as good in the center at higher frequencies, but clearly does better off axis, which is what we would expect from a prime lens compared to a zoom.

 

Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

Those were very good lenses, but the f/2.0 prime lenses are just spectacularly good.

Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

Even at f/2.0, their MTF is clearly better than either of the f/2.8 lenses. Any fanboys who want to claim one is better than the other are being pretty silly. They’re different, sure, with the Canon 200mm f/2.0 IS L is a tiny bit better at higher frequencies, but having more off-axis astigmatism than the Nikon 200mm f/2.0 VR II. But both are totally amazing.

One other lens that I get asked about a lot is the Sigma 120-300mm f/2.8 HSM A1 zoom. I put its curve beside the Canon 70-200 f/2.8 IS II lens just for comparison purposes.  I’ll be the first to admit I’m surprised at just how good the Sigma 120-300 f/2.8 Sport lens is.  I put the Canon 70-200 f/2.8 IS II graph beside it to make the comparison easier and show just how excellent the Sigma is.

Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

The overall the 200mm summary is pretty easy: they’re all damn good. And yes, I’m aware that everyone wants to know more about the 70-200 f/2.8 zooms. That post will be out in a week or two; I just put this one example up for a comparison point today.

Longer Focal Lengths

I started off the 200mm section using some examples that a lot of people were likely to be familiar with, and I’ll do the same thing here. Below is the MTF curve for the Canon 300mm f/4 IS lens. It’s an older design, but one that’s pretty popular. I’ll put it beside the Canon 200mm f/2.8 we used above to give you a comparison point. The 300 f/4 IS is a pretty good lens, but the absolute resolution isn’t quite as good as the 200mm f/2.8. While we aren’t presenting consistency numbers with these lenses, we did do 10 copies of the 300 f/4 IS and it did show significant variation (Consistency 4.2 for the 300 f/4 IS, compared to 8.9 for the 200 f/2.8).

Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

The 300mm f/2.8 lenses are pretty expensive, and the MTF curves show some of the reason why; they are amazingly good. Like the 200mm primes, the Canon 300 f/2.8 IS II and Nikon 300 f/2.8 VR II they are slightly different, although the differences are minimal. The Canon, again, has a bit better center and sagittal resolution, the Nikon has less astigmatism away from center giving it better tangential resolution, but these are nit-picking, pixel-peeping differences. They’re both incredible.

Roger Cicala and Brandon Dube, Lensrentals.com, 2015
Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

The Sigma 120-300mm f/2.8 Sport at 300mm surprised me again. I knew it was very good, but in the center it’s every bit as good as the two name-brand prime lenses. It drops off away from center, obviously, but even there it’s still very good. That’s amazing performance for a zoom lens. Obviously, there is a lot more that goes into choosing a 300mm lens than just MTF curves, but the Sigma puts in an amazing performance.

400mm f/2.8

The 400m f/2.8 prime lenses are generally considered to be the absolute best lenses made. The MTF curves, which are at f/4, in my mind, suggest that they’re maybe a teeny tiny bit better than the 300 f/2.8s, but I’d be hard pressed to say I think you’ll see the difference in a picture. I do need to mention here that the aperture of the collimator in the MTF bench means these lenses aren’t being actually tested at f/2.8; the effective aperture is close to f/4.

 

Roger Cicala and Brandon Dube, Lensrentals.com, 2015

 

So What Did We Learn Today?

Not much, really. I learned the Sigma 120-300 f/2.8 HSM OS is really good; better than I thought it was. A lot of you already knew that. You may have learned that manufacturer’s MTF charts are sometimes better than any actual lens is (no copy of the supertelephoto lenses we tested approached the published MTF, period). Other than that, this post just documents what we already know. Super telephoto lenses set the high end of the MTF bar. They are as good as photo lenses get.

 

Roger Cicala and Brandon Dube

Lensrentals.com

July 2015

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Lenses and Optics
  • Lee Saxon

    Very interesting about the resolution limits of the sensors. I’m surprised to hear that they’re so low. If the sensor can’t resolve above 0.5 I don’t understand why we can see a difference between a lens measuring 0.6 and a lens measuring 0.8? I also wonder how much the bayer filter / debayering affects MTF (visually, we can see looking at Epic Monochrome vs regular Epic that the difference is significant)

  • Thanks as always Brandon!

  • AJ

    Would have been nice to see the older Nikon 300 F4 pitted against the new 300mm PF F4 VR.
    Also the Nikon 70-200 F2.8 vs the F4 but understand that time is limited, especially when it comes to the more lengthy procedures for zooms.

  • Brandon

    The loss from the bayer array is approximately 50%. With sharpening one can achieve about a “100%” utilization of the camera’s resolution (e.g 4000lp/ph on an 8000px camera). However, for a non-monochromatic scene using a standard converter you can expect 50% losses. The loss from the sensor itself increases towards the edges due to angle of acceptance issues (1). An AAless camera will lose about 15-20% peak resolution in the center, as a baseline. To empirically determine it, it would be best to use a telecentric lens that is diffraction limited at a large f/number that is designed for the proper coverglass thickness. You will find these conditions can rarely be met and it must be extrapolated from many nonparametric data points found as resolution data from many high performing lenses.

    (1) http://blog.teledynedalsa.com/2012/05/the-angle-on-optical-acceptance/

    All the best,
    Brandon

  • Thanks Brandon! That’s a good answer for me. I was just wondering if there’s some kind of formula for doing it, especially for estimating the loss from the Bayer CFA, but it sounds like perhaps this is empirically determined?

  • Brandon

    Andre,

    Keeping this a bit short, but the absolute limit of detectability is when two airy disks overlap more than 50%, for diffraction limited systems it is known as the rayleigh criterion. This correlates very closely to an MTF of 50%. The 5DsR has a pixel density of about 200px/mm, but you lose some due to the bayer filter and the non perfect MTF of the sensor itself.

    Please email me if you would like more detail.

    All the best,
    Brandon

  • Brandon, how does one generally compute the limits of MTF resolution for a sensor? I noticed that you mentioned the 5DSR’s limit is an MTF of 0.5 at 50 lp/mm.

  • Roger,

    when you put the Zooms on the bench, please do me a favor and test the Sigma 50-150mm f/2.8 APO EX DC OS HSM! Not sure how much you have in stock, but it’ll surely do 😉 .

    The Sigma Art 18-35mm f/1.8 is on top of the list for sure, but just in case… please test it, too.

    Regards,
    Lasse

  • Roger Cicala

    Bill, the repair rate for Sigma is better than it was a few years ago, although 120-300 Sports still have a fairly high rate. Given the mass of that lens I suspect it’s going to always have a tendency to suffer when it gets bumped around.

  • Roger Cicala

    Alan, we measured 12 copies of the 300 f/2.8 IS II and the measurement variation was less than 2%. My guess is that the data you reference is measured at best focus for each point across the image plane. Or, if their bench was an Optikos bench, there is a setting that removes tilt. That’a valid way to do things, and pertinent if you want to see what the MTF would be at that distance if you chose, for example, and AF point at that position. However, if the lens is slightly tilted, as all of the 300 f/2.8 IS IIs we measured were, you would get an ‘idealized’ MTF.

    Our measurements are all made at the best focus position for the center of the lens. This means a slight tilt puts the other points a bit out of focus and therefore MTF is lower. My own feeling is this is important in the real world since you can’t focus your lens differently at 20 points in one photograph. More practically speaking, since my purpose in testing is to find lens defects, however minor, seeing tilt is important to me. If we measured at best focus point for each position we’d have curves much closer to their curves. I’m just not interested in doing that.

    Roger

  • Alan Fersht

    As a comment has been made about the 300mm f/2.8 II data, I must add that Lensfreaks have published data from the Hasselblad factory equipment on the Canon 300mm f/2.8 II. Their measured MTFs for 20 lp/mm are nearly flat (~0.92-0.88 from centre to edge) as are those at 30 lp/mm (~0.82-0.75) for both sag and tan http://www.lensfreaks.com/lens-reviews/canon/canon-ef-300mm-f28-l-is-ii-usm-review-a-nice-little-beast/. They have measured only one lens, but the discrepancy is huge.

    Have you done repeat measurements for the same lens and looked at the variances for those?

  • Brandon

    Doug,

    The absolute limit of detectability for the 5DsR is an MTF of 0.5 at 50lp/mm. Realistically anything as bad as 0.3 will look completely fine. The worst corner of the worst copy of the 300mm f/2.8L IS II is still better than even the 5DsR can detect.

    Regards,
    Brandon

  • Doug McEwen

    Thank you for publishing such valuable data. Perhaps because of the extreme difficulty of testing supertelephotos on the optical bench, a few of these results don’t seem to correlate well with the ISO 12233 tests at TDP. For example, the EF 300mm f/2.8L IS II USM shows mediocre tangential MTF but I don’t see any astigmatism at all on the ISO 12233 results even with the 5DsR which looks flawless to my eyes. The same with the EF 200mm f/2L IS USM. But curiously, the EF 200mm f/2.8L II USM shows the same tan and sag MTF, which correlates well with the ISO 12233 results that don’t have astigmatism, so it seems unlikely that there is some kind of error in the optical bench tangential results at long focal lengths. I am not an optical expert, so perhaps just I just don’t understand something here.

    Again, thanks for sharing such valuable data with the photographic community.

  • Wally

    CaF2 lenses are single crystal, and the birefringence is very visible if the crystal is held in the correct orientation. Lenses are normally cut so that the lens is not birefringent on the optical axis, but if it is misaligned it can have a little bit of birefringence. The amount is very small at small angles. I am not saying this is the issue with the Canon lenses, I am a materials engineer but other than normal cameras have little experience with optics, so just bringing up the possibility.

  • Wondering if you’re also amazed at the number for repairs you’re experiencing with the 120-300mm Sport as compared to the old model 120-300mm Sigma?

  • Tobias

    Hi,

    I know Brandon is going back to school, any chance of getting the 400 5.6 tested? It’s a lot more affordable and I’d like to see the real life trade-off =)

    Thanks and keep up the great work!

  • Brandon

    Wally,

    I am no expert in materials, so I may be wrong. However, the size of individual crystal cells of CaF2 is much smaller than the wavelengths that make up the visible band, so birefringence should not be an issue in visible light applications.

    -Brandon

  • Wally

    CaF2 crystals are birefringent, if the lens has not been cut perfectly or ground perfectly some of this birefringence might show up in the actual lenses, while it would be excluded from the computer models as the lenses are normally orientated to eliminate this effect. Could this be effecting the astigmatism.

  • obican

    #HireBrandon

  • Brandon

    Alan,

    I do not believe it is a systematic error. These lenses required a close watch be kept on the measurements as they proceeded due to a high chance of the IS units settling in a different position from the vibration of spinning the sample. More so than other lenses.

    Here’s one sample of the 400 Nikkor: https://www.dropbox.com/s/8nz3kr2ezhzttuc/MTF_Rotations.png?dl=0 and another: https://www.dropbox.com/s/ur1o3lttasb99uh/MTF_Rotations.png?dl=0

    With sample 250241 it is clear from the MTF that there is a decenter and a tilt balancing each other. In the “0” orientation they are adding constructively, producing a large loss of resolution in the direction of the coma flare. In the “90” orientation, they add destructively and mostly cancel out. 244986 clearly has more coma (decenter) and less astigmatism (tilt).

    The sample size for these is very small. We are at n=5. If we had the resources to measure to n=25 or n=50, they would mostly cancel out. In wider angle lenses with some field curvature the tilt makes one side worse and one better, so it cancels nicely in the average. The superteles have very flat fields, so tilt hurts both sides and brings the average down. The field of view is very small, so build aberrations do not have far to grow and it produces a more uniform loss of resolution.

    “out of range” is arbitrary with this type of equipment. There is a compensation function in the MTF calculation based on the magnification, which is derived from the EFL. In the config of the ImageMaster this function is fit from a 10mm EFL to a 1000mm EFL. The 200mm limit suggested in the software is due to the spread between the two crosses used to measure on-axis EFL. A longer EFL will push them too close together for accurate computation. A second consideration is aperture; past 200mm you cannot measure large apertures, so they simply do not recommend measurement. Centering also becomes very difficult as the adjustment range limit is approached and exceeded and the reference “0” image height must be changed.

    Regards,
    Brandon

  • Alan Fersht

    Brandon
    For the MTFs of all of the shorter focal length lenses (all the 200mm or less, and basically true at 300mm), the sagittarial and tangential are the same at the centre. But, for the Nikkor 400mm that you have really serious deviations between sag and tan at the centre, and smaller ones for the Canon 400mm. It would seem very unlikely that the average of the Nikkors would have tilt or decentering (random tilts etc should cancel out in an average). Is it more likely that you have a systematic error because you are out of range of the equipment?

  • Brandon

    Ala,

    In aberration theory, Astigmatism is forbidden on axis by rotational symmetry. If you break symmetry with a decenter or a tilt, you can get any aberration on axis that shouldn’t be there =)

  • Ala Fersht

    Why does the Nikon 400mm have different values for sagittarial and tangential at the centre for 40 and 50 lp/mm? I thought they were always the same at the centre.

  • Brandon

    Tony,

    Fluorite (CaF2) is functionally very similar to any “super ED” glass. They all have low refractive index (nD~1.35-1.45) and very low dispersion (Vd>80) and an anomalous partial dispersion. They are also all very fragile. The edge astigmatism in all of these is due to so-called build aberrations, not design faults. In the case of these lenses it is mostly caused by small tilts inside visible in the 4 quadrant reports for the various copies.

    Here’s CaF2’s refractive index plot: http://refractiveindex.info/?shelf=main&book=CaF2&page=Li

    Compare it to a super ED glass: http://refractiveindex.info/?shelf=glass&book=FK51A&page=SCHOTT

    And compared to a “normal” ED glass: http://refractiveindex.info/?shelf=glass&book=BAK1&page=SCHOTT

    And a highly dispersive glass: http://refractiveindex.info/?shelf=glass&book=SF11&page=SCHOTT

    Note the axis changes.

    -Brandon

  • Tony

    Here’s a random thought. The 200, 300, and 400 primes have an interesting correlation. The ones that use fluorite elements have a bit more central resolution but more edge astigmatism. The new Nikkor 400 E is so different than it’s older siblings, yet it’s similar to the Canons. Maybe this is a ripple effect of using either fluorite or ED glass for the control of chromatic aberration; there will be different things to deal with “downstream” in the design.

  • Jim Thomson

    Any chance of doing the 400 mm f5.6L? It’s the only super telephoto prime that mere mortals can afford. It also has very few lens elements and I am curious about how that may affect consistency.

    The consistency score for the 200mm F2.8 is remarkable for such an old lens.

  • Brandon

    Anton,

    I don’t know, we’ve never talked about it since MTF vs spatial frequency is not a chart commonly used on the consumer side of the photo/video industry. It is certainly possible.

  • Brandon, will more diffraction limit charts (like the one referenced above) be made available ?

  • Brandon

    Brian,

    Here is a plot of one of the superteles vs spatial frequency with the diffraction limit: http://i.imgur.com/ecHmdKr.png

    I believe the superteles are approximately diffraction limited as designed, but for whatever reason this is not the case as built.

    The 200mm limit imposed by the bench is largely to do with magnification; the target projected at infinity for the lens has some angular size, which is magnified by the lens. On a 14mm lens 2mm on the sensor covers over 8 degrees, on a supertele it is a mere fraction of a degree. Eventually the target’s width becomes an issue for MTF calculation, but the bench doesn’t seem to be erroneous until past about 900mm or so.

    I believe the biggest issue is the mass of the optics, the front couple of elements would pose quite the challenge to hold absolutely still even in transit. I believe that either they become decentered or tilted over time, the IS unit does not remain perfectly aligned over time, or the drop in filters develop alignment error over time. The astigmatism in most of these plots isn’t real per-se, but is an artifact of small, highly magnified tilts inside the lenses.

    Regards,
    Brandon

  • Brian

    I have to disagree with you, Roger, and say that the 300mm f/2.8s measure a little better than the 400mm f/2.8s. I wonder if the (effective) f/4 aperture is producing a little loss of resolution due to diffraction? Hard to draw a conclusion since you are so far out of the test equipment’s designed capabilities.

Follow on Feedly