Testing Lenses: Stopped-Down MTF Curves

For quite a while now we’ve been publishing our MTF curves. They’ve always been the same thing; at least ten copies of each lens, in 4 rotations, and measured at the widest aperture. The reasons for that are (to us) very straightforward. Our purpose in doing the tests is to establish reasonable expectations, learn what type of optical adjustments are possible, and figure out how much copy-to-copy variation to expect. We often publish the results so you can see them, too.

There are other things that photographers and videographers would like to see us do; field curvature, stopped-down MTFs, high-frequency testing, etc. We’re in kind of a catch-22. The thing we’ve considered most important to show; variation between copies; requires we test at least ten copies of each lens before publishing results. Testing 10 copies mean we just don’t have time to do those other tests.

We’re rewriting our proprietary software and exploring some interesting ways to present data. While we’re doing that, I thought we’d revisit our ’10-copy’ rule and consider presenting some more data for you all to look over. One of those things is stopped down MTF. We don’t have time to test 10 copies at every aperture stop. But we can test 10 copies wide-open, pick out one average copy, and test that. It’s not full 10-copy data, but at least that copy will be representative of the norm.

Today, I’m going to present data from Canon, Sigma, Rokinon, and Zeiss Cinema primes. Why? Because those are the same as the photo primes optically, so what happens with the Cine lenses is representative of what will happen with photo lenses. We get a two-for-one special. (The one exception I’m aware of is that the Canon CN-E 35mm T1.5 lens is optically the same as the Canon 35mm f/1.4L original version, not the Mk II version.)

I’m not going to try every aperture for these lenses, but I will compare them wide-open and at T4. In the future, we’ll probably add T5.6 and perhaps T8 data (I’m open to suggestions) for most lenses. But that will take time, and this is what I have right now. And this is meant as an introduction to some of the other things we’re testing that we haven’t shown yet.

T4 MTF Curves

I’ll just go down the list, by focal length. For each lens, we’ll show an average copy wide open on the left and stopped down to T4 on the right. Remember the starting apertures are different. We would expect the jump from T1.5 to T4 to be more impressive than that from T2.1 to T4.

85mm

Yes, everything is sharper at T4. But the 85mm graphs, I think, may surprise some of you. They did me, at least a bit.

The Canon and Xeen are basically soft-focus lenses wide open, and jump to quite good and certainly adequate, respectively, at T4. The Sigma is already sharp wide open and just breathtakingly sharp at T4. The two Zeiss are nearly identical at T4 (as they should be, they’re very similar lenses). From a resolution standpoint, though, at T4 neither is nearly as sharp as the Sigma, nor quite as sharp as the Canon. (That last part surprised me a bit.)

Canon CN-E 85mm T1.3

Sigma Cine 85mm FF T1.5 

 

Rokinon Xeen 85mm T1.5

Zeiss CP.2 85mm Super Speed T1.5

Zeiss CP.2 85mm T2.1

50mm

At 50mm things are slightly different. The Sigma 50mm is again the sharpest of the wide-aperture lenses wide open, and the sharpest stopped down. The Canon and Zeiss lenses all sharpen up a lot in the center (especially the Zeiss) but still stay somewhat soft in the outer half of the image. The Xeen was the most significant surprise here, going from the lowest MTF wide open to being quite good at T4. In the outer half of the image, it’s the second best lens at T4.

Canon CN-E 50mm T1.3

Sigma Cine 50mm FF T1.5

Rokinon Xeen 50mm T1.5

Zeiss CP.2 50mm Super Speed T1.5

Zeiss CP.2 50mm T2.1

Looking at these 50mm MTFs my first question is whether the Zeiss lenses really get worse off axis when the aperture is stopped down. That makes no sense. So what is happening? For now, I’ll throw out a couple of possibilities, because the next post in this series will get into this more, and also show you some useful new tools. Things to consider are focus shift and field curvature.

The takeaway messages are in two parts. First, standard MTF curves don’t show everything, and occasionally can be a bit misleading. Second, if we delve into them a little more in-depth (which we will in the next few articles) they actually give a lot more information than the standard curves show. Much of that is useful information.

35mm

There’s a bit more uniformity at 35mm with all of the lenses sharpening up nicely at T4. The Zeiss lenses don’t get quite as sharp in the center but have smoother performance away from the center. The Xeen has a lot of astigmatism-like separation of sagittal and tangential lines away from center even stopped down. That can actually be astigmatism, or it could be lateral color. Which is it? That’s another thing we can look at with more detailed testing in the next post.

Canon CN-E 35mm T1.5

Sigma Cine 35mm FF T1.5

Rokinon Xeen 35mm T1.5

Zeiss CP.2 35mm T2.1

 

Wider Lenses

There’s nothing spectacular going on here. Some of the patterns are the same. The Xeen, like the other Xeens, is barely usable wide open but is not only quite good at T4; it’s probably the best off-axis. The old-designed Zeiss 21mm looks terrific at T2.9 and even better at T4. The Canon serves as an excellent demonstration that testing a single lens sometimes results in jerkier graphs than our usual averages of multiple copies. The Sigma raises a similar question to what the Zeiss 50mm lenses did. Is that weakness off-axis at T4 field curvature or is the lens just not sharpening up because its aberrations are not very aperture sensitive?

Canon CN-E 24mm T1.5

 

Sigma Cine 24mm FF T1.5

 

Rokinon Xeen 24mm T1.5

 

Zeiss CP.2 28mm T2.1

 

Zeiss CP.2 21mm T2.9

 

So What Did We Accomplish Today?

Well, I think it’s evident that the idea of testing multiple copies wide open and then doing more extensive tests on a ‘typical’ copy is worthwhile. As we all knew, stopping down improves all of the lenses, but it doesn’t do so equally. I’d think we should have added T2.9 comparisons, too. The Xeens, in particular, have changed so dramatically at T4 that I want to know how they look at T2.9.

Whether we test at T5.6 is less obvious to me, and time is money. I can see it being worthwhile for the T2.9 lenses to at least check the ‘two stops from wide-open is the sweet spot’ theory we all assume is correct. Honestly, I see no reason to test at smaller apertures than that. T8 may be a bit better for some lenses, but diffraction softening is going to be kicking in there for higher resolution cameras. I’m open to having my mind changed, but I need better arguments than ‘just cause.’

We also, as usually happens when we do more testing, added more questions. Why are some lenses not improving in some areas? What else can you tell me about a given lens that is useful and not shown by MTF curves? I’ve talked at length about how much more information we get when we do MTF field curvatures (they’re officially called MTF vs. Field vs. Focus). That’s too time-consuming a test to do at multiple rotations on multiple copies, but if we go with our ‘in-depth testing of a typical copy’ plan, it becomes reasonable and gives us a lot more information.

That’s what I’ll show you in the next post.

 

Roger Cicala, Aaron Closz, with invaluable input and programming from Brandon Dube and Max Bruggerman

Lensrentals.com

November, 2017