fbpx
Geek Articles

More Ultra High-Resolution MTF Experiments

GEEK ALERT!!

Let’s be absolutely clear; this is not a practical or useful article. It won’t help your photography or cinematography become better. It won’t help you choose equipment any time in the next couple of years. It won’t provide any fodder for your next Forum War. It’s just a geek article that may interest some people. It may give a little peek into what may come in the future, and some insight into the kind of work we’re actually doing behind the scenes at Olaf. So if you’re interested in that kind of stuff, read along

A couple of years ago, a testing customer asked us to find which lenses could get maximum resolution from a 150-megapixel sensor. Many people assumed that the highest resolving lenses at standard resolutions would be the highest resolving lenses at higher resolutions. Assumptions are the dark matter of the internet; we can’t see them, but we know they account for most of the mass.

We try hard not to assume, so we tested a bunch of lenses at high frequencies on the MTF bench (high-frequency MTF is basically high resolution on the camera). This requires testing MTF at ultra-high resolutions, way higher than any camera of today. The manufacturer wanted 240 lp/mm (compared to the 50 lp/mm we use currently). I wasn’t sure that was necessary, and actually wasn’t sure we could do it, so we settled on 200 lp/mm. If you want some understanding of what all that means, you can read the above post or Brandon Dube’s excellent post. Or you can just accept it and move on.

There was no photo or video lens that could resolve 200 lp/mm wide-open. (Our standard for ‘resolve’ was an MTF 0f 0.3; an MTF of 0.2 was borderline. There’s some evidence to support those cut-offs, but someone could argue them. Wait, this is the internet. Someone WILL argue them; it’s what someone lives for.)

We did find several prime lenses that could meet those criteria stopped down to f/4 in the center of the image, but none could near the edges. The best results were for the Zeiss Otus 85mm f/1.4 lens at f/4. A few other lenses (Zeiss 135mm f/2 APO-Sonnar; Sigma 135mm f/1.8 DG HSM Art; Zeiss 55mm f1.4 Otus) were acceptable at f/4 in the middle portion of the image. Nothing wider than 50mm was really acceptable, although the Canon 35mm f1.4 Mk II and Sigma 35mm f/1.4 Art were close.

Back to the Future

Two years later, that customer asked us if we knew of any other lenses that they should consider. There’ve been a lot of lenses released since we did these tests, and some of those lenses fit the criteria for possible ‘ultra-high’ resolution; primes with focal lengths of 85mm or more. The manufacturers are obviously making these lenses with at least moderately higher resolution cameras in mind. So perhaps some of the newer lenses would resolve ‘ultra-high’ frequencies better than some of the older lenses we had tested.

So we checked some new lenses all the way up to 240 lp/mm, something sufficient to make a 200 megapixel FF camera worthwhile. To be clear, this is NOT coming to a camera near you anytime soon; it’s a research project. But if the researchers are making such a sensor, it makes sense they want to know which lenses would get the best results from the sensor. It is not a test for if the lens or sensor out resolves the other, because that doesn’t happen.  (For those of you who believe in perceptual megapixels or that the earth is flat, I’ve included an appendix to clarify this a bit – with no math more complicated than multiplication.)

The Best We Found Last Time

Last time we found some tendencies: longer focal lengths perform better, and f/4 is the minimum aperture that any lens can resolve such high frequencies. To give you a reference, here are several lenses that met our criteria last time (notice on these we set the highest frequency at 200 lp/mm).

Sigma 135mm f/1.8 DG HSM Art at f/4

Lensrentals.com, 2017

Zeiss 85mm APO-Planar Otus at f/4

Lensrentals.com, 2017

Zeiss 55mm Otus Distagon at f/4

Lensrentals.com, 2017

Canon EF 35mm f/1.4 Mk II USM

Lensrentals.com, 2017

The 35mm Lens Whose Name Cannot Be Spoken

You’ve seen the best results we had from lenses you could actually buy and use. We did see one lens that you can’t purchase, a prototype lens that will probably never be made, that was truly amazing, particularly for a 35mm focal length.

Olaf Optical Testing, 2017

So here is a 35mm lens that is as good as the 55m Otus; it’s that it’s this good at f/2.8 and doesn’t improve much at f/4. I put this up just to show that current lenses are not designed to do their best at ultra-high resolution, but they could be.

 We Wanna Take it Higher

Last time around, we set our peak at 200 lp/mm, even though the client really wanted 240 lp/mm. We didn’t think the higher number was really necessary, and honestly, we weren’t sure that any of our lenses would adequately resolve even 200 lp/mm. This time, though, we felt comfortable we could test at 240 lp, and doing that, let us also test at 192 lp/mm, which is pretty close to the original 200 lp peak.

For starters, we repeated the test using a Sigma 135mm f1.8 Art. You can compare it to the one done at 200 lp up above. Remember, these are single copy tests, so there is a little sample variation, but you can see the light blue line of this run (192 lp/mm) is comparable to the purple line of the previous lens (200 lp/mm). Even at 240 lp/mm, the MTF still exceeds our ‘borderline’ MTF cut-off of 0.2, at least in the center, but it doesn’t quite reach and MTF of 0.3.

Lensrentals.com, 2019

We’ll consider this our standard for the new tests, then, and see if any of the other lenses do as well.

Results for New Lenses

Sony 135mm f1.8 GM

We’ll start with the usual “Roger had expectations and was disappointed” part, because, after all, disappointment is the sole purpose of expectations. One of the reasons I was excited about testing new lenses was that the Sony 135mm f/1.8 GM had the best normal-range MTF scores we’d ever seen, besting the Sigma 135mm f/1.8 Art. It seemed logical that it might also best the Sigma in ultra-high resolution tests. Reality 1, Logic 0. We tried the lens at f/4, but it was actually a bit better at f/5, which we show below. At high frequency, it’s not as good as the Sigma.

Lensrentals.com, 2019

Let me repeat, for those of you who want to mistake this test as having something to do with, say, the 60-megapixel full-frame camera you’re shooting; it doesn’t. Somewhere around 80 lp/mm would be more than sufficient for that. If you compare the orange lines of the Sony and Sigma 135mm graphs, you’ll see that at 96 lp/mm the Sony is actually a bit better than the Sigma. At ridiculously high frequencies, the Sigma is better. The takeaway message is important: better MTF at one frequency doesn’t mean better MTF at all frequencies.

So let’s look at a couple of other candidates I thought might do really well.

Sigma 105mm f1.4 Art at f/4

Even though this copy was slightly tilted, in the center, at least, it is the first lens to make, albeit barely make, our ‘acceptable’ cut-off of MTF 0.3 at 240 lp/mm. At 192 lp/mm, it actually touches MTF of 0.4. So we have a new high-resolution champion, and that’s despite, as you can see below, this copy of the lens having a slight tilt.

Lensrentals.com, 2019

Zeiss APO Sonnar 100mm f/1.4 Otus at f/4

Another good contender; not quite as good as the Sigma 105mm, but very similar to the Sigma 135mm.

Lensrentals.com, 2019

Canon 90mm f2.8L TS-E Macro

This one I had low expectations for, but we had one handy, so we thought we’d give it a try. Set your expectations low, and they will be met. Again, don’t get me wrong, this is a really, really good lens, it’s just not as good at ultra-high resolutions.

Lensrentals.com, 2019

Sigma 40mm f1.4 Art

We learned from earlier testing that wider-angle lenses don’t do as well at these frequencies. But the Sigma 40mm Art tested so spectacularly at normal MTF that we thought it was worth a shot, at least.

Lensrentals.com, 2019

Here’s a case where good at normal resolution translates to good at ridiculously high resolution. This is close to, although not quite as high in the center as the Otus 55mm. It stays acceptable further away from the center than the Otus 55mm does, however.

Summary:

I say summary, because there are no practical or useful conclusions to be made. The only thing of interest, probably, is that only really good lenses can resolve ultra-high resolutions you’ll never need. However, even among these really good lenses, you can’t assume how a lens will perform at ultra-high resolutions based on its results at normal resolutions. You can also see that ultra-high-resolution performance is a bit easier to obtain in short telephoto focal lengths than in standard or wide-angle lengths.

Oh, yeah, and you can also wonder why someone, somewhere, is wondering what lenses will perform well at resolutions more than twice as high as what you might need today.

 

Roger Cicala, Aaron Closz, and Brandon Dube

Lensrentals.com

October, 2019

Appendix: Why Perceptual Megapixels are Stupid

I get asked several times a week if this lens or that is ‘capable of resolving’ this number of megapixels. Some people seem to think a lens should be ‘certified’ for a certain number of pixels or something. That’s not how it works. That’s not how any of it works.

How it does work is this. Any image you capture is not as sharp as reality. Take a picture of a bush and enlarge it to 100%. You probably can’t see if there are ants on the leaves. But in reality, you could walk over to the bush (enlarge it if you will) and see if there are ants by looking at a couple of leaves.

What if I got a better camera and a better lens? Well, theoretically, things would be so good I could see the ants if I enlarged the image enough. MTF is somewhat of a measurement of how sharp that image would be and how much detail it contains. (The detail part would be the higher frequency MTF.) That would, of course, be the MTF of the entire system, camera, and lens.

Lots of people think that will be ‘whichever is less of the camera and lens.’ For example, my camera can resolve 61 megapixels, but my lens can only resolve 30 megapixels, so all I can see is 30 megapixels.

That’s not how it works. How it does work is very simple math: System MTF = Camera MTF x Lens MTF. MTF maxes at 1.0 because 1.0 is perfect. So let’s say my camera MTF is 0.7, and my lens MTF is 0.7, then my system MTF is 0.49 (Lens MTF x Camera MTF). This is actually a pretty reasonable system.

Now, let’s say I get a much better camera with much higher resolution; the camera MTF is 0.9. The system MTF with the same lens also increases: 0.7 X 0.9 = 0.63. On the other hand, I could do the same thing if I bought a much better lens and kept it on the same camera. The camera basically never ‘out resolves the lens.’

You could kind of get that ‘perceptual megapixel’ thing if either the lens (or the camera) really sucks. Let say we were using a crappy kit zoom lens with an MTF of 0.3. With the old camera; 0.3 X 0.7 =.21. Let’s spend a fortune on the newer, better camera, and we get 0.3 X 0.9 = 0.27. So our overall system MTF only went up a bit (0.07) because the lens really sucked. But if it had been just an average lens or a better lens (let say the MTF was 0.6 or 0.8), we’d have gotten a pretty similar improvement.

If you have a reasonably good lens and/or a reasonably good camera, upgrading either one upgrades your images. If you ask something like ‘is my camera going to out resolve this lens’ you sound silly.

Roger’s rule: If you have either a crappy lens or crappy camera, improve the crappy part first; you get more bang for your $. I just saw a thread for someone wanting to upgrade to the newest 60-megapixel camera, and all of his lenses were average zooms. I got nauseous.

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Geek Articles
  • P. Jasinski

    Thanks! Read through the background so far – very interesting.

  • The “edge” of a 24x36mm sensor would be 18mm from the center. The diagonal of a 24x36mm sensor is 43.3mm (thanks, Pythagorean Theorem!). So the 20mm in the plot is equivalent to the corner-ish, but not the absolute tip of the corner. 21.6mm would be the absolute corner.

  • There’s also the part about if you’re using autofocus, what exactly is it autofocusing? For many camera manufacturers AF is set using a bar chart that probably is the equivalent of 10 or 20 lp/mm. I don’t pretend to know this stuff, it’s proprietary and for phase-detection, at least, I don’t understand it well enough to say. But it’s an interesting question.

  • Brandon has spoken, so it shall be done! This is the sagittal image plane for the Sigma 35mm at f5.6. Top to bottom 20, 50, 100, and 150 lp/mm. At 200 lp/mm we didn’t have MTF of 0.2 so the graph really is just a little purple fuzz. You can see how the peak MTF lowers from yellow (>0.9) through orange (>0.7) to purple (>0.4) as frequency increases, and the depth of field narrows.

    https://uploads.disquscdn.com/images/80c77b17c241f191a176c133da0a9c0d11011c5396e3934f95d70ae0a326ca3b.jpg

  • OK, and by “edge” do you mean laterally straight to the “edge” of a full-frame image, not a corner? Somehow I was adding 20mm and 20mm and asking myself if you were, in fact, testing the entire length of the intended image circle, and your “edge” is actually a “corner” in full-frame terms.

  • Dragon

    Thanks for the reference. When I have some time will play with that. You are a spoilsport, though. I was trying to wind Roger’s spring.

  • Dragon

    Due to aliasing, the output above Nyquist is not zero in your example, but it is junk.

  • Carleton Foxx

    It is my understanding that “diffraction limited” is not the demon we have been taught because many sharpening algos can easily deal with it now. From what I’ve read, diffraction is predictable so removing its effect is trivial.
    Or is that just an internet fairy tale?

  • DrJon

    I currently have the Star button set to pinpoint AF with a general-purpose tracking algorithm, partly for Owls in trees or anywhere you want accurate AF on a single thing. The AF-On button has tracking optimised for BIF and all the AF points. It’s very handy to just be able to move your thumb sideways and completely change the AF setup.
    I’ll certainly consider a high-MP RF body, although it depends on other things. What I’d really like is a combination of my 5Dsr with a deeper buffer and an extra fps or few and my GH5, so IBIS, 10-bit video, etc. However currently people don’t seem to be able to read high-MP sensors fast enough to do really good video, and people have serious heating issues with some more ordinary sensors (fan in the latest Panasonic, large heatsink in the Sigma).

  • intrnst

    “Assumptions are the dark matter of the internet; we can’t see them, but we know they account for most of the mass.”
    “Disappointment is the sole purpose of expectations”

    – Roger, if I use them, should I quote you or it’s public domain*?

    * something that I, with a nonchalantly vague grin, will take the credit for.

  • Larry, my major problem is it gives a number that takes into account: 1- lens performance at all apertures, 2- lens performance all across the field, 3- for a zoom performance at all focal lengths, 4 – a formula of calculation that is not shared (but apparently is normalized down to a lowest common denominator, than multiplied up to a sensor in question, which is pretty insane), 5- a method of measurement that isn’t shared and cannot be confirmed or denied by anyone else, and 6 – for lenses that work on multiple cameras and mounts the calculations are done on one camera and mount and therefore aren’t valid.

    So we have this number that equates to something but we aren’t sure what, based largely on measurements that we definitely don’t know what, and people are making purchasing decisions based on that.

    More to the point, people are misinterpreting it to say things like ‘this lens is rated at 16 megapixels so if I buy a 42 megapixel camera my images won’t improve’ when, in fact, they will.

  • Nicholas Bedworth

    Right, the spectral distribution… Wavelength is perhaps an imprecise manner of expressing the idea, as compared with spatial frequency distribution. Something like that… 🙂

  • Brandon Dube

    If you want to do a simulation yourself, see prysm.otf.diffraction_limited_mtf, but diffraction limited at F/4 is a little higher than .4 for green-yellow light.

  • Brandon Dube

    If you google “on the use of classical mtf measurements to perform wavefront sensing” you will find my thesis. In it, you will find color surfaces of the MTF vs focus and frequency for some aberrations (see, e.g., fig 24). Curvature in the MTF vs Focus vs Frequency is a signature of spherical aberration. There is an intuitive physical optics explanation for it – email me brandon at retrorefractions dot com if you want to hear it.

  • Brandon Dube

    Our standard MTFvFvF measurements record up to 250 lp or so. Been a few years since I looked. Roger just has to change a setting on the GUI to a much larger number 🙂

  • Brandon Dube

    No. MTF = |OTF|. PTF = angle(OTF). M is for Modulation, not real. If you ever see a negative MTF, someone has made a basic error. With defocus, the real part of the OTF can be negative.

    For modern, high performance detectors — a regime consumer imaging lives entirely within — the pixel aperture MTF dominates everything by the lens by a spectacular margin.

  • Brandon Dube

    My python package can calculate the MTF associated with seeing (turbulence). See prysm.otf.longexposure_otf.

  • I don’t know how helpful a “takedown of perceptual megapixels” really is. Reading that section it felt like there was an assumption that anyone working with the “silly” (read: utterly moronic) idea that a lens can out-resolve a lens (or vv.) is blatantly trying to disprove sacred optical principles, or make a claim as outlandish as declaring the Earth to be flat.

    Using DxOMark and their perceptual megapixel ratings requires matching up a camera and a lens. So if someone wants to know if X lens will net him more resolution on A camera, the perceptual megapixels rating he’s working off of seems similar to the (camera x lens) equation referenced above. Is “perceptual megapixels” an imbecilic, uneducated and foolish term? It sure sounds like it. But despite the dim-witted grandstanding that‘s implied by using such a silly term, it seems like we’re still *somehow* working out the ‘better camera’ and ‘better lens’ equation in a way that works for our pedestrian needs.

  • Dragon

    Also interesting that most of the lenses you tested have the highest MTF (at the highest resolutions) at f stops that put the system into pretty severe diffraction limiting.

  • Dragon

    One interesting side note. At these resolutions, the atmosphere becomes a limiting factor in system MTF much sooner than it does at lower resolutions. I have an 800mm L, which on a good day produces awesome results with a 5DSR. I have observed that there aren’t very many “good days” (at least for distances of over a few hundred feet). At 200 MP (FF referred), that distance will reduce to maybe 100 ft. unless you live in high desert country. When you concatenate that with your observation that shorter lenses are not as good as longer ones at very high resolutions, that results in a 200 MP camera that may be useful (most of the time) with lenses from maybe 40mm to 85mm. Kind of like flying a U-2 at 80,000ft. At 49 knots indicated airspeed it stalls and falls out of the sky and at 51 knots indicated it goes supersonic and the wings rip off and it falls out of the sky. So here is a challenge for you Roger – given the limits of lens design and the atmosphere, what focal length is exactly at the top of the bell curve for system MTF (given a camera body with an arbitrarily high number of pixels)? :-).

  • Dragon

    Those settings sound quite useful for fast moving stuff (like swallows, hummingbirds, and bees)
    It will be interesting to see what the next high res body looks like. By most accounts, it will be an R, but I am a bit torn as I am not so sure but what I would rather see a 5DSR mark II. When the target is moving fast, EVF lag (even when it is short) can make tracking a challenge. OTOH the focus tracking algorithm seems to be leaning toward dual pixel winning out over an OFV sensor. At least that is the case with the 90D and I suspect that may be instructive re the future. Also, an R body would be more likely to have IBIS which could be very helpful with that much resolution. Either way, if Canon makes an 83MP FF sensor that has the pixel performance of the 90D, I will preorder the camera.

  • Nicholas Bedworth

    Correct. The MTF is the real component of the OTF; the Phase Transfer Function PTF is the imaginary component… There are many physical and electronic elements that contribute to these values and thus the overall performance of the system for a given application.

    For example, there may be fine diffraction patterns etched onto the back of the final lens to shape the PSF for subsequent processing. And of course there’s the Bayer color filters, and the numerous methods for forming RGB channels from the respective pixels. Pixel size, acceptance angle, quantizing noise, readout noise, dark current, clock jitter and on and on and on all help determine the overall OTF/MTF/PTF of an optical system, from the real world scene to the RAW file. Simplistic discussion of pixel size, etc., is inadequate, and is just a small part of the overall picture, as it were.

  • P. Jasinski

    Wait… What?
    Does this frequency related focus shift effect has any practical implications for photography at standard set of frequencies? like 20 vs 50 lp/mm.

    I’m thinking – product/fashion photos – would it be visible in fabric structure versus, say, perceived sharpness of facial features of models. Would “subject” distance be a significant factor here?

    This is truly fascinating.

  • Erik Kaffehr

    On the perceptual megapixels thing.

    I would think, that looking at the sensor and lens as a system, it may be meaningful to talk about megapixels of resolution. The way I would use it is to calculate the number of pixels that deliver say 20% MTF. To that correctly, we would need to measure MTF on the whole sensor, which is a bit difficult as it needs perfect alignment and a very large test target.

    But, I would thing it is a useful measure for image degradation. Say that the lens can deliver 50 MP (at 20% MTF) at f/4 but only 34 MP (at 20% MTF) at f/11.

    On the other hand, I don’t think simple figures of merit are relevant, anyway. I would guess that we need the MTF across the field at diffferent frequencies. That also gives some hint about field curvature.

    Unfortunately, it seems that field curvature varies with aperture. That really means that we may need far more data than what normally is available to make best use of our lenses.

    Best regards
    Erik

  • DrJon

    I did like Darkons… 🙂
    Oh and the 5Dsr is a great camera, there’s still nothing I’d change mine for (currently). I also love using two buttons for BBF (with completely different AF recipes, inc tracking options – e.g. “My current AF Settings (subject to change)…” is a slightly older version, search with quotes) and am not sure I’d want to give that up, which would rule out a number of future possibilities.
    My Bee shot on dpreview should be found from “Bee… 100-400 (on 5Dsr)” which gets you to the thread and shows what we can do with all those pixels (the most indented post, in threaded view, has the Bee closeup).

  • bokesan

    For 35mm you have many options. When the Otus line was released (roughly $3000-4000) who would have thought you’d get roughly the same quality for a quarter of the price one or two years later from Sigma? And the 50mm Z Nikkor costs 380€ currently. Quite affordable I’d say. And even some of the cheap Chinese brands are showing quite good optical quality. My feeling is that excellent quality is getting quite affordable these days, and it’s a trend that will continue. (That might not apply in the same amount to zooms and wide-angles).

  • Keith Cooper

    How do multi-shot sensor shift technologies affect the ‘number of MP’?
    I tried out the Panasonic S1R with amongst others, the TS-E50 and there certainly is quite a bump in resolution at ~180MP (the multi-shot output size)

    Also thanks for sticking the boot into that old ‘sensors out-resolving lenses’ hokum – it’s appeared at almost every increase in Camera MP for over 15 yrs now…

  • 1.4 vs 4.0

  • bokesan

    Isn’t this already the case in the normal (10-50 lp/mm) MTF graphs? E.g. lens A might be higher than lens B at 20, but lower at 50? I thought that would be one of the things that experienced MTF graph readers (not me) can tell at a glance.

  • bokesan

    Thanks, that’s a relief. I’d love to see an MTF vs Field vs Focus graph for one of these. I know it takes a lot of time – one sample (say the 55 Otus) to see that things are not that bad off-center would suffice.

Follow on Feedly