Equipment

More PhotoGeekery: Finite Conjugate MTF Bench Tests for Macro Lenses

For the last several years we’ve talked about the fact that lens performance can vary with focusing distance. Our optical bench tests at infinity. Computerized target analysis (Imatest and DxO) tests at close distances. But when the results of target analysis and bench tests differ it’s hard to decide if the difference is because of different testing methods or because lenses actually do perform differently close-up and at a distance.

When I did target analysis testing, I was most uncomfortable about wide-angle lenses. They are being tested at distances under 6 feet and that’s not the distance they’re often used at. With bench testing, I have the opposite discomfort. We were testing macro lenses at infinity, but they are most likely going to be used close up. In both cases, I wondered how much testing distance affected performance.

macro

Random macro image because social media consultants say we should have more pictures since reading is hard.

 

Faced with this dilemma, I did what I usually do; I spent more money and bought a finite conjugate modification for our optical bench. This allows us to test lenses at close focusing distance with the same methods we use to test at infinite focusing distance. So now we can use the same testing method to see how much difference focusing distance makes.

The finite conjugate is rather difficult to use, time consuming to test with, and it has some limitations. The most important of these restrictions is it only works at really close focusing distances; 3 feet or less. But that’s perfect for macro lens testing.

Remember, please, that we’re letting you look over our shoulder in nearly real-time. This whole piece of equipment is a new addition to our lab. Diego has spent most of the summer working the kinks out, and I’m very comfortable with the results we’re getting — right now. But a few thousand lens tests from now I may realize we should be doing something differently.

Some Lens Background Information

The vast majority of photography lenses are designed for best performance at infinity. When lenses are focusing close, we expect performance to deteriorate to some degree. Most photographers are aware of this and expect a lens to be softer near the minimum focusing distance. Field curvature can also change with focusing distance.

144464063.InFhjP0W.print5

Another one like the other one, but different brand to minimize Fanboy uproar.

 

Macro lenses, at least the better macro lenses, must keep good performance when focusing closely, so they should be as good, or even better at close distances than far away. They should also have a very flat field at close focusing distances.

Comparison of Macro Lenses at Infinity and at 1:2 magnification

We’ve tested five macro lenses both at infinity and at 1:2 magnification focusing distance. For all lenses, the infinity MTF is on the left, the 1:2 macro distance MTF is on the right. (For each lens, we tested 5 copies and averaged the results to obtain the MTF charts you’ll see below.) And, of course, you can click on the image to see it in more detail.

Canon 100mm f/2.8 IS L

OlafOpticalTesting.com

OlafOpticalTesting.com

The Canon 100mm f/2.8 L lens is a little better for low frequencies (red lines) at close distances than it is at infinity. Otherwise, it doesn’t change a lot.

 

Zeiss 100mm f/2 Macro Planar

OlafOpticalTesting.com

OlafOpticalTesting.com

 

The Zeiss stays the same at low frequencies, or maybe a hair better, while it falls off a bit at higher frequencies. But the flatness of the curves from center to edge, especially considering this lens is being tested at f/2, is just amazing.

 

Sony FE 90mm f/2.8 Macro

OlafOpticalTesting, 2016

OlafOpticalTesting, 2016

The Sony FE 90mm f/2.8 Macro totally shocked me. It’s clearly better at all frequencies at macro distances. In fact, it’s the highest resolving off all these lenses at macro distances, at least in the image center. I’m not certain if it’s related or just coincidence, but the Sony lens has electronic motors controlling both the focusing and the compensating element. As best I know, all of the other macro lenses tested have mechanical movements of the compensating elements.

(Note: as with all Sony FE bench tests we do, the 20mm results are probably inaccurate because we have far fewer measurements at 20mm because of the light baffle.)

 

Zeiss 50mm f/2 Makro-Planar

Zeiss50mmsml

OlafOpticalTesting, 2016

The Zeiss 50mm f/2 Makro-Planar behaves much like the 100mm. It’s better off-axis at macro distances, and about the same on axis.

 

Sigma 150mm f/2.8 APO Macro

OlafOpticalTesting.com

OlafOpticalTesting.com

The Sigma 150mm f/2.8 APO Macro also is better at close distances than it was at infinity, but it’s very good at both distances. It has a bit less astigmatism at closer distances.

Overall I was a bit surprised that some of the macro lenses were better at macro distances than at infinity. That’s what we would hope, of course, but I’m surprised that it happened. My expectation was that they might be as good, but not that they would be better.

Micro-Nikkor 105mm f/2.8 G VR

We didn’t have this one done in time to make the original article, but I know there is interest so I’ve added this. This graph was done after the others with a new widget Marcus programmed, that I think makes comparison easier. The Macro 1:2 distance is on the left, infinity on the right. Again, you can see performance is better at macro distances, particularly in the center.

Olaf Optical Testing, 2016

Olaf Optical Testing, 2016

Field Curvatures

We don’t average field curvature images from multiple lenses like we do with MTF data, there are just too many data points. These field curvatures are taken from a single copy, but we selected these from the batch we MTF tested, choosing average copies to show you.

Now we’re going to look behind the curtain a bit, so if you haven’t the stomach to look at actual single-lens images you might get queasy here. I see this all day, every day, but don’t write about it much because I’m afraid it will cause mass hysteria. But the reality is when we look at many copies of a lens, some of them are going to show field tilt, usually in the tangential plane. It’s a variation; just like MTF and aberrations vary a bit copy-to-copy so do field tilts.

A couple of the examples I’m going to show, do just that. DO NOT go online and look like a fool saying “Well, this lens has field tilt in Roger’s blog post.” Because then I’ll show you a couple of dozen examples of your favorite lens with more field tilt. No brand is without it, and trust me, none of these tilts are bad at all. It’s one of those unpleasant secrets like ‘there is no Santa Claus’ that we just have to learn to cope with.

For each lens, I’m going to show you a group of 4 field measurements: sagittal on the left, tangential on the right, infinity focus above and macro distance below.

Canon 100mm f/2.8 IS L

OlafOpticalTesting, 2016

OlafOpticalTesting, 2016

 

If you look closely, you should notice two things. First, at infinity, there is a bit of tilt to the tangential field that goes away close up. This is specific to this copy; one of the cams that move an element isn’t perfectly parallel to the axis of the lens, so there’s a bit of tilt at one end. This is pretty common. Every zoom has a bit of it, and some primes do too.

Second, if you ignore the tilt and look at the tangential field shape, you’ll see that the sagittal and tangential fields match up better at close distances. This probably has something to do with the change of ‘astigmatism-like’ separation on the MTF curves above. (I use the term astigmatism-like because lateral color and some other things affect that MTF pattern.)

 

 Zeiss 100mm f/2 Makro-Planar

OlafOpticalTesting, 2016

OlafOpticalTesting, 2016

 

In this copy there is a bit of field tilt that doesn’t change much with focusing distance, so it probably doesn’t involve a focusing element. The field curvature changes shape a bit as we go from infinity to macro distances, but if anything it’s a bit it’s pretty flat at either end, much like the MTF curves were.

 

Sony FE 90mm f/2.8 Macro

OlafOpticalTesting, 2016

OlafOpticalTesting, 2016

Here we have another example of tilt in the tangential field that isn’t changing with focusing distance. If you mentally rotate that tangential infinity curve, though, you may visualize that while both sagittal and tangential fields change shape at closer distances, they do it in the same way.

 

Zeiss 50mm f/2 Makro-Planar

zeiss50sml


OlafOpticalTesting, 2016

 

It’s an interesting exercise to compare the change in field curvature with this lens to the change in astigmatism-like pattern in its MTF charts. While the field doesn’t flatten at close distances, the sagittal and tangential fields do become more similar.

 

Sigma 150mm f/2.8 APO Macro

OlafOpticalTesting, 2016

OlafOpticalTesting, 2016

This lens gives us another example of a field tilt that only shows up at one end of the focusing range, and also of field curvature that changes the pattern a little bit and matches up better at the macro than at the longer end. Look back at the MTF charts to see how that affects astigmatism with this lens.

 

So What Did We Learn Today?

Well, first and foremost I think we confirmed that I’m overly cynical. The macro lenses we tested were all at least as good at macro distances, and in some cases better than they were at infinity focus. Like they are supposed to be. And like I thought they wouldn’t be.

We also confirmed that the shape of field curvature does change a bit at different focusing distances. And I learned that lens designers apparently take this into account to flatten the field at macro shooting distances. (Yay lens designers!!!)

Some of you probably learned that lenses vary more than you thought they did, at least when you test with lab equipment. You’ll have to take my word for it, but these are all really minor field tilts that you would not notice in a photograph. If you were pixel peeping with careful test chart or brick wall pictures, you might be able to detect the tilt if you had several copies to compare, and a lot of free time.

We’ll probably show you some more impressive tilts with zoom lenses one of these days. I guarantee most of you will get nauseous when you see how your ‘good as a prime’ zoom lens does with field tilts.

 

Roger Cicala, Aaron Closz, and Diego Martinez

Lensrentals.com

June, 2016.

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Equipment
  • We can’t; it won’t mount properly on our bench.

  • Ralph Hightower

    Roger C,

    Can OLAF test the Canon Macrophoto 20mm f3.5? A bellows is required for focusing.

    http://global.canon/en/c-museum/product/sp8.html

  • Robert York

    Indeed, he claims to be unbiased, but look at the pictures. Notice the solder joints. The Canon board is showing rough flow on the solder, while the Nikon has nice shine and uniform solder. Clearly the superior camera.

  • Thanks for clearing up the confusion :).

  • I’m hiking 2650+ miles across America; the weight and size is an issue for me. I would still consider this lens at a later date, if it were an exceptional lens.

    I get the impression it’s not; it’s a 15 element lens (astronomically high for a prime where the sensor to mount distance is so low). Anecdotal information regarding colours doesn’t sway me.

    “But if you don’t want to buy it – don’t test it ;-)”

    I’ll do what I like thanks.

  • Perhaps. I think that they may well be reflective spurious data as an alternative – out of focus and near the edge of the lens, we might be getting a reflection off of our equipment or the lens.

  • obican

    Don’t take it personally, he does that to absolutely everything even remotely related to Sony.

  • I guess the islands of sharpness are “spurious resolution” or “nisen bokeh” or “nisei bokeh”
    http://toothwalker.org/optics/spurious.html
    Thoughts?

  • Kers

    Very unique informative test; A lot of good lenses;
    Indeed to test a macro at macro-distance is a good idea, but i am even more interested in macro with F5.6 -f11 for these are the working apertures ; hardly ever macro lenses are used wide open. Also similar F-numbers makes it a more equal comparison.

  • JJ

    Hi,
    The 90mm FE is big, but the weight is very well balanced (not front heavy), so it’s a pleasure to use on my A7s. Manual focus is well implemented and handling is great, flare is well controled and colours/contrast remind me a little to Minolta – which is not a bad thing at all.

    Btw focus breathing is quite pronounced with this lens – would be interested in the real focal length at different distances.

    But if you don’t want to buy it – don’t test it 😉

    Roger – thanks for the great work! This blog is one of the best sources of reliable information all over the net.

  • Hi Phillip,

    SAR has the habit of putting screaming headlines linked to my articles with a ‘click to buy’ link right below. I’m not a lens reviewer, I have neither the patience or the skill set. I simply provide a bit of a different testing perspective on one aspect, doing MTF testing on multiple copies on an optical bench. It’s much more a site for photogeeks and not at all a review site.

    We’ve taken apart some 90mms, but not published a teardown, except, I believe in our article on electromagnetic focusing motors, it was one of the examples.

    Roger

  • I was linked here from SAR — “Lensrentals big macro test: 90mm FE macro beats all other Nikon-Canon-Zeiss lenses…”

    Obviously you’re interested in objective data being compared as opposed to subjective attributes such as bokeh; I respect that, and I won’t ask you to compare bokeh rings. However, it is easy for people to lack information and then draw the conclusion that you’re claiming “Lens A is better than Lens B”. Whereas you might be claiming “Lens A’s sharpness is better than Lens B at X distance with X aperture”.

    You’ve examined a few attributes of a few different lenses (thanks for doing that; I find it a laborious task) but would you be interested in examining more attributes of a lens (assuming you haven’t done so… I’ve not looked around this site much)? For example, focus breathing, colour rendition, flaring, etc.

    I shoot Sony, so naturally I am somewhat interested in the 90mm FE lens but a part of me believes Zeiss can make something with less elements, better colours, better flare control and with half the volume. Most importantly, I find the 90mm FE lens a bit too big so I don’t think I will buy it.

    Slightly off topic, but do you have a lens disassembly of the 90mm FE lens? In my experience, autofocus to manual focus mechanisms like the one in the Sony are rarely durable.

  • Bill

    Here’s what I read above, regarding the Sony 90mm macro: “In fact, it’s the highest resolving off all these lenses at macro distances, at least in the image center.”

  • Hakann Vatansever

    you’re so right about wide angle lenses tested at close distances…such as batis25, fuji XF16 that i’ve both…wide open they’re great at close distances (batis is much greater) but wide open at infinity, resolution is simply way lower than close distance resolution..photos are totally useless wide open at infinity/far focus distances which i bought these lenses for.

  • King

    the sony rumors admin is already misquoting you with a stupid headline.
    maybe i missed it but did you say the sony 90mm is the best macro you have tested?

  • Freddo

    “Random macro image because social media consultants say we should have more pictures since reading is hard.”

    This made me laugh so much, reading the full article became hard. 🙂

  • I wouldn’t expect a great deal of change, but we haven’t done it so I’m not certain.

  • yaley

    What would happen if you tested them at 1:1 distance? At least the ones that go that far.

  • Maya

    Thanks. That’s what I thought as well. Anyway, that article was great !

  • Maya, I think it’s out of the realm of purposeful correction. They could replace the cam and or barrel and things would be randomly different: maybe better, maybe worse, but that’s not an area, at least in any lens I know of, that is adjustable.

  • Maya

    Interesting article. I’m looking forward to see more nauseating field curvature tests :D.
    Do you think that a tilt in one plane at one end is the sort of thing lens manufacturers’ customer services can correct or is it beyond realistic expectations ? For example, in the case of the Canon, by replacing / correcting the cam ?

  • l_d_allan

    RC > I was most uncomfortable about wide-angle lenses. They are being tested at distances under 6 feet and that’s not the distance they’re often used at.

    FWIW: I incorporated the idea of “tilted near-infinity horizon lens testing” (TNIHLT) with a large brick wall … two story exterior of a middle school gymnasium. I figure there was about a 500x Focal Length magnification factor for a 14mm ultra-wide angle.

    http://berean.zenfolio.com/160816_brick_wall/h734e70a5#h734e70a5

    The “baseline” along the wall from far left target to far right target is about 60′. The tripod is located about 20′ from the wall. With the 14mm lens … about 1/2″ … that works out to about 500x focal length … if I did the math correctly.

  • Chris, We’ve just done a very little testing at that but the ones we’ve done really don’t seem much, if any, worse close up. BUT all we’ve really looked at are newer, top end primes. I suspect most of those are very well compensated. We might see more differences with older designs or zooms.

  • Speedy, guess what cameras are usually left on the shelf when I go grab something to take pictures with 🙂

  • Claudia Muster

    Thanks for this article. Contrary to you, I’m not surprised at all. Considering the huge complexity of lens design, it’s pretty much expectable that lenses perform quite differently at different focus distances. BTW, that’s one reason why I don’t give any value to those Imatest review sites: If at all, they say something about the lens at more or less headshot distance (they usually don’t even bother to declare the applied focus distance), but nothing at all about how it performs at infinity or at macro distance. (Likewise, usually none of these sites discloses whether their off-center MTF values are actual unsharpness just field curvature. And much more, but this would be a veeeery long rant.)

  • Brandon Dube

    There are a lot of aberrations. Each aberration even has multiple flavors.

    Let’s say the lens only has coma, which is linear in field of view. There is “basic coma” which is linear, and there is “higher order” coma which is or quadratic in field of view. Let’s also say the two have opposite sign. In the middle, the lens is “bad” because of the low order coma. In the edges it’s really bad because of the high order coma. Somewhere in the middle it is very good because they are cancelling each other out.

    Of course, in a real lens there are around 8 significant aberrations away from the middle and they can interact, but you get the idea.

  • Taki

    Hi Roger

    I’ve been anxiously waiting for this article and it’s wonderful that you’ve done it so quickly. And how encouraging the results are, these lens designers really do amazing work!

    Thanks again.

  • speedy fisher

    You talk about being unbiased, but of of those photos were taken with a Canon camera! Shill! Seriously though, awesome work as usual.

    One question about the field curvature charts though. I understand what sort of thing could cause there to be curvature in the first place, I understand why the curvature would be tilted, I understand why a lens would be sharper away from the middle. I have no idea though what would cause the “islands” of sharpness which can best be seen in the Canon macro tangential sample though.

    Assuming I have the slightest clue what I’m on about is this saying that if we move an object at the “ideal” focus point towards the lens it will gradually go out of focus (as you’d expect) before coming slightly back more into focus again? Is there some relatively simple reason for this? Obviously I understand we’re talking about tiny differences at tiny distances that I wouldn’t notice in a real photo, but data is data and it just seems interesting.

  • Chris Jankowski

    Do we really know that non-macro lenses of similar specifications i.e. 50mm, 100mm and 135mm either F2 or F2.8 are indeed worse at their minimum distances than they are at infinity?

    Canon EF 135mm f/2L USM would be a good guinea pig due to its popularity.

  • This is excellent! Thanks Roger, Aaron & Diego!

Follow on Feedly