Resolution Tests

The Great 50mm Shootout

Published January 2, 2012

Every so often the Universe realizes I’m getting a little cocky and sends someone to ask me a simple question I can’t answer.  It happened again the other day. Michael Plumridge and Peter Lik asked Tyler what the sharpest 50mm lens was. They were shooting on Red Epics with adapters so neither brand, mount, nor price mattered. But they needed to know right away. Tyler told them he’d ask me and give them the answer in a few minutes.

But I didn’t have a clue. I have tons of data on 50mm lenses for Nikon and Canon cameras, but all of that was shot at widest aperture, which is great for finding lenses with problems, but not great for finding which is sharpest at f/4. And we had no factual data for Leica mount lenses because Rangefinder cameras are very difficult to test using programs like Imatest. But when someone like Peter Lik wants an answer and tells you Jim Jannard is curious, too, you get them an answer. So we tested every kind of 50mm lens we had available and got a pretty clear answer by the end of the day.

As always, my first thought when presented with a bunch of work to do is “what’s in it for me”? Turns out not much. Tyler got a beautiful signed picture that hangs in his office so I can see it every time I walk by. I got a nice thank you email. But, I figured if I did all that work, I’d at least  make a blog post out of it so I didn’t have to do a bunch of research over the holiday weekend. Wait, I mean, I was certain other people would be interested in what I found, so I thought I’d share the results.

What This Is, and Is NOT

This is the MTF50, measured by Imatest, of a lot of 50mm lenses to determine their relative sharpness. (Actually we’re determining resolution and acutance, but since using that word immediately loses half of all readers, I’ll stick with sharpness.) The working distance for our 50mm Imatest setup is about 15 feet, so actually we’re only measuring sharpness at that distance. The lenses might perform somewhat differently at infinity, or very close up.

It is not an attempt to determine the best lens. No real-world photographs are taken. Bokeh is not analyzed and compared. Color rendering, autofocus accuracy, chromatic aberration, build quality, flare resistance, and a dozen other real-world considerations aren’t considered at all. I also want to emphasize this little test is of one copy of each lens. My preference would be to test a dozen or more copies of each lens and average the results. But there wasn’t time to do that kind of testing.

Still, it’s an interesting test. Some of these lenses have never (to the best of my knowledge) had MTF data released before. It also gave us a chance to compare SLR lenses mounted on Canon and Nikon cameras with M mount lenses on an M9.

The Methods

We used our standard Imatest  testing protocol for the SLR cameras: SLR cameras were manually focused using Live View and a Seimen’s star chart. Several exposures were made with each lens at each aperture and the best result at each aperture used. M9s were focused with the camera’s rangefinder and then several focus bracketed shots taken. As with the SLRs, we used the best result at each aperture.

Two measurements were recorded: the MTF 50 at the center (peak resolution) and the weighted average of center, midrange, and corner sharpness. MTF 50 results are presented as Line Pairs / Image Height from completely unsharpened RAW images. (We’re starting a pool about how many emails I get asking “what do the numbers mean” because no one ever reads this stuff. The over-under is 32 if you want in.)

We used 3 cameras for this exercise, all of them test cameras that we have used extensively at Lensrentals: a Canon 5D Mk II, a Nikon D3x, and a Leica M9. From extensive (a few thousand) previous tests, we knew that the Canon and Nikon cameras have identical resolution as far as our testing can tell. (The D3x has more pixels and in theory should resolve a bit better, but if it does we can’t detect it.) We had no idea about how the M9 would compare: it has fewer pixels but its lack of an AA filter should compensate for that to some degree.

The lenses tested were as follows:

Canon: 50mm f/1.2; 50mm f/1.4; 45mm TS-E; 50mm f1.0 (that is correct, the 50mm f1.0)

Cooke: 50mm Panchro T2.8 (Cinema lens)

Leica: 50mm f/0.95 Noctilux; 50mm f/1.4 Summilux ASPH; 50mm f2.0 Summicron

Nikon: 50mm f1.4 G; 50mm f1.2 AIS (another classic); 45mm PC-E

Schneider: 50mm f/2.8 PCTS; 50mm Cine Xenar T2.0 (cinema lens)

Sigma: 50mm f/1.4

Voigtlander: 50mm f1.1 Nokton

Zeiss: 50mm f/1.4 (ZE/ZF); 5omm f/2.0 Makro planar (ZE/ZF); 50mm f/1.5 Sonnar (M mount); 50mm f/2.0 Planar (M mount)

Interpreting the Results

Sample variation

Whenever we look at numerical results from Imatest or similar testing there’s a tendency to get all absolute: lens A resolved 600 LP/IH and lens B resolved 570, therefore lens A is the sharper lens. In the image below I’ve graphed the average (mean) MTF 50 (the vertical axis is weighted average for the entire lens, the horizontal axis is spot MTF from the center of the lens) for a number of 50mm lenses we have tested extensively. Note these results are all at each lenses “wide open” aperture.

Looking at the graph the Zeiss f/2 Makro is obviously sharper, but it’s being shot at f/2, so that’s not a fair comparison. Just looking at these single numbers it’s tempting to draw conclusions like “the Sigma 50 has higher center resolution than the Nikon”, and “both the Sigma and Nikon at f/1.4 have higher resolution than the Canon at f/1.2”. Reality is not that simple. Below is a similar chart, but this time I’ll plot the actual data points for each copy tested. Reality is a lot messier than the single averaged number show above.

The takeaway message is when you are presented one data point, either in this article or in a lens review, it’s just that: one data point. In the top graph, the Sigma MTF50 is 620/510 for peak/average while the Nikon is 580/530 (rounded to the nearest 10 because that’s about as accurate as Imatest numbers are). But claiming the Sigma is really sharper in the center, or the Nikon has a better average sharpness is a stretch. There’s a bit of a tendency that way, but if I hand you one copy of one and one copy of the other you wouldn’t necessarily get that result. On the other hand, I’d be pretty comfortable that if I hand you either a Nikon or the Sigma to compare to the Zeiss 50mm f/1.4 you would notice a difference.

We’ve found that a difference of 75 LP/IH in peak resolution generally means there’s a real difference between the lenses. A difference of 150 certainly means there is difference and it’s usually quite obvious. For average (rather than peak) MTF 50 the range is smaller and we consider differences of 50 and 100 LP/IH signifiant. But this is a generalized rule-of-thumb we use in-house. It’s not a law of physics or anything.

How I graphed the results

I tried to graph the results to reflect this. The lens with highest MTF50 at each aperture has a dark box around it’s results and is colored red. Any lens within 75 LP/IH (for center MTF) and 50 LP/IH (for weighted average) of this number is also colored red, indicating there probably is no real-world difference between them.

Lenses between 75 and 150 LP/IH lower than the best result for peak MTF50 (between 50 and 100 LP/IH for average MTF50) are colored yellow. There is probably a real difference in sharpness between those lenses and the best lens, but not a large difference. Lenses that are more than 150 LP/IH for peak and 100 LP/IH for average MTF lower than the best result are colored blue. The difference between those lenses and the best lens should be readily apparent.

I should make it very clear that the colors are only pertinent within each table. That is why, for example, the Sigma 50mm f/1.4 rates a red box for weighted average at f/1.4 in Nikon mount, but an orange box in Canon mount: it’s basically as sharp as the best Nikon mount lens at those settings, but not quite as sharp as the Canon 50mm lenses.

The Results

The Canon Mount Lenses

There was not much difference in center sharpness between the two Canon lenses, the Sigma, and the Zeiss Makro Planar. As expected, the Canon 50mm f/1.0 wasn’t sharp (everyone knows it’s soft). I knew the Zeiss 50mm f/1.4 wasn’t very sharp wide open, but I was pleasantly surprised at how well it did by f/2.8.

I was rather surprised at the weighted average results, though. I had expected the Zeiss 50mm f/2.0 to be the best in the corners and edges. I shoot Canon with 50mm primes a lot and am always complaining about soft corners with the Canon f/1.2 and bragging on the corners of the Zeiss Makro-Planar. This test made me realize a couple of things. First, I usually shoot the Canon at f/1.2, so I’m mentally comparing f/1.2 images to f/2.0 images. Not a fair comparison. Second, when I shoot the Zeiss lens I use LiveView focusing which is a lot more accurate than autofocus. I tend to shoot the Canon on autofocus because I can. Just one more case of “what I know isn’t so”.

From a pure resolution standpoint, the two Canon lenses are the best corner to corner (on a test chart at 15 feet – obviously things can be different at other distances and with real world objects). As far as center resolution the Canon and Sigma are even at wide apertures, with the Zeiss their equal at f/2.0 (for the f/2.0) and f/2.8 for the f/1.4. The Schneider and Canon tilt-shift lenses didn’t compare as well: they just start coming into their own at f/4 and if we had extended the test range up to f/5.6 or f/8 they probably would have caught the others.

The Nikon Mount Lenses

The Nikon tests turned out about as I expected. (Unfortunately we no longer had any 50mm f/1.4 D lenses, which we thought were sharper in the center than the G). The Sigma and Zeiss f/2.0 were a bit better in the center, the Nikon G a bit better in average sharpness. The G has the reputation as one of the weaker Nikon prime lenses because it’s center sharpness isn’t great, but perhaps its designers gave up a bit of center sharpness to keep the corners sharp. The Nikon 45 PC-E, just like the Canon, really comes into its own about f/5.6 and doesn’t fare well at these apertures. (BTW – we only took the apertures to f/4 because we rarely see a wide aperture prime get significantly shaper past f/4.) To be honest, I had expected more from the f/1.2 AIS and had thought it might be the best lens in this group, but for this copy, at least, it wasn’t so.

 

The Cine Lenses

Just because they were possible mounts to the Red Epic, and because we had no idea as to how sharp they might be, we tested a couple of $10,000 Cine lenses. (Before you ask, we didn’t test Zeiss CP.2 lenses because they are optically identical to the ones already tested above.) Since these are PL mount lenses we had to test them on a Hot Rod modified Canon 7D, the highest resolution camera we have in PL mount. Obviously that should give these lenses an advantage: they’re only covering a crop frame rather than a full frame.

Even with that advantage, they didn’t really resolve as well as the photo lenses. Cinema lenses have never been designed for absolute resolution, they have other priorities. So for all of you thinking about buying a set of Cooke Panchros for still photography, I’ve just saved you $50,000. Makes it worth reading my blog, doesn’t it?

The M-mount lenses

This was the part of the testing about which I was totally clueless., but also the part about which I was most excited. M-mount lenses are about to explode since they’re going to be shot on Sony NEX-7s and RED cameras a lot in the near future. And while we think they’re excellent lenses it’s difficult to find actual test results with them.

I will mention (for those of you thinking of doing this at home) that performing Imatest on a Rangefinder camera is, at the very least, time consuming and frustrating. Lots of patience  and focus bracketing are necessary. It took longer to do each of the M mount lenses than it took to do all of the Canon and Nikon mounts. An Eye-Fi card would have been a good idea since we had to unmount the camera from the tripod to remove the card for every image run.  Of course, we thought of that right after we finished testing.

In one way the results weren’t shocking. I expected the Leica f/1.4 ASPH would be the sharpest, but I was shocked at how amazingly good it was. I was certain the $10,000 Leica 0.95 would not be  as sharp — you can’t get that wide aperture without some other compromises — but was impressed with how much better it performs than the Canon 50mm f/1.0. The Voigtlander was a pleasant surprise (but then I had low expectations) and while the Zeiss lenses seem a bit disappointing it’s largely because of the fast company they are keeping here. Their results, at f2.8 and f/4 at least, compare well with the SLR lenses.

Overall Comparison

There are some limitations to my attempt to answer the original question of course. Until we can test on a common body, we don’t know exactly how much difference being mounted to an M9 makes compared to being mounted to a 5DII or D3x (we do know the latter two are equivalent). The results would be at least slightly different at different focusing distances than the 15 feet we used in our setup. And of course, the limitation of only testing one copy of each lens means the results for any one of the lenses might be slightly better or worse with a different copy.

But if we keep those limitations in mind the results are interesting. I’ve regrouped the best lenses from each category above and put them in one table for a simple comparison. I’ve also recolored the results of this table according to the rules set out originally: the highest resolution at each aperture being marked, etc. It’s obviously a bit of a stretch, but this was the original assignment, finding the 50mm lens with highest resolution.

If you are shooting at f/1.4, as most people usually are with a 50mm lens, there are a lot of good choices. The two Canons, Leica Summilux and the Sigma are about the same in the center. The Sigma may be a bit softer in the corners, but if you’re worried about corner sharpness why are you shooting a 50mm at f/1.4? At smaller apertures the Leica lenses get better than the rest, but for practical purposes, even the blue lenses in the table above are very sharp. There’s not a bad lens in the bunch.

In Conclusion

I will note again that this is strictly a resolution test, not a best overall lens test. Every test like this will ignite the fanboys who feel their favorite lens was slighted. Before all the accusations begin let me note that my favorite 50mm lens (and this test certainly won’t change my mind) is the Zeiss 50mm f/2.0 Makro, which didn’t fare nearly as well as I expected.

I was very curious about how well the M9 would resolve. I would not have been shocked if it couldn’t resolve quite as well as the Canon and Nikon since it has fewer pixels. In that case we might have seen all of the lenses reach a similar peak MTF after which the M9 wasn’t capable of further resolution. That’s obviously not the case and I’m left assuming the lack of AA filter more than makes up for the fewer pixels. But that’s just an assumption. It may be that the M mount lenses are just that much better than the SLR lenses.

I should mention that both the Canon and Nikon cameras are able to resolve MTF50 numbers in the 1100 – 1200 range (we’ve gotten those numbers with several lenses, but obviously not wide aperture 50mm lenses) so this test is not limited by sensor resolution.

It will be interesting to see how all of these lenses compare when we get Sony NEX-7s in stock. Then we’ll be able to shoot EOS, Nikon, and M mount lenses on the same camera. That should let us make a better comparison, at least in the center, but we won’t be able to compare extreme corners since it is a crop sensor.

Or at least it will make an interesting comparison if I ever get time to do something like this again.

 

Roger Cicala

Lensrentals.com

January, 2012

 

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Resolution Tests
  • I agree with Alexander Nelson and would want to see the Sony/Zeiss 55mm f/1.8 FE lens thrown in the mix.

  • Alexander Nelson

    Nice test and really interesting results. But it’s time to do an update and include the newest 50 mm normals on the market, like the Zeiss Otus and Sigma 50/1,4 Art.
    The swedish photomagazine FOTO has recently published a list of 50 mm lenses they call “the sharpest normal lenses” that includes the Otus and Sigma. And they also measure MTF but on the lenses themselves, no resolution limiting cameras involved. I think these MTF-diagrams are much easier to read and understand than a bunch of numbers in a table.

    http://tidningenfoto.se/de-skarpaste-normalobjektiven-fotos-topplista/

  • Bill Geissler

    This was a good article and revealed an objective way to compare 50mm lenses. There were earlier generation 50 and 58mm lenses which will compete well with 2013 prime lenses — esp. the Topcon 58mm F 1.4, the Konica 50mm f 1.4, the Super Takumar f 1.4, and the German made Zeiss 50 mm F 1.4 for the Rollei 35s. These would have to use adapters of course to attach to a Canon digital camera.

  • Andrew R

    Excellent informative post, as always. Thanks!

    I see you have some Leica APO-Summicron ASPH 50mm lenses on their way… any chance you feel like testing one of these in the same way and publishing the results? There are very few independent measured results available for the Summicron AA yet.

  • bossa

    I would have liked to see some Pentax lenses in that comparison; Any 50mm and the DA*55.

    I have owned a Nikon 50/1.4G and it was just terrible on my D800E’s so I swapped it out for a Sigma 50/1.4, which does the job in style, although it’s pretty soft in the corners. But no ’50’mm lens I’ve owned has been better than my DA*55mm Pentax on my K-5 which thus far leaves them all for dead.

  • i need to know if the lens of 50mm/40mm are good for landscape pictures if not what lenses are ? and i have a very tight budget since im a student. but do tell me tho please thanks

  • Bob

    I guess you couldn’t find a Pentax SMC A 50mm 1:1.2?

  • I’m now not certain where you are getting your info, however good topic. I must spend some time learning more or figuring out more. Thanks for wonderful info I used to be on the lookout for this info for my mission.

  • Adam

    I can speak from experience on the HD depth of field issue, as I have extensive television studio experience.
    Roger has it on the money that it is purely a sensor size issue, tilt shift lenses are not in use and I have never even heard of one for HD, though that does not mean that someone, somewhere hasn’t got one mounted to their HD rig right now.
    To give you an idea of how small we’re talking, a “wide” lens in HD is generally in the 6-8mm range.
    Studios, especially news studios or anywhere things are recorded live or “live to tape” simply move too fast for the use of HD prime lenses (which definitely exist) much lens tilt/shift lenses.
    As an example, a lens I see in common use is a 9-540. You’ve read it correctly, a 60x zoom. (Oh and it has a built in 2x extender too!) Obviously, most studios are willing to sacrifice sharpness for flexibility. (Keep in mind though that an HD image is roughly equivalent to two megapixels, so critical sharpness is less apparent, and also that this lens weighs close to fifty pounds and has a front element that’s 8”-10” in diameter, so they’re not sacrificing as much as a still photographer would be with a 60x SLR lens)
    Now the lens goes to 540 and very occasionally there will be a need to grab a very tight shot of something at 540 (or even 1080mm with the extender) and there needs to be enough light to have decent depth of field at that extreme. Even if it is decided that being as open as f4 or 5.6 will do the trick, that means the wide shot will ALSO have to be at f4 or 5.6 because the talent will be under the exact same lights, and you can’t change shutter speeds on the fly in live TV they way you can in still photography. 5.6 at 8mm buys a LOT of depth of field.
    Reality TV is still pretty fast paced, though they may occasionally use primes, if for no other reason than to have more options for low light because they don’t have time or resources to light every space they’ll be shooting.
    Shows like Planet Earth almost certainly use primes when possible, and it’s why they look GORGEOUS!

  • ben

    really cool article Roger…thanks for the hard work.

  • Heru Anggono

    I am hoping Roger would bring the “sharpness” thing a bit further. How about describing (in a measurable standard) elusive stuffs such as “looks” from certain a lens, contrast quality (micro contrast, edge sharpness, etc), and colour rendition.

  • Carl

    Roger,

    Thanks very much for the info. I had thought of the smaller sensors too, but I just don’t know. Some of it is like news achors on a set where sometimes the background is in razor sharp focus, and yet you can tell the camera is relatively close to them, the background 6 feet or more behind. Of course other times on a news set the background is intentionally diffuse.

    Also, a lot of the “reality” shows do seem to have a lot of deep focus…and even Discovery channel material at times. “Planet Earth” and similar of course are spectacular, and certainly the telephoto work in that is shallow depth of field, so at least that looks more familiar to me.

    Well, I was only voicing my wishlist. No time constraints, and I know how you feel! I dread doing photo editing because it means sitting at the computer for many hours, and even if I take breaks and switch pillows, it still hurts! I need to get faster at it! It’s much more fun to just shoot the pics, as you know all too well!

    I can’t wait to see your take on the D4 vs the 1Dx. Some of the sample shots online from the D4, actually look like they might be MORE noisy than the D3s, especially at ISO 25.6k. Wow wouldn’t that be a disappointment! I’m sure it’s not actually true, though. Either would be a dream to use, but I think I would still prefer the slightly lower price and increased pixel density of the 1D4. I guess those will be fewer and farther between, and the price might just go up for the few remaining new units, might it not? I wish I could afford to buy one now!

  • Roger Cicala

    Carl,

    I don’t know about the TV lenses. Some of it, I’m sure, has to do with smaller sensors in even HDTV cams. Otherwise I can’t say, but don’t have any tilt function that I’m aware of.

    As to further testing, I’d love to do it like that. Unfortunately, the primary use of the testing equipment (and me) is for checking all of the lenses that come back every day. December is slow and I was able to spend a full day doing these tests, but the chance of doing more than that is slim, at least in one setting.

    Roger

  • Carl

    A couple of other interesting things about this blog post:

    No Nikon fanbois seem to be in an uproar about Roger’s finding that the D3x didn’t out-resolve the Canon 5D2. I guess most of those are “noise fanbois”, and thus all they care about are the 12mp (and soon to be 16.2mp) bodies.

    The finding that the typical sharpest aperture for a wide aperture lens is at f/4, because few people shoot a wide aperture lens at smaller than f/4. That doesn’t seem right to me, I’m sure most photographers shoot at a wide range of aperture (at least the ones that don’t leave everything in auto mode). If this is referring to most cinematographers, that’s a different story, and I’d understand that more.

    Nobody ever seems to mention what lenses are used in much of the HD tv programming I see on most every channel, where there is an ever-present, incredibly deep focus, yet it doesn’t have the look of a closed down aperture to me. I have asked Roger several times, but he never replied…Are there “cinema” or other lenses in common use, which employ a design similar to tilt-shift types, where a deep focus can be achieved while leaving the aperture relatively open?

    Sorry if that is a dumb question or is widely known by everybody…I’m not a cinematographer but have always been curious.

    And not to beat a dead horse, but there is a 45mm lens included in the test, and that is well within the 8% of finder view difference, of a lens like my 58 (actually ~54mm) Nokton.

    And Roger, a couple more suggestions. If you do the test again on the Nex 7 or whatever, don’t limit it to 50mm lenses, include more lenses in the “normal” range, so that might include the Leica 75mm f/2 Summicron. And maybe try to get a bit more distance, 20 to 25 feet (I’m sure you have the building space now!). And go up to f/5.6 instead of f/4.

Follow on Feedly