Equipment

Wide-Angle Micro 4/3 Imatest Results

Published May 9, 2012

Note: The results for standard range m4/3 lenses and zooms at 25mm are in the next article.

A lot of Sony users liked seeing some Imatest results for NEX lenses, and some micro 4/3 users asked us to do the same thing. That’s a more complicated undertaking for several reasons. First, there is a much wider selection of native lenses for micro 4/3. Second because there are a lot of different micro 4/3 cameras. But we wanted to tackle the project because, quite honestly, we’d never done any lab testing on m4/3 lenses. We wanted to know, too.

Making the project something we could do in reasonable time means cutting some corners. First we can’t test every camera. We chose the Olympus OM-D EM-5 for several reasons. We wanted a camera with 16-megapixel resolution, which eliminated most m4/3 cameras. Also, the physical layout of the Olympus lets us test more quickly in our lab than we can with any of the 16-mPix Panasonic cameras. There just wasn’t enough time in one day (all we could set aside for this right now) to do two different cameras.

Second, we split the test into two sets of lenses because there were so many to test. In this part we did lenses that were 25mm or wider in focal length. We’ll do the longer lenses later. Zooms with a long range were tested near their wide end in this test, and will be tested again at longer focal lengths when we do the longer lenses.

Disclaimer

The usual “don’t read this stuff and go insane” cautions apply: this is a test of lens and sensor resolution for MTF 50 (detail resolution) done at a distance of 10 to 30 feet, depending on focal length. The results might be different at 4 feet or 400 feet. The results might be difference if we tested at MTF 10 for best acutance. Focus is done manually and bracketed so it’s not a test of a camera’s autofocus ability. The tests use a controlled-lighting test-target with a hardware mounted camera at ISO 400, so photos taken in the dark of night or hand-held for 12 second exposures will give slightly different results. It doesn’t measure autofocus speed, lens size, manual focus feel, bokeh, color rendition or anything like that. It’s not a lens review; it’s one simple test of resolution.

More importantly, remember that while we are testing RAW files, both Olympus and Panasonic seem to do some in-camera corrections on their own lenses, but not the other brands. We don’t know what effects, if any, that might have on this test. Possibly Panasonic lenses might do a little better on Panasonic cameras and Olympus lenses a little worse. We’ll figure that out later, but I doubt any such corrections will affect the MTF 50 at all. If it did the effects should be minor and limited to the edges and corners. But we don’t know for sure. Yet.

Addendum 4/13/12: I had hoped that the m4/3 community might not be quite as Fanboy riddled as Canon and Nikon. Obviously I was wrong about that. Because there was such an uproar about the results of the Panasonic / Leica 25mm I’ve repeated the tests on 6 copies (all that we have) and also on Panasonic cameras, because Fanboys have been on suicide watch and full-attack-mode since they’ve taken them so out-of-context. The average of 6 lenses on both Olympus and Panasonic 16 Mpix cameras are now reported. The results are basically unchanged.

The Lenses

We had a nice selection of wide-angle m4/3 lenses in stock on the day we conducted these tests:

Imatest Results

The values are for MTF-50 (which correlates with fine detail resolution) measured in line pairs / image height. The first (higher) number is the MTF 50 measured at the center of the lens only. The second (lower) number is the average of the MTF-50 measured at 13 points including the center, 4 corners, 4 sides, and 4 mid points. I keep a point total for every time someone posts the question “what do the two numbers mean” as an indicator of the reading comprehension for each group of camera users. I have high expectations for m4/3 users, so please don’t let me down.

I’ll also mention that numbers sometimes make minor differences seem really large. A difference less than 50 lp/ih is probably not noticeable in real-world photography. So if you print a reasonably sized image from a lens resolving 860/750 lp/ih and compared it to a shot with another resolving 810/700 lp/ih the difference would probably not be noticeable. You would almost certainly notice a difference of 100 lp/ih.

At Widest Aperture

The first graph shows the numbers for each of our lenses (zooms measured at 30mm) with the lens aperture wide open. This test isn’t a level playing field, since some lenses are f/4 wide open while the primes are at anything from f/1.4 to f/3.5. But since some people tend to shoot every lens wide open, it may be useful to know how well (or not) the lenses do at their widest aperture. I’ve listed them from widest to longest focal length. The zooms were all shot at 14mm so they might be a bit better or worse at other focal lengths.

MTF 50 at Widest Aperture

Lens Max Avg
Olympus 12mm f/2.0 860 730
Panasonic 14mm f/2.5 950 785
Panasonic 7-14 f/4 965 765
Olympus 9-18 f/5 805 680
Olympus 12-50 f/3.5 825 680
Olympus 14-42 II f/3.5 720 600
Olympus 17mm f/2.8 720 590
Voigtlander 17.5mm f/0.95 565 475
Panasonic 20mm f/1.7 870 735
Voigtlander 25mm mm f/0.95 530 435
Panasonic/Leica 25mm f/1.4* 690 590

*Panasonic-Leica results on this test are the average of 6 copies. The average numbers are slightly better at f/1.4 than  the first version of this article, which reported only one copy, but the difference is not really significant.

There’s not a lot to see from this set of numbers. As usual, good primes are as sharp, or sharper, at wide apertures than consumer-grade zooms are at lower apertures. The Panasonic pancake lenses pleasantly surprised me. Pancakes are usually not the greatest lenses, but he Panasonic 14mm and 20mm are really quite good.

Stopped Down to f/2.8 or Smaller

We can level the playing field a little bit by shooting the wide primes at f/2.8, as shown in the graph below. It’s somewhat more meaningful: the zooms are still at smaller apertures, but everything else is shot at f/2.8 here.

MTF 50 at f/2.8

Lens Max Avg
Olympus 12mm f/2.8 1000 845
Panasonic 14mm f/2.8 1010 850
Panasonic 7-14 f/4 965 765
Olympus 9-18 f/5 805 680
Olympus 12-50 f/3.5 825 680
Olympus 14-42 II  f/3.5 720 600
Olympus 17mm f/2.8 720 590
Voigtlander 17.5mm f/2.8 965 810
Panasonic 20mm f/2.8 1050 875
Voigtlander 25mm mm f/2.8 995 855
Panasonic/Leica 25mm f/2.8* 960 820

*Panasonic-Leica results on this test are the average of 6 copies. The average numbers at f/2.8 are slightly better than  the first version of this article, which reported only one copy, but the difference is not really significant.

By f/2.8 the superiority of most of the primes over most of the zooms becomes pretty apparent. The Olympus 17mm f/2.8 is easily the weakest of the prime lenses, which isn’t really surprising. It’s an older design and a consumer-grade pancake lens. The two Panasonic pancakes being so good surprised me, though. They’re not much more expensive than the Olympus 17mm. I had expected a bit more out of the Panasonic Leica 25mm given it’s cost and build quality. It has other good qualities, this is simply the results for MTF 50. The Voigtlander lenses, despite being built for ultra-wide aperture (which usually results in a rather soft lens) did quite well. But as easily the most expensive lenses in the group, that seems appropriate.

Personally my biggest disappointment was in the Panasonic 7-14 f/4 zoom, which is probably my favorite m4/3 lens. But we were testing at 14mm, and that lens is actually a bit better at the wide end. And really, it’s probably a bit much to ask a lens to be this incredibly wide, and also incredibly sharp. The Nikon 14-24 pulls it off for about $2,000, but that’s the only one I can think of.

MTF 50 at f/4

We can level things out even further by shooting all of the lenses at f/4. Most of these lenses would get a bit sharper still at f/5.6, especially in the average (as opposed to center) resolution.

MTF 50 at f/4

Lens Max Avg
Olympus 12mm f/4 1040 870
Panasonic 14mm f/4 1025 860
Panasonic 7-14 f/4 965 765
Olympus 9-18 f/5 805 680
Olympus 12-50 f/4 830 685
Olympus 14-42 II f/4 730 605
Olympus 17mm f/4 735 610
Voigtlander 17.5mm f/4 1070 835
Panasonic 20mm f/4 1075 880
Voigtlander 25mm mm f/4 1030 925
Panasonic/Leica 25mm f/4* 980 850

*Panasonic-Leica results on this test are the average of 6 copies. The average numbers at f/4 are slightly lower than  the first version of this article, which reported only one copy, but the difference is not really significant.

There’s really not much new information here that we didn’t already see at f/2.8. The Olympus 9-18 and 12-50 zooms are clearly better than the little 14-42 zoom. The Panasonic 7-14, despite working at such wide angles, is better than either of the others. Overall another “you get what you pay for” situation – slapping  II on the name doesn’t make it a great lens. (BTW – although I expect m4/3 shooters to go a little less Fanboy than the Canon and Nikon guys, let’s get this out of the way: some Fanboy always says “the rental guy wants you to rent the most expensive lens”.  Actually the rental guy could care less: the profit margin is the same on a really expensive lens and a cheap one.)

Of the prime lenses, only the Oly 17 pancake isn’t very good; the Panasonic pancakes were clearly better. The Panasonic/Leica 25mm disappointed a bit: it was good but I had expected it to be THE best of the bunch and it’s not. The Olympus 12mm was just excellent, though, and the Voigtlanders provide an amazing wide aperture lens.

Part 1 Summary

Compared to the NEX system lenses I just tested, m4/3 users have a host of wide-angle, native-mount choices. There are good pancakes, standard primes, interesting (and expensive) ultra-wide aperture primes, all at reasonable prices. There’s a good f/4 ultrawide zoom and a couple of reasonably sharp zoom lenses. The big hole, as I see it, is that there’s no wide-aperture zoom, period. For my shooting style, the m4/3 begs for a wide aperture.

Remember, we’re just talking about resolution here. When I’m choosing a lens, it’s my starting point: I require good resolution. But that’s all it is, a starting point. If you’re seriously considering some of these lenses, go to a good review site where you can get more in-depth information about distortion, handling, focusing accuracy, etc.

In this part we’ve looked at most, but not all, of the m4/3 lens choices for shooting at 25mm or wider. We’ve only tested on the Olympus OM-D. We’ll do some testing on at least one of the Panasonic 16mm cameras just to make certain there’s no significant difference soon (I doubt there will be, but it’s possible). We’ll also do a second post on the longer lenses in the next couple of weeks.

Roger Cicala

Lensrentals.com

May 2012

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Equipment
  • Thanks, this is a very useful piece. Any chance of adding any newer lenses to the list?

  • Roger Cicala

    Carlos,

    Different cameras! Different focal lengths!Different systems!f/2.8!
    Just like every site that uses Imatest data says: comparing between different systems is something one does with a large, large, large grain of salt because the numbers will always be different. Comparing lenses meant for different systems is best done with an optical bench, not with Imatest.

    Plus I said I was a bit surprised that it wasn’t world beating at f/1.4. At f/2.8 it’s marvelous. Have you read my articles, or just what someone said they said?

  • carlos

    I have a question.
    You said that Panasonic 25mm disappointed but, if you compare with the results of the great 50mm shootout the pana is better than all the others at the widest aperture and at f/2.8, only loosing to the leicas at f/4. How can it be?

  • Roger Cicala

    Juw – Imatest numbers aren’t really comparable across systems for a lot of reasons. You aren’t going to see more pixels from a 16 Mpix system than from a 24 Mpix system.

  • Juw

    Hi Roger, I am a little confused here. Does it mean that these scores show that a MFT system resolves more detail than a full frame system?

    This is strange especially since we are looking at wide angle where the larger sensor has the greatest advantage. Am I reading this wrong?

Follow on Feedly