LensRentals.com

Wide-Angle Micro 4/3 Imatest Results

Posted by

Note: The results for standard range m4/3 lenses and zooms at 25mm are in the next article.

A lot of Sony users liked seeing some Imatest results for NEX lenses, and some micro 4/3 users asked us to do the same thing. That's a more complicated undertaking for several reasons. First, there is a much wider selection of native lenses for micro 4/3. Second because there are a lot of different micro 4/3 cameras. But we wanted to tackle the project because, quite honestly, we'd never done any lab testing on m4/3 lenses. We wanted to know, too.

Making the project something we could do in reasonable time means cutting some corners. First we can't test every camera. We chose the Olympus OM-D EM-5 for several reasons. We wanted a camera with 16-megapixel resolution, which eliminated most m4/3 cameras. Also, the physical layout of the Olympus lets us test more quickly in our lab than we can with any of the 16-mPix Panasonic cameras. There just wasn't enough time in one day (all we could set aside for this right now) to do two different cameras.

Second, we split the test into two sets of lenses because there were so many to test. In this part we did lenses that were 25mm or wider in focal length. We'll do the longer lenses later. Zooms with a long range were tested near their wide end in this test, and will be tested again at longer focal lengths when we do the longer lenses.

Disclaimer

The usual “don’t read this stuff and go insane” cautions apply: this is a test of lens and sensor resolution for MTF 50 (detail resolution) done at a distance of 10 to 30 feet, depending on focal length. The results might be different at 4 feet or 400 feet. The results might be difference if we tested at MTF 10 for best acutance. Focus is done manually and bracketed so it’s not a test of a camera’s autofocus ability. The tests use a controlled-lighting test-target with a hardware mounted camera at ISO 400, so photos taken in the dark of night or hand-held for 12 second exposures will give slightly different results. It doesn't measure autofocus speed, lens size, manual focus feel, bokeh, color rendition or anything like that. It's not a lens review; it's one simple test of resolution.

More importantly, remember that while we are testing RAW files, both Olympus and Panasonic seem to do some in-camera corrections on their own lenses, but not the other brands. We don't know what effects, if any, that might have on this test. Possibly Panasonic lenses might do a little better on Panasonic cameras and Olympus lenses a little worse. We'll figure that out later, but I doubt any such corrections will affect the MTF 50 at all. If it did the effects should be minor and limited to the edges and corners. But we don't know for sure. Yet.

Addendum 4/13/12: I had hoped that the m4/3 community might not be quite as Fanboy riddled as Canon and Nikon. Obviously I was wrong about that. Because there was such an uproar about the results of the Panasonic / Leica 25mm I've repeated the tests on 6 copies (all that we have) and also on Panasonic cameras, because Fanboys have been on suicide watch and full-attack-mode since they've taken them so out-of-context. The average of 6 lenses on both Olympus and Panasonic 16 Mpix cameras are now reported. The results are basically unchanged.

The Lenses

We had a nice selection of wide-angle m4/3 lenses in stock on the day we conducted these tests:

Imatest Results

The values are for MTF-50 (which correlates with fine detail resolution) measured in line pairs / image height. The first (higher) number is the MTF 50 measured at the center of the lens only. The second (lower) number is the average of the MTF-50 measured at 13 points including the center, 4 corners, 4 sides, and 4 mid points. I keep a point total for every time someone posts the question "what do the two numbers mean" as an indicator of the reading comprehension for each group of camera users. I have high expectations for m4/3 users, so please don't let me down.

I'll also mention that numbers sometimes make minor differences seem really large. A difference less than 50 lp/ih is probably not noticeable in real-world photography. So if you print a reasonably sized image from a lens resolving 860/750 lp/ih and compared it to a shot with another resolving 810/700 lp/ih the difference would probably not be noticeable. You would almost certainly notice a difference of 100 lp/ih.

At Widest Aperture

The first graph shows the numbers for each of our lenses (zooms measured at 30mm) with the lens aperture wide open. This test isn't a level playing field, since some lenses are f/4 wide open while the primes are at anything from f/1.4 to f/3.5. But since some people tend to shoot every lens wide open, it may be useful to know how well (or not) the lenses do at their widest aperture. I've listed them from widest to longest focal length. The zooms were all shot at 14mm so they might be a bit better or worse at other focal lengths.

MTF 50 at Widest Aperture

Lens Max Avg
Olympus 12mm f/2.0 860 730
Panasonic 14mm f/2.5 950 785
Panasonic 7-14 f/4 965 765
Olympus 9-18 f/5 805 680
Olympus 12-50 f/3.5 825 680
Olympus 14-42 II f/3.5 720 600
Olympus 17mm f/2.8 720 590
Voigtlander 17.5mm f/0.95 565 475
Panasonic 20mm f/1.7 870 735
Voigtlander 25mm mm f/0.95 530 435
Panasonic/Leica 25mm f/1.4* 690 590

*Panasonic-Leica results on this test are the average of 6 copies. The average numbers are slightly better at f/1.4 than  the first version of this article, which reported only one copy, but the difference is not really significant.

There's not a lot to see from this set of numbers. As usual, good primes are as sharp, or sharper, at wide apertures than consumer-grade zooms are at lower apertures. The Panasonic pancake lenses pleasantly surprised me. Pancakes are usually not the greatest lenses, but he Panasonic 14mm and 20mm are really quite good.

Stopped Down to f/2.8 or Smaller

We can level the playing field a little bit by shooting the wide primes at f/2.8, as shown in the graph below. It's somewhat more meaningful: the zooms are still at smaller apertures, but everything else is shot at f/2.8 here.

MTF 50 at f/2.8

Lens Max Avg
Olympus 12mm f/2.8 1000 845
Panasonic 14mm f/2.8 1010 850
Panasonic 7-14 f/4 965 765
Olympus 9-18 f/5 805 680
Olympus 12-50 f/3.5 825 680
Olympus 14-42 II  f/3.5 720 600
Olympus 17mm f/2.8 720 590
Voigtlander 17.5mm f/2.8 965 810
Panasonic 20mm f/2.8 1050 875
Voigtlander 25mm mm f/2.8 995 855
Panasonic/Leica 25mm f/2.8* 960 820

*Panasonic-Leica results on this test are the average of 6 copies. The average numbers at f/2.8 are slightly better than  the first version of this article, which reported only one copy, but the difference is not really significant.

By f/2.8 the superiority of most of the primes over most of the zooms becomes pretty apparent. The Olympus 17mm f/2.8 is easily the weakest of the prime lenses, which isn't really surprising. It's an older design and a consumer-grade pancake lens. The two Panasonic pancakes being so good surprised me, though. They're not much more expensive than the Olympus 17mm. I had expected a bit more out of the Panasonic Leica 25mm given it's cost and build quality. It has other good qualities, this is simply the results for MTF 50. The Voigtlander lenses, despite being built for ultra-wide aperture (which usually results in a rather soft lens) did quite well. But as easily the most expensive lenses in the group, that seems appropriate.

Personally my biggest disappointment was in the Panasonic 7-14 f/4 zoom, which is probably my favorite m4/3 lens. But we were testing at 14mm, and that lens is actually a bit better at the wide end. And really, it's probably a bit much to ask a lens to be this incredibly wide, and also incredibly sharp. The Nikon 14-24 pulls it off for about $2,000, but that's the only one I can think of.

MTF 50 at f/4

We can level things out even further by shooting all of the lenses at f/4. Most of these lenses would get a bit sharper still at f/5.6, especially in the average (as opposed to center) resolution.

MTF 50 at f/4

Lens Max Avg
Olympus 12mm f/4 1040 870
Panasonic 14mm f/4 1025 860
Panasonic 7-14 f/4 965 765
Olympus 9-18 f/5 805 680
Olympus 12-50 f/4 830 685
Olympus 14-42 II f/4 730 605
Olympus 17mm f/4 735 610
Voigtlander 17.5mm f/4 1070 835
Panasonic 20mm f/4 1075 880
Voigtlander 25mm mm f/4 1030 925
Panasonic/Leica 25mm f/4* 980 850

*Panasonic-Leica results on this test are the average of 6 copies. The average numbers at f/4 are slightly lower than  the first version of this article, which reported only one copy, but the difference is not really significant.

There's really not much new information here that we didn't already see at f/2.8. The Olympus 9-18 and 12-50 zooms are clearly better than the little 14-42 zoom. The Panasonic 7-14, despite working at such wide angles, is better than either of the others. Overall another "you get what you pay for" situation - slapping  II on the name doesn't make it a great lens. (BTW - although I expect m4/3 shooters to go a little less Fanboy than the Canon and Nikon guys, let's get this out of the way: some Fanboy always says "the rental guy wants you to rent the most expensive lens".  Actually the rental guy could care less: the profit margin is the same on a really expensive lens and a cheap one.)

Of the prime lenses, only the Oly 17 pancake isn't very good; the Panasonic pancakes were clearly better. The Panasonic/Leica 25mm disappointed a bit: it was good but I had expected it to be THE best of the bunch and it's not. The Olympus 12mm was just excellent, though, and the Voigtlanders provide an amazing wide aperture lens.

Part 1 Summary

Compared to the NEX system lenses I just tested, m4/3 users have a host of wide-angle, native-mount choices. There are good pancakes, standard primes, interesting (and expensive) ultra-wide aperture primes, all at reasonable prices. There's a good f/4 ultrawide zoom and a couple of reasonably sharp zoom lenses. The big hole, as I see it, is that there's no wide-aperture zoom, period. For my shooting style, the m4/3 begs for a wide aperture.

Remember, we're just talking about resolution here. When I'm choosing a lens, it's my starting point: I require good resolution. But that's all it is, a starting point. If you're seriously considering some of these lenses, go to a good review site where you can get more in-depth information about distortion, handling, focusing accuracy, etc.

In this part we've looked at most, but not all, of the m4/3 lens choices for shooting at 25mm or wider. We've only tested on the Olympus OM-D. We'll do some testing on at least one of the Panasonic 16mm cameras just to make certain there's no significant difference soon (I doubt there will be, but it's possible). We'll also do a second post on the longer lenses in the next couple of weeks.

Roger Cicala

Lensrentals.com

May 2012

64 Responses to “Wide-Angle Micro 4/3 Imatest Results”

Dante said:

Thanks so much, Roger.

This is the kind of review that I really appreciate, direct comparisons and telling it like it is. Great job!

Dante

Sam said:

Roger this is not a criticism. Your test results for Panasonic 14mm f2.5 are so different from others. This lens is generally regards as a mediocre lens. Only advantage is the size.

LensRentals Employee

Roger Cicala said:

Hi Sam,

I'm aware of it. It's one of the things that got me to write this: http://www.lensrentals.com/blog/2012/09/why-arent-the-damn-numbers-the-same

Could be any of a number of things: test chart used, testing cameras, testing distances, lighting. I'm not sure what. I've repeated our tests a couple of times (there were some other results I was surprised at) and gotten the same results.

So this ends up being one of those " the majority are correct " things in my mind, and a good example of why one person's results (including mine) have to be correlated by others before they're considered fact. When a number of reputable testers disagree with me, I conclude they are probably correct and my results are wrong. But in this case, I haven't been able to figure out why mine are different.

Roger

Sam said:

Thanks Roger for the reply and the link. According to tests you posted, Panasonic 14mm gives 90 to 160 lp/ih more than Sigma 19mm at f4 and f2.8. Does it mean Panasonic 14mm is sharper than Sigma 19mm?

I am asking this question because I don't know whether it is meaningful to compare MTF of two different focal lengths.

Ethan said:

Haha, there must be something off if the 14mm is at the top and the 25mm at the bottom. I'm sure the copies of the 25mm is fine but maybe the testing method should be in question? I've own most of these lenses except for the Voigtlanders and I think this chart is all sorts of wrong compared to real world use IMHO.

Lt. Gruber said:

Excellent test, thank you! The Panasonic Leica 25mm fanboys probably lie crying under the bed by now. Six copies, six, and I have no reason to believe you are a sloppy tester. I've used both the 25mm and the 20mm and in the end I held on to the 20mm. Not only do its photos look sharper en more contrasty, it's also the most portable lens of the two. It seems that my eyes did not deceive me.

Geoff said:

Re: "Actually the rental guy could care less."
Actually you actually could NOT care less. If you cared less, then at the moment you care a bit more.

jseliger said:

Any chance we'll see the Olympus 17mm f / 1.8 added to the table, or has this ship sailed?

tex said:

Thanks for the tests, Roger. I'd love to see the SLR Magic 12mm t/1.6 added to the mix - 12mm native mount m43 lens, and you offer them for rental, right? I'm sort of intrigued by this lens, and would like to see more data on it than I've been able to find online, so far.

jseliger said:

Ethan:

Haha, there must be something off if the 14mm is at the top and the 25mm at the bottom. I’m sure the copies of the 25mm is fine but maybe the testing method should be in question?

Keep in mind this: "The first graph shows the numbers for each of our lenses (zooms measured at 30mm) with the lens aperture wide open." If you compare the 25mm to other lenses at 1.8 or 2, the results might be very different, as the 2.8 test implies.

Juw said:

Hi Roger, I am a little confused here. Does it mean that these scores show that a MFT system resolves more detail than a full frame system?

This is strange especially since we are looking at wide angle where the larger sensor has the greatest advantage. Am I reading this wrong?

LensRentals Employee

Roger Cicala said:

Juw - Imatest numbers aren't really comparable across systems for a lot of reasons. You aren't going to see more pixels from a 16 Mpix system than from a 24 Mpix system.

carlos said:

I have a question.
You said that Panasonic 25mm disappointed but, if you compare with the results of the great 50mm shootout the pana is better than all the others at the widest aperture and at f/2.8, only loosing to the leicas at f/4. How can it be?

LensRentals Employee

Roger Cicala said:

Carlos,

Different cameras! Different focal lengths!Different systems!f/2.8!
Just like every site that uses Imatest data says: comparing between different systems is something one does with a large, large, large grain of salt because the numbers will always be different. Comparing lenses meant for different systems is best done with an optical bench, not with Imatest.

Plus I said I was a bit surprised that it wasn't world beating at f/1.4. At f/2.8 it's marvelous. Have you read my articles, or just what someone said they said?

Leave a Reply