Equipment

Sony Goes World Class: The 24-70mm f/2.8 GM MTF and Variance Tests

24-70gmaster

 

I was one of those who noted Sony had some troubles, as manufacturers often do, with some of their first generation lenses for full-frame FE mount cameras. When they announced the G Master lens series I was really excited to test them. I was expecting Sony to have made progress both with optical quality and with copy-to-copy variation. I wasn’t sure that they’d be as good as the best lenses coming from the long-term photo manufacturers, but I did expect they would be close. In case you don’t like to read or look at charts, you can stop here. The 24-70 f/2.8 GM lens is as good as any 24-70 f/2.8 zoom from any manufacturer, at least as far as bench-test results go.

A Note About Sony MTF Testing

As I discussed in our post about the 70-200mm f/4 lenses, our testing algorithms and presentations are changing as we improve things and try to make them more scientific. If you missed that there are two major points.

First, the variation algorithms are different, both to make the charts easier to see (we show a 1 Standard Deviation range, rather than the 1.5 S. D. we used to) and to eliminate the Consistency Number. We found that the Consistency Number was too blunt of a tool; it showed only one limited part of variation. There were cases when there were two lenses had the same consistency score, but one was quite a bit worse than the other in ways the number didn’t show.

Second, I want to continue to point out that the MTF bench is not designed to test lenses that require power to maintain focus position, which FE lenses do. We’ve worked around that by making an electrically live mount, but the electronics block some of the test points at 20mm from the center (the right side of the graph). For that reason, the measurements at the edge have fewer measured points than the other points tested. Take them with a mild grain of salt.

Finally, one note about this test. We found that the 24-70mm f/2.8 GM lens performed best with 2mm of optical glass placed in the pathway, simulating the cover glass of a camera sensor. This is of NO significance to persons shooting with the lens, but I mention it for completeness sake. We check every lens we test with and without glass in the optical pathway and present the best MTF results.

MTF Tests

We tested 10 copies of the lens at 24mm, 35mm and 70mm focal lengths. I’ll print the results full size first because sometimes it’s hard to see the smaller comparison charts.

OlafOpticalTesting, 2016

OlafOpticalTesting, 2016

OlafOpticalTesting, 2016

OlafOpticalTesting, 2016

70mm

OlafOpticalTesting, 2016

 

These are really impressive MTF curves with excellent resolution, but it’s always helpful to compare things a bit, so below are side-by-side comparisons with the Canon 24-70mm f/2.8 Mk II and Nikon 24-70 f/2.8 AF-S VR lenses.  First, we’ll compare the MTFs at 24mm.

OlafOpticalTesting, 2016

OlafOpticalTesting, 2016

 

I know the graphs are a bit small (click them for larger), but it should still be apparent that the Sony is at least as good as the other two lenses. It actually has the best center resolution, particularly at higher frequencies, which backs up Sony’s statement that this lens was designed with high-resolution sensors in mind. Off-axis, it maintains good sharpness to the edge of the field, although it does have a bit more astigmatism or lateral color (it’s not possible to differentiate the two on a single-aperture MTF test). All three of these are excellent lenses at 24mm. The Sony might be a hair better, but it’s a pretty minor hair.

The 70mm end of 24-70mm zooms tends to be the weaker end, so we’ll do the same comparison there.

70mmcompsml

OlafOpticalTesting, 2016

 

None of these lenses are quite as strong at 70mm as they were at the wider portion of the zoom range. Again the Sony is at least as good as the others. It has, by a tiny hair the best center resolution, and while it isn’t quite as flat across the image as the Nikon, it is better in that regard than the Canon. It does have just a bit more astigmatism / lateral color on the MTF bench, though.

Fanboys can split hairs about which lens is best if they need to, but really all three of these are excellent lenses with only minor differences between them.

Copy-to-Copy Variation

This was the part of testing that I was most interested in since we’ve seen a few Sony FE lenses that had pretty large copy-to-copy variation. I’ll remind you again that the variance algorithms are different now, so for comparison, I’ve redone the variance graphs using the raw data from the Canon and Nikon 24-70 lenses, too. Both of them we consider pretty good, with a reasonable amount of variation for zoom lenses.

OlafOpticalTesting, 2016

OlafOpticalTesting, 2016

 

To my surprise and pleasure, the variation of the Sony G Master lenses at 24mm is at least as low as, and perhaps a bit better than, the Canon and Nikon zooms, both of which we consider good for zooms.

We knew that both the Canon and Nikon lenses had more variation at 70mm than at 24mm, so we expected the Sony would, too.

OlafOpticalTesting.com

OlafOpticalTesting.com

 

Like the Canon and Nikon, the Sony did have more variation at 70mm. But the Sony doesn’t worsen more than the others, they all end up fairly similar. As an aside, we also checked variance in the middle of the zoom range and the Sony was really quite good there. In other words, if your copy has problems, it will almost certainly be at 70mm, but if you buy a Sony 24-70mm f/2.8 you are no more likely to have an issue than you would with a Canon or Nikon.

Summary

When the Canon 24-70 f/2.8 Mk II came out a couple of years ago, I hailed it as dramatically better than what had ever been available in this range. Sony FE shooters now have it’s equal in a native-mount lens. Sure the fanboys can split hairs about this or that, but the differences in the lenses themselves are tiny. Of course, the proof is in the image, but early reports from reviewers seem to agree the new Sony performs just as well in the field as it does on the MTF bench. Well done, Sony!

 

Roger Cicala, Aaron Closz, and Brandon Dube

Lensrentals.com

April, 2016

 

 

 

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Equipment
  • EcoR1

    It’s so funny. When some people are told that there are very good lenses out there, released by brand they don’t support, they start listing reasons why it would be bad to own those lenses. Price/customer support/ ergonomics of the bodies, you name it. I guess it feels pretty bad, when you hear that there might be some lenses that could be optically as good or better as the ones that you already own. Because you made the correct brand choose when you bought your first system camera. Right?

  • Master,
    Actually I spent a lot of time a couple of years ago doing all those things. Not nearly as easy as you think. Humans shake very differently (I’m a physician, I know) depending on position, age, gender, and a host of other factors. Even then, several prototypes were built. All failed pretty badly.

    Roger

  • Well, see you in 4 years 😉

  • “Question tarif, il a été annoncé autour des 2700 euros”
    http://www.focus-numerique.com/test-1481/objectif-canon-ef-24-70mm-f2_8-II-usm-presentation-caracteristiques-1.html

    “Prix public conseillé : 2699€ TTC”
    http://www.fnac.com/Canon-sort-3-nouveaux-objectifs-dont-le-EF-24-70mm-f-2-8L-II-USM/cp14989/w-4

    Yes it was (announced at $2300/£2200/€2700).

  • Master

    and to add this…. there a actually some sony acolytes (always the same) who say sonys support is even better than canons.
    i mean obviously everyone with a clue knows that´s not true.

  • Wesley

    I see.

    How about a 35mm comparison chart? Did you test 50mm?

  • Master

    no it was not that expensive.

    $2249 at launch… in europe ~2249 euro

  • Master

    when canon released the 24-70mm Mk2 it has cost 2249$.

    but canon had no competition back then.
    the canon 24-70 MK2 was a zoom lens so good it had no equal.

    today there is a lens (said canon 24-70 MK2) that is as good as this new sony.. and it cost 700 euro less.

  • Wesley

    One would put on a battery grip as well in this instance.

  • S.Yu

    Well following the same logic I could still say you’re wrong. The Canon needs an extra adaptor to work at all on a Sony, also for it to work on a Sony you first need a Sony, for it to work on a Canon , you need another Canon for that, so for you to make use of the fact that the Canon could be used in two systems, you need two bodies and an adaptor, which costs a lot, so if you need a lot of extra investments to achieve that extra function, the extra function would have significantly less potential value, or shall we say the cost greatly offsets the practical value.
    For the Sony to achieve those functions, you only need that one Sony body, without an adaptor, and it happens to come with other advanced functions like a full electronic shutter and advanced video capabilities.
    To further the argument, the Canon, even with the adaptor, is not 100% functional on the Sony, it just gets pretty close, so having the potential to be adapted does raise the value of a Canon lens a little, but far from 2x, also any adaptation basically means you’re willing to make your gear more bulky though the adaptor, so there’s that issue too.

  • Master

    2249$… yes.
    that´s what wikipedia and other websites say.

    but canon had no competition back then.
    the canon 24-70 MK2 was a lens so good it had no equal.

    today there is a lens as good as this new sony.. and it cost 700 euro less.

  • Master

    there are no optical achivements over canons lens.
    not even close to worth 700 euro more.

  • S.Yu

    You don’t have to care, but he’s putting it like Sony should make its new lens immediately as cheap as Canon’s years old product, or else its optical achievements should be ignored.

  • Master

    you test of course with a typical human shake.
    humans normaly won´t do a 20cm amplitude with 1000 hertz.. right?
    ask some people with expertiese in this field any you will get answers. it´s no magic.

  • Ilan

    That’s so clever.

  • The Canon was 2700 euros when released.

  • leh

    Your argument has no sense. Sigma or tamron should take 4 times more money because they produce lenses for some systems?

    But for example in-body stabilisation is real advantage on FE system. You don’t have it on ANY canon dslr body so GM lens has a solid advantage over canon set.

  • leh

    When you lack arguments or doesn’t understand you start with ‘fanboy’ stuff? Really? Is it canon-user level of discusion?:/

    How many FF canon cameras have dual pixel technology? In sony all can AF anywhere in frame. When you can’t use it you don’t do it and accept that, but when you can – you do and it’s a very nice feature which you lack when you don’t have;)

    Focus on 100% of the frame is only one of the advantages… You have
    also ‘eye AF’ wchich focuses on the eye, all the time you see real image and
    setups (like BW) with EVF, you can enlarge a part of the frame to
    check/correct focus, you can backup pictures on your phone through WIFI
    and using it as a remote LCD and pilot, you have a ton of aps, you have
    tiltable screen in each body… it all (and much more) incrises
    capabilities of a lens attached to the FE body.
    Lens is only a part of the set. The second one is a camera and they work as a team. So when A7vII series have stabilisation it works with GM lens so the GM have more possibilities and advantage over a system that doesn’t have that.
    You can say ‘oh well I don’t need this, this, that and that’ but they are still advantages.

  • Brandon Dube

    And how do you test fairly? Does one focus on low frequency high amplitude, high frequency high amplitude, high frequency low amplitude? What frequency distribution? Even in a controlled test there are perhaps a half dozen to a dozen variables to test.

  • Master

    can´t you build a rig that produces a repeatable shake pattern.
    should not be to difficult for a mechanical engineer.
    the important part here is repeatable.
    so the amplitude and frequenz must be the same.
    take the weight of the body lens combo into account.
    you than mount the cameras into the rig and test the image stabilization.

  • Master

    System.. Lens.. now what exactly are you talking about Fanboy?

  • Master

    Yeah.. i wait even longer.
    And who cares?
    It´s what it will cost you now.

  • Master

    NO. 1 reason i don´t buy Sony camera gear.
    I don´t spend a few thousand Euro on stuff i get not repaired properly.
    Canon does a fantastic job and has so for 15+ years.

  • Master

    how does the sony bEnd mount deal with heavy lenses?

  • S.Yu

    And since Sony gave a lot of PR on their XA elements, we should have a comparison of onion rings between those two and the Sony.
    Huge irony should the Sony lose though.

  • Shark, it’s no secret 🙂
    Sony doesn’t do any in-house repair here, they subcontract it out and the support isn’t OK at all. I’m sure they’ll fix it someday, but that’s going to require building an infrastructure they just don’t have yet.

  • S.Yu

    “what does that have to do with the lens?”?
    This has everything to do about the system, many Canikon fanboys tout their system when their body and/or lenses can’t beat the Sony equivalent.

  • S.Yu

    “When the Canon 24-70 f/2.8 Mk II came out a couple of years ago…”
    This explains a lot. Wait at least as many years then compare the price of the Sony then to the Canon now, and correct for inflation.

  • Shark

    roger can you secretly send some lenses to sony for repair and write about your experience?
    the sony supports is terrible here in europe.
    i don´t pay 2400 euro for a lens and have support that needs forever.
    i have some sony HIFI stuff i did not get repaired after 4 years.
    im a burned customer when it comes to sony.

  • Shark

    handling is a nightmare with sony anyway.
    i have a a6000 and i only use it with the kit pancake.
    everything else makes no sense.

Follow on Feedly