An Update and Comparison of the Sony FE 70-200mm f/2.8 GM OSS
About a month ago, Roger posted his MTF bench results for the Nikon 70-200mm f/2.8E FL ED AF-S VR (TL;DR: it’s optically superb), and down in the comments there were some requests to compare it not only to the Canon 70-200mm f/2.8L IS II in the article, but also to the Sony FE 70-200mm f/2.8 GM OSS. There were MTF charts linked:
These are not very good looking results, especially for a $2600 lens, but Roger wasn’t sure if they were correct results. In his comments after these charts he said:
Let’s kind of keep this here for right now – I haven’t published it because I’m still a little uncertain about the results. Sony has suggested that a change in cover glass thickness might improve the results some. Not dramatically, but some, off-axis. This lens also has to focusing motors and we have to focus it electronically via a camera to test. I’m not absolutely certain that ‘setting it at infinity focus on the camera’ and ‘manually focusing on a an object at infinity’ are absolutely the same. So take these lab results with a grain of salt. On the other hand, they do seem to agree with what we see in real world results.
I try hard not to put out anything until I’m just absolutely certain our results are correct. We’re doing some stuff here that’s pretty cutting edge, honestly. No one does 4 rotation MTFs, for example. I’m pretty certain these are good results, but not absolutely certain. So I’ll post them in this discussion on my own site but I’d rather not see them reproduced all over the internet yet.
Someone suggested we try all three 70-200s on their respective camera bodies and shoot the same detailed scene with each, then share the files. But that would involve three different cameras and wouldn’t really be an apples to apples comparison, would it? Roger suggested he have one of our photo techs shoot all three lenses on a Sony camera with adapters, admitting that adapters add another variable, but there would be some good practical implications.
So I did just that. I took all three lenses, the Canon 70-200mm f/2.8L IS II, the Nikon 70-200mm f/2.8E FL ED AF-S VR, and the Sony FE 70-200mm f/2.8 GM OSS, and I shot our test chart with all of them on the Sony a7R II. For Canon, I used a Metabones T Smart Adapter IV, and for Nikon, I used a Novoflex Nikon to Sony E adapter. I manually focused all three lenses to get the most consistent results, and here’s what I got:
Full Resolution Examples are Available Here
If you view the charts at 100%, you’ll see that they’re consistent with the MTF charts Roger generated for all three lenses. The Sony just isn’t that great, and that’s really disappointing considering the price tag and how long customers have had to wait for that lens to be available. But if you need f/2.8 and working autofocus, it’s really the best option out there for Sony mirrorless cameras.
108 Comments
ZM ·
That hurts
Adam Sanford ·
[Full disclosure, I’m a Canon guy] Your 100% samples don’t present such a cut and dry indictment of the Sony vs. the Nikon to me. They don’t look nearly as far apart as your prior published MTFs do.
But: focus by wire and not clearly sharper for $500 *more* sure seems a big whiff to me.
picbod ·
I downloaded your files of the 70mm results for Canon and Sony and displayed them, untouched, at 100% in layers in Photoshop. Unfortunately for your conclusions, the comparison significantly favours the Sony lens. Huge difference actually, especially at the edges. So, I am perplexed how you could use your chart shots as evidence of poor Sony performance. Perhaps you mixed up the chart labels? And for the record, I’m a longtime professional Canon shooter who has used Canon 70-200 2.8 IS lenses on a daily basis since they were first introduced.
Guido ·
These lenses are usually more used at the long end. If you’re primary looking for something around the short end you will use something like one of the many excellent 85mm. Much faster, much cheaper, less weight. But there are not so many 200mm primes on the market. At 200mm the Sony is clearly much worse than the Nikon and worse than the Canon. It’s not a worse lens at all, but regarding the highprice tag it’s very disappointing.
Omesh Singh ·
I’m also a Canon shooter. In all honesty I chose the Tamron 70-200 VC + 135L for less money which gives me two options for events/portraiture to use on two different bodies (or having backup to cover the same medium-tele range) The 135L is magic SOOC.
Zinc ·
Huge difference at 200mm between Nikon and Sony. Sony should drop the price below 2000….
Zinc ·
https://uploads.disquscdn.com/images/f684ba197c2d95f2002a1b2a668f368ce0a44152b93c840e9e32d74e1290af82.png
Brian Smith ·
Yes, I pointed this out as well. The Sony at 200 is either decentered or titled or the test is not parallel.
Chris ·
Based on Sony’s track record, I would bet it’s decentered. This issue has become so common with Sony lenses it’s disturbing.
Terence Morrissey ·
Brian, Sony lenses are notorious for being decentered, see diglloyd.com
Sator Photo ·
The other oddity about the Sony 70-200mm f/2.8 for FE mount is the way it is formulated with two aspherical elements. Most designers avoid aspherical elements in 70-200mm lenses because of its adverse effect on bokeh. The reasons for using aspherical elements might include improving on acutance and for reduction of lens size. Due to the need to make mirrorless lens designs as telecentric as possible to reduce the light angle incidence in the corners, fast mirrorless lenses for ultra-short flange distance mounts like the FE mount tend to have blow-outs in size, which is why they are usually made with slower maximum apertures, making size reduction a likely principle motivator driving Sony to the use of aspherical elements. Yet despite resorting to the use of aspherical elements, Sony have been thoroughly beaten by Nikon who have no asphericals in their new FL 70-200. Canon will soon follow in Nikon’s footsteps with a refresh of their 70-200 and it will probably truly leave the Sony behind in the dust. It highlights the fact that one of the key weaknesses of mirrorless mounts with ultra-short flange distances is that telecentric lens design results in blow outs in lens size thus imposing engineering constraints when designing lenses with ultra fast apertures. Sony might do better if they stuck to slow lenses for the FE mount and focused on fast lenses for the A mount.
Chris ·
I’m no expert in lens design but if the flange distance is too short, the lens designers can move the rear elements out from the sensor plane to effectively duplicate a typical DSLR flange distance regardless of the actual mount. So I’m not convinced flange distance is a factor.
Larry Templeton ·
I think that was their plan, and then reality got in the way…
Athanasius Kirchner ·
@satorphoto:disqus
All telephoto zoom designs try to maximize telecentricity. On DSLRs, it’s almost a requirement because of the large flange distance, hence the retrofocal wide angle designs. This ridiculous legend about the short flange distance hurting E-mount needed to die yesterday.
Sony chose to use asphericals because they’ve invented a refined process for their molding which supposedly eliminates the dreaded “onion rings”. Panasonic pioneered this approach in some Leica-branded designs, and has reportedly worked very well.
I do agree that these results do not hold up to Sony’s boast of “future-proof” lenses, designed for 100 MP sensors. Either something’s wrong on their assembly line or their engineers are high on the good stuff.
Brandon Dube ·
A telephoto lens will not attempt to be telecentric; the rear member has net negative power, but not a large amount. The aperture stop would have to be very far in front of the front group to be telecentric. Unless there is a huge airspace in the lens, it is impossible to make a telecentric telephoto.
There is not much need to try to be telecentric in a telephoto lens, because the angle of view is small the chief ray angle is constrained to be small. Mapping something 10 degrees off axis onto the image certainly isn’t going to mean e.g. a 25 or 35 degree chief ray angle like you may find in a 90-degree FoV wide angle lens.
The refining process isn’t pioneered by Panasonic, though they may hold that honor in the consumer ring. A lot of pioneering work on super smooth aspheres was done by Itek corp, which is now dissolved with IP owned by a number of companies including Harris and Raytheon, among others. In the 80s Zygo’s precision polishing group did a lot of work there. Later in the 90s, QED technologies created the MRF process, which is among the smoother technologies available today at reasonable prices.
I disagree with the assertion that most aspheres today (or even 3 years ago when Panasonic announced their new process) are molded; the majority are still ground and polished using a ring tool geometry on any number of commercial CNC platforms whether it’s statis-loh, optipro, opto-tech, schneider, in house, or a different manufacture. There are only 8 or so optical glasses and plastics that can be molded, out of the 150 or so options available. None of them have a high refractive index, and none have very low dispersion. That is quite restrictive in the design process.
Molding is far cheaper; but it is the developing technology, not the established one.
Sator Photo ·
Somebody doesn’t know the difference between telephoto lens design and telecentric lens design!
Application of telecentric design principles to photography lens optics is highly unusual. It’s more the sort of thing you’d find in a confocal microscope. Optical engineer, Egami (in his now AWOL blog) explained that making a lens more telecentric does have the advantage of making the light rays more parallel when hitting the corners of the sensor. The disadvantage is that it causes a blow out in lens size. The larger the maximum aperture, the steeper the corner angle of light incidence, and the worse the compensatory size blow out necessarily becomes. Egami explained that this is the reason why designers fight shy of ultra fast lenses for the FE mount, as it tends to disproportionately impact on the negative size blow out of the lens.
Sony engineer Ken Nakajima, from the a7RII design team, confessed in an interview with Japanese magazine CAPA (July, 2016 issue page 171) that the reason the a7RII sensor had to have a BSI design (despite this resulting in significant loss of dynamic range) was to compensate for the corner shading that the steeper angle of light incidence from an ultra short flange distance causes. Nakajima explained that this is because, unlike film, digital sensors have a three dimensional structure, and this means they cannot register light hitting it at too steep an incident angle.
The use of BSI sensors and telecentric optical design principles are ways of compensating for the steep angle of corner light incidence on ultra short flange distance mounts such as the FE mount. But these compensatory measures have their own unintended consequences.
Although Sony’s advertising department made a big song and dance about how their new generation aspherical glass doesn’t degrade bokeh as much, the recent Photozone review of the Sony FE 24-70mm f/2.8 GM also cast graves doubts over the claims to superior bokeh from their new aspherical elements. It is far more likely that their prominent use of aspherical elements in FE lens design has other optical motivations. Namely, in the fact that aspherical elements can be used to make a lens smaller, which in this case may be necessary to partially mitigate the size blow outs inherent to fast FE mount lens designs (which Photozone amusingly called “the toad you have to swallow”).
Brandon Dube ·
I am quite well versed in most aspects of optical design.
telecentricity is not a design principal, it is a specification. A system is fully telecentric either in image or object space if the chief ray angle is zero One would specify the chief ray angle, for microscope objectives (of all kinds, there is nothing special about confocal microscopy in this regard) it is commonly 0.2 degrees. Telecentricity is desirable in microscopy because it means there is no change in magnification with defocus. Absolute scale is often important in microscopy, so it is important small defocuses do not alter the size of something. For a microscope, the object is usually the aperture stop as well.
To make a system telecentric, you place the aperture stop at the focus of the elements in front of it, or behind it. If it is at the focus of the elements in front of it you create an object-space telecentric system, the elements behind an image-space telecentric system.
In a telephoto lens, the rear member has a focal length longer than that of the front member. This can be found through the lagrange invariant; the change in the marginal ray angle from the front member is larger than that of the rear member, so the focal length of the rear member must be longer.
For e.g. a 100mm lens, the rear member may have a focal length of -75mm. This would require the aperture stop to be three inches in front of the rear member. A telephoto lens by definition is shorter than its focal length, Q.E.D. the aperture stop would have to be in front of the front member as well.
Reduced ray angles in the corners of the picture is not inherently advantageous. Film, for example, does not care what angle the rays are incident at. A symmetric wide angle has a large chief ray angle, but is easier to correct than an inverted telephoto which will have a lesser chief ray angle.
Telecentricity somewhat increases track length by restricting aperture stop location, but the real price is paid in the diameter of the elements. If the chief ray angle is zero and the rear element is the diameter of the image you have full vignetting. The rear element must be the size of the exit pupil in addition to the size of the image.
Generally, an increase in the aperture also increases the diameter. This compounds with the increase in diameter from telecentricity. The short flange distance of E/FE mount is irrelevant to such a discussion, as the rear element can be placed any distance from the sensor greater than the flange distance. These telecentric cinema lenses from Schneider are just as telecentric in front of an E mount as they are a PL mount.
https://www.schneideroptics.com/ecommerce/CatalogSubCategoryDisplay.aspx?CID=1842
Because light tunnels into silicon a significant distance before abortion and pixels have finite width, a sensor has what is known as an angle of acceptance, which is the highest angle of incidence which will be properly absorbed by a pixel. You can find some information here:
http://blog.teledynedalsa.com/2012/05/the-angle-on-optical-acceptance/
Because the capacitors and other complementary circuitry for each pixel are placed behind the photodiode in a BSI sensor, there is less ‘shading’ due to electronics, and the angle of acceptance is larger. This is quite good, as it alleviates constraint on wide-angle lens design. There is an increase in noise for a BSI sensor, as the electronics are more closely packed together, which results in a loss of dynamic range. The natural progression of things is towards all sensors being BSI; I would not say the A7rII “had” to be BSI. It is just Sony’s first camera with a large sensor to utilize the technology.
Aspherical lenses would never be applied to improve the out-of-focus rendering of a lens. They are used to compensate aberrations an raise the performance of a design, or reduce the number of surfaces needed to attain a given level of performance, reducing size. Sometimes both of these can be obtained simultaneously.
Aspheres are more costly to fabricate and test than spherical lenses. Often, they are not necessary. Generally, the older and more traditional a a company or designer, the more infrequently they will use aspherics. In optics Sony is young and they are a technology company at heart. It should not be surprising they use a comparatively large number of aspheres compared to e.g. Canon or Nikon.
The 85GM is quite massive, and is a very poor example of using aspheres to reduce size.
P.S., you are fond of photozone – you may find some articles on lens design I wrote for them some time ago.
Regards,
Brandon
Liberty_666 ·
Interesting post, Brandon. It was nice to learn why confocal microscopy uses a telecentric principle — you explain it very well. You had mentioned that in optics, Sony is a young company, which might explain why they use aspherics. But didn’t Sony acquire Minolta about 10 years ago? I would assume that many Minolta engineers are still designing lenses for Sony, where they can draw upon 90 years of expertise as a company. I don’t know Sony’s history well, so I am only speculating.
Brandon Dube ·
10 years is not a long time to be in a multi-faceted industry. If you’re around Canon and Nikon and Zeiss who have been doing this for 75+ years, you are a spring chicken.
No one from Minolta is still with Sony, to my knowledge.
Athanasius Kirchner ·
Well, that’s a lot of information, thanks for posting it.
Brian Smith ·
I have to agree with the previous posters. How did you conclude the results mirror the MTF results previously posted? At 70mm the Sony beats the Canon significantly and is very close to slightly better than the Nikon. At 200mm you can easily see the Sony is either titled/decentered or the test is not parallel. The right side of the Sony at 200mm is much sharper than the left. Based on the right side along it again matches the Nikon and beats the Canon.
Ideally all three lenses would be tested for decentering/tilt and then an outside shot at infinity (with plenty of top/bottom, left/right detail) would be taken. This setup would rule out decentering issues and/or alignment issues.
Perhaps you labeled the images wrong, but from what I am seeing the Sony beats the Canon and matches the Nikon. Other than the decentering at 200mm (which IS an issue) the Sony performed very well IMO which does NOT correlate with the MTF charts posted heretofore.
Zinc ·
Took an another look at those pictures, totally agree with you!
Chris ·
The center sharpness of the Sony at 200mm is noticeably soft, before you even compare to the other lenses. Unacceptable really. I suspect my 24-240 is sharper on center at 200mm than this lens. And if it’s slightly decentered, you wouldn’t expect center sharpness to improve much with a properly entered copy. So at least at 200mm, the test charts seem to support the MTF results.
Jekabs ·
Look at the corners on the right side of that image. There is something very wrong with test or the lens.
David Kilpatrick ·
OSS parking. Bet you they find this is the issue. Wonder how they test, for example, a Sigma OS lens where the parking is dramatically off-centre unless the lens is mounted and powered up?
Jekabs ·
Who knows, but the initial LR E-mount test result article says the lenses are electronically hooked up to the camera when mounted in OLAF, so that the focus can be adjusted ( https://www.lensrentals.com/blog/2015/10/sony-e-mount-lens-sharpness-bench-tests/ ). If so, this is unlikely an issue of OSS parking, as the lens is powered on and thinks it’s on the camera. Also, since the lens is powered on, the AF actuator voice coils are engaged and the focus group should not be able to slip out of position.
Geometry suggests that a similar misalignment of mount on lens or body, or sensor itself, would cause more tilt effect with a mount with shorter flange distance. Perhaps this is why Sony seems to be having more tilted lens issues than the DSLR competition, or even smaller sensor mirrorless competition?
David Kilpatrick ·
As a result of testing Milvus lenses on adaptors on A7RII – and a few earlier tests – I’ve found that Sony bodies can vary at least ±0.1mm in mount register, and most adaptors vary this much too. As the Sony mount has only four support points, it can tilt easily (Fuji’s six-point system, and the Alpha A-mount six-point, prevent this). What I found with the Milvus 15mm test was something I’ve suspected when using floating element ultrawides and zooms before, even a .05mm register discrepancy can throw the centre to edge field correction out. The article on the 70-200mm implies that the lens had to be set up in focus and then remounted on the test rig, not the method described in the link above.
I now have one A7RII body where I know the register collimation is as exact as I need for lenses down to 10mm and longer with very wide apertures. This was not the case with my A7R which needed a Tough E-mount to get correct register. I use Rosco Cinefoil to make shims. Longer lenses are, in general, less sensitive to errors in the mount and body but zooms can be very sensitive to internal alignment. I don’t intend to buy this huge lens as for me it defeats the whole point of the A7/FE system but the MTF figures look far from what any designer would want see.
Munchma Quchi ·
Every Sony I ever bought/tested and returned had some degree of sensor flange alignment that revealed itself in Imatests. On the whole about 1 in 6 was a keeper.
Carleton Foxx ·
Mother of heaven! How do I test for sensor flange/lens mount register and alignment? Does this affect DSLRs as well? Or is it only a mirrorless problem?
David Kilpatrick ·
DSLRs are affected by sensor to flange alignment, AF module to flange alignment, reflex mirror alignment, secondary AF mirror alignment, focusing screen alignment and centering, further alignments inside each lens, and also in some cases sensor carriage alignment including pitch and yaw adjustment and f adjustment… guess why half the industry really loves the mirrorless revolution!
Carleton Foxx ·
So how does one go about checking such a thing?
Timothy Renzi ·
I suggest you forget you ever read that and do not attempt to do so, lest you spend the rest of your life fretting even more about every piece of equipment you buy.
Just MHO
Carleton Foxx ·
Thanks for the reassurance. I’m sure it would make a difference, but I don’t really shoot anything that’s flat, so perhaps it’s not an issue. It does raise the question of how long do camera bodies stay in reasonable alignment. Do they start to sag after two years, three years? Or do bodies stay aligned as long as you don’t mistreat them.
Roger Cicala ·
I nearly wept in gratitude. I’ve been talking about this for years, but no one wants to hear it.
Munchma Quchi ·
Get Imatest and make sure that you have the target aligned and a stable test bed. Use a variety of lenses and all processes to avoid affecting the results. If you start to see a pattern of lower scores in one corner or one side you can guess the flange and sensor isn’t aligned. Sure it can happen in any camera. Especially after a drop with a heavy lens.
Brian Smith ·
You found 5/6 Sony a7R IIs to have sensor/flange alignment issues? What about other brands (i.e. Canon, Nikon)?
Munchma Quchi ·
Affects all cameras.
Brandon Dube ·
Four screws is mechanically overconstrained, six is only more so.
David Kilpatrick ·
The three-legged stool principle might seem sensible – zero rock – but the mount connections are not exactly robust. In practice I’ve found four-screw flanges do often show some tilt across one of the two axes created. Six-screw flanges might have another axis of tilt but in practice have fewer problems. Sony’s original E-mount body flanges were very low quality with coarse CNC machining, poor flatness and inconsistent thickness. I’ve not examined Fuji X as closely, or dismantled them. Would you say the Fuji engineering is better?
decentrist ·
Thank Roger for the testing and stop the whining. The Sony does not perform according to it’s price.
Brian Smith ·
BTW, thanks to Joey for posting these and making the effort. Much appreciated!
Mr T ·
Looking at all four corners and center.
The Sony is looking strong at 70 mm in all corners and center ok. Canon worst of the three, corners bad compared to the others.
200 mm Sony is soft in center and left side. But right side corners are clearly better than Canon. Very close with Nikon here. Nikon overall solid performance at 200 – all corners and center.
My conclusion. Bad copy of Sony lens. Nikon clear winner.
Roger Cicala ·
Mr. T – I don’t disagree the bad copy of the Sony, but this was middle of the pack from a fairly large set we tested on the bench. On the other hand, as we’ve mentioned, these were all early copies.
Hakann Vatansever ·
Roger means all the lenses in their stock is in similar condition.He cant say it loud but i think sony again screwed up with production quality….do not be early owners of this lens.period.
David Kilpatrick ·
Roger, maybe this info is there, but how do you maintain the power/status to fix the OSS mechanism in a neutral (off) but not parked position if you are not using a Sony body? Same for the focus and any related object distance compensation?
Roger Cicala ·
David, see above. We have a body wired into the bench when we test Sony lenses.
Munchma Quchi ·
Roger, – where does one typically find the best versions? Thinking there are two schools – 1. initially upon release where QC is high or 2. later when production issues are worked out. How about a series testing lenses across production dates?
Roger Cicala ·
We’ve really never seen major changes in QC optically. We have seen electronic issues addressed, like bad IS units, inaccurate AF, etc.
UnSean ·
I agree Sony looks a mess with soft points all over and varying from side to side with focal length, so either the camera is moving on it’s mount relative to the target or the lens has a moving decentering issue with focus shift, which is my opinion the problem.
The good news is that at certain points it matchs the Nikon which looks excellent.
Magnar W. Fjørtoft ·
What about testing the lenses at infinity? At a long distance? At test chard distance? And manybe even closer distances? How would this affect the test results?
Also, how will autofocus precision (!) affect real world performance, using a camera like the Sony A7rII versus a DSLR with less precise phase focus?
From what I can see, all 3 lenses are great, and any of them would for sure do well in the hands of a competent photographer. For real world use, I think aspects like autofocus speed and precision, distortion, vignetting, handling, etc., would be of more interest than slight differences in resolution wide open. What about bokeh, which is one of the priorities from Sony for this lens? People/customers would probably not be able to tell any difference between these lenses even when inspecting large format prints at a close distance!
Munchma Quchi ·
I tested the f4 version of the Sony against the Nikon and the Sony typically scored less than 50% of the Nikon. Hit rates and speed were also issues. The Sony was worthless. These were all Imatest results.
Magnar W. Fjørtoft ·
Less than 50%? Are you sure? Your own MTF measurements? In which respect? All aspects of image quality? Please explain! Sample photos are great! Or else I might think you are just trolling.
Munchma Quchi ·
I had a blog that documented this problem for more than a year called Sonyvnikon. It had over 5000 readers and then Sony’s lawyers came after me. But yes, I used Imatest and the Sony is garbage.
Magnar W. Fjørtoft ·
Saying that the Sony lens is garbage is pretty big words. I would really like to see how you could back up such a claim. Any links?
ManFay ·
What about testing the lenses at infinity? At a long distance? At test chard distance? And manybe even closer distances? How would this affect the test results?
Also, how will autofocus precision (!) affect real world performance, using a camera like the Sony A7rII versus a DSLR with less precise phase focus?
From what I can see, all 3 lenses are great, and any of them would for sure do well in the hands of a competent photographer. For real world use, I think aspects like autofocus speed and precision, distortion, vignetting, handling, etc., would be of more interest than slight differences in resolution wide open. What about bokeh, which is one of the priorities from Sony for this lens? People/customers would probably not be able to tell any difference between these lenses even when inspecting large format prints at a close distance!
Zé De Boni ·
I wonder how this article was published at LensRentals, where we usually find the best descriptions of lens performances, and the only place where I’ve seen tests of sample variations, which for me tells a lot from a product!
At first look I was shocked by the uneven lighting of the images. In a glance I questioned what the heck is that “negative vigneting”? One second later, looking at all images from the 3 lenses, I realized that the lens tester could not perform the simple task of lighting a repro subject. So far, if he cannot achieve the simplest task, what to expect from all the other technical issues needed for a reliable experiment?
I could comment other aspects of this post but as the procedures are not trustful, that would be a waste of energy and time.
I will just rather disregard and forget what is published here.
Sadly, this experience will make me think twice before following a link to LensRentals. It is a shame that they did not notice what I am pointing out!
C_S ·
i have bought both gm zoom lenses and both have issues and are back for adjustment or replacement. the 24-70 was heavy decentered my loaner lens ist much better and incredible sharp far better than my canon 24-70 2,8 II. the 70-200 has another strange problem i have never seen before. my dealer suggest that it is maybe related to a faulty IBIS for me it is simply also a decenter issue. but both lenses are very close to primes when stopped down a little and in every respect far better that my canon L glass. but for me it is clear that sony has some serious quality control issue.
Larry Templeton ·
Yes, if absolutely nothing else can be agreed upon, I think it’s safe to say that Sony has some QC issues. Their 35mm f/1.4 was (is?) all over the place wrt corner sharpness, a new 50mm f/1.4 turned up on eBay (because Sony said the cost to fix the focusing “unit” was more than the cost of a new lens), and their 85mm f/1.4 had (has?) issues with noise, and something scratching against the barrel during focus.
My 3 main cameras are Sony mirrorless, so while these issues are frustrating, I’d rather Sony addressed these issues publicly (for once) and would start standing behind their expensive, premium gear.
Pretending like issues don’t exist only breeds mistrust and a sense of unease about putting down thousands of dollars for a new lens.
For instance, did Sony address the issues with its 35mm f/1.4, or is there still a 1 in 3 chance your $1,500 lens is going to have soft corners?
Hakann Vatansever ·
85gm has no scratching issue.its just the noise of af motor used.nothing is being scratched…check LensRental’s deep investigation about this phenomena.
KuriousOranj ·
This sort of stuff is basically why I’m back to Canon, in spite of the benefits of the EVF and sensor tech. I’m a serious amateur who can’t afford a second FF Sony as back up so 2 trips to the repair centre (i.e. 7 weeks lost from less than 100) in under 2 years for my A7r was not an acceptable performance.
David Kilpatrick ·
Although I have four decades plus as a lens reviewer, I’ve never had access to the number of samples or the sophisticated methods used by Lens Rentals. I would only raise the following points: first, the cover glass issue, which is real (even between Sony models): secondly, the focus in these lenses generally is only maintained and locked as long as the lens is under power and it seems the lens once focused may be removed from a body and placed on a test rig, which would leave the focus unpowered and perhaps able to shift slightly; thirdly, the OSS even when inactive requires power to maintain the centering of its moving components, which do not ‘park’ perfectly optically centered when powered off; fourthly, this lens uses floating/compensated elements or groups and depends on an exact mount to sensor register collimation for performance corner to corner. If you were for example to focus this lens on infinity, then mount it on a manual rig and sensor-focus on a closer target leaving the lens set to infinity, the outer field resolution would be degraded (it would only be correct if the lens was refocused on an infinity target). I don’t know any way of testing this lens except on an A7RII ‘blueprinted’ for mount collimation and sensor SSI carriage adjustment. There are simply too many variables, too many moving parts, and too many factors which depend on feedback while powered up.
Roger Cicala ·
David, all are valid points. Our bench setup actually does exactly what you suggest; the lens is electronically connected to an A7Rii throughout testing, maintaining focus at infinity according to the camera. It’s worked accurately for all other FE lenses and Sony’s engineers have come here in fairly large numbers to look at it and ooh and ahh. BUT, and it’s a reasonably big but, we’ve never tested a lens with dual AF motors separately controlled like the 70-200 GM. I’m not absolutely certain the camera’s report of infinity focus is absolutely accurate in this case, although every internal test suggests it is. But that’s why I’ve never made an ‘official’ blog post about it and put so many disclaimers on these MTF results.
David Kilpatrick ·
If the lens is set to infinity focus, but your target relies on physical sensor refocusing to achieve a closer focus, you will get unwanted curvature of the focus plane and this will result in progressive lowering of MTF towards the outer field. The lens must be set to the actual reported/marked focus of the target whether this is via an optical collimation system, or an actual physical distance. Otherwise, the very small adjustments made for both the focus and zoom mechanisms to optimise group spacing at closer distances are being ignored. Can I suggest that you try focusing the lens so that it shows and reports the actual distance of your target, and lock that in, before doing another test? I may be wrong but there’s a chance I am right and only you can check this.
Brandon Dube ·
The charts are shot only on a camera. The camera and lens will behave for them exactly as they will any other scene. If there is a problem with the focusing groups on the chart, it is a firmware or mechanical issue and you should not see improvement in day-to-day images.
On the MTF bench, the mount for the lens is electronically connected to a sony camera (in this case, an A7rII). All other aspects of the MTF bench procedure are identical to any other lens; the addition is that Sony lenses are powered, and talking to the camera. As far as they are concerned they are looking at a star (at infinity); nothing more, nothing less.
David Kilpatrick ·
That’s reassuring but since Sony lenses don’t have a hard infinity position, and can not be focused unless powered, I remain doubtful (not about your test procedures, but about the ability of the Sony lens to work correctly and to position groups in an absolute relationship to the focal plane, with a relative relationship to each other).
Brandon Dube ·
…yes? They receive power from a camera just the same whether they are used on a camera (as for the charts here) or on the MTF bench. The 70-200 GM is not the first lens to feature two focusing groups – many true cinema lenses have two, at times even three focusing groups. The Leica SL 24-90 has two focusing groups as well, and its performance is stellar both on camera and on the MTF bench. I do not think there is a real problem here, only that Sony is unable to build their 70-200 as well as Nikon can build their newest one. I don’t think the MTF indicates the lens is very soft at all at 200mm it is comparable to e.g. the Nikon 105 micro, Canon 200/2.8 II prime, or even Nikon’s new 24-70 at 70mm. None of these have a reputation of being particularly soft.
Sony’s bayonet construction may play a role; I know a lot of plastic is used around the mount of at least the older cameras, though the A7rII may be updated. I think IBIS is also likely very bad for the alignment of the sensor, just as OIS is bad for the alignment of lenses. These factors in conjunction with the higher resolution of the A7rII and the invited comparison to the new best-in-class lens from Nikon make what is probably a pretty good lens, even if not a great one, get a worse rap than it deserves.
Roger Cicala ·
David, our protocol is fairly simple, but has been very effective. We confirm ‘lens focused at an infinity object reads infinity focus on camera’. We then mount the lens to the bench and using an attached camera set focus to infinity leaving the camera on to maintain that focus while we test. That has, to date, worked fine. It’s very reproducible in repeated tests of single copies, in batches, and every way we have looked at it. That includes running field curvatures, which are both reproducible and reasonable.
I have a tiny concern that the algorithms for this lens might actually be something like “Focus to infinity, then adjust compensating element to sharpest focus”. I have no reason, other than the MTF results for this lens failing to meet expectations, to think that. But I can’t be certain there’s not that, or something else I haven’t thought of, going on.
The following is my speculation only: I think the early group of lenses not meeting expectations with most being slightly decentered, followed by a lack of supply of them, might indicate there was a manufacturing problem that is being corrected. That’s the reason I haven’t spent more time on this until we get newer copies and repeat our tests with those. I asked Joey to look at this for me, hoping it would clearly prove the optical bench results were wrong or right. But it didn’t prove either, at least to my mind.
David Kilpatrick ·
Your tiny concern was what I had in mind. The whole Sony E mount system seems to depend on self compensating setting of the lenses. Personally – I’m very happy with their 70-300mm G OSS, which given its exceptional close focus is one of the best (despite the steep reduction in max aperture as you zoom). It’s as big as I ever want to go! Never tried it on a chart. Real life subjects have been sufficiently good not to need to diagnose any problems.
Munchma Quchi ·
Roger – what the hell is wrong with DXO? Are they being bribed? Inept? I lost faith in them a long time ago when their reviews on the FE55 seemed to all be calculated against pixel count and not actual tests. But now they are pissing all over your work here and have marked the Sony as the greatest zoom ever with 38mp sharpness.
john sweeny ·
Have you ever talked with a DXO “expert” I have. I have been saying it for a while now. Some pros should get together and create a new pro analysis for cameras and lenses. It takes months for DXO to even get to the new lenses. It shouldn’t be that difficult. Simplicity is the best policy
m r ·
I suppose several 70-200 gm were tested? Dumb question I am sure. Little disappointed (mainly because of price/performance aspect rather than absolute performance).
Roger Cicala ·
We picked an average one of the samples we have, which are all from very early shipments. As I’ve mentioned, though, I still wonder if later lenses are going to be better than these early ones.
FantasticMrFox ·
Do you have any plans to test a later badge of the lens in the future?
Sator Photo ·
I suspect that there may be a major methodological flaw in the test here in that the Nikon and Canon lenses are mounted via adapters to a Sony a7RII.
DSLR lenses (with the mirror box in the optical pathway) are not designed to work on a mirrorless mount with an ultra short flange distance because there is nothing in the lens formula to reduce the angle of incidence of light in the corners. So you get corner performance degradation when you adapt DSLR lenses to a mirrorless mount.
I would suggest that Roger measures MTF performance of a DLSR lens first at its native flange distance, and then adapted to a mirrorless flange distance. Example Sigma 35mm f/1.4 with and without the new Sigma FE mount adapter. I would hazard a guess that the adapter will cause significant performance degradation. This is why Sony doesn’t merely shorten the flange distance of A mount lenses when designing E mount lenses, because doing that degrades performance too much. So instead, they design entire new optical formulae from scratch for the mirrorless mount.
A mirrorless lens is just not your grandfather’s DSLR lens with a shortened flange distance!!!!
A better option when shooting test charts would be use the a7R for the Sony lens and a D810 for the Nikon, so that both are shot using a Sony 36MP sensor but with the lenses mounted natively on the optical system they were designed for.
Zé De Boni ·
Guy! Do you realize what you just said? The optical path of a DSLR lens with the mirror up position while the image is being captured is exactly the same as on a mirrorless with the proper adapter. There is no optical element on the way (just the same amount of air!) and the only thing that may cause a different result is the internal reflection, hardly a problem if the adapter has good quality.
There is absolutely no corner degradation when you use a DSLR lens on a mirrorless camera. In my own practical use and tests I got better results from my Minolta HS 300mm f/2.8 (handheld and on a tripod) on the A7RII than on the A99, despite the more demanding 42 MP sensor. That, I suspect, was due to the IBIS, which I recommend even when mounted on a tripod. My interpretation is that there is a subtle loss of sharpness because of the shutter action on the A99, like it is the case on the A7R.
Nado ·
There is absolute degradation of quality on EF lens adapted to FE. It has been well documented. Not using a native flange/mount you never have the same mating from lenses to camera. Using an adapter the lens mates to third party adapter then adapter to third part camera, two specific contributors to degradation of image quality is a flange not square to a sensor let alone the diffraction passing through an extra flange mount. Absolutely less performance. If you cannot interpret this, obviously I am not able to explain properly if that is the case.
Omesh Singh ·
Residual tilt due to the adapter will affect corner performance. If if there is sag then the top two corners will be behind center focus and bottom corners will be in front of center focus. Depending on field curvature of the lens this could have positive or negative impact on corner performance. If you add in some famous manufacturer decentering then all bets are off in terms of what one can expect in the corners. With adapted glass you can only assess performance in a small area around the selected AF point.
Sator Photo ·
Lens Rental testing by Roger Cicala has previously confirmed image degradation from the use of adapters:
https://www.lensrentals.com/blog/2013/09/there-is-no-free-lunch-episode-763-lens-adapters/
However, I think there is still some uncertainly as to the exact mechanism of this. Micro-tilt is certainly one of them. I also suspect that design mismatch resulting from a DSLR mount lens being placed on a mirrorless mount is another of them. There may also be difference arises from changes to the sensor microlens or in the design of the coverslip glass. More interesting would be to empirically document with MTF plots the exact degree and characteristic of degradation from the use of adapters.
Zé De Boni ·
Wow! When I followed the link to read this article I expected the highest level of information based on my previous experience with LensRentals. The article itself is terrible, but there are very good comments by optical science experts (like David Kilpatrick) or literates. Now I am trapped in a discussion with what looks more like a circle of goblins and gnomes believers. I will try a last explanation, sorry if I may disappoint some of you.
– The primary function of an adapter is to provide the right dstance from the lens back flange to the image plane, according to the the original design of the adapted lens, so that the lenses work exactly as projected.
– To make an adapter or an extension tube is the easiest task for the industry, based on centuries long mechanical Engineering expertise.To talk about “residual tilt” or “a flange not square” sounds like chasing ghosts! Much more could be reputed to lens centering, but that doesn’t mean flange position and it is related to that of individual lens elements (for which adapters cannot give any good or bad contribution (except those with additional lens elements, like the turbo adapters).
– Yes, the shorter flange distance allows new optical designs, for more compact lenses or other theoretical improvements (like on the Sony RX1 design). However there is nothing that restricts the design or use of lenses on which the last element sits far away from the sensor. There would be a problem of corner shading due to the tight margin of the lens mount diameter, but in practical terms I do not get this problem when using a legacy preset 800mm f/8 lens (on which the last element is about 5 inches far from the lens flange) on my A7RII.
– So far, despite all possible compromises, mirrorless cameras may use the same sensors as DSLR’s. Or just the opposite, as Sony Alpha 99II is using the same A7RII CMOS (actually, full sensor+shutter+stabilization assembly).
– Good adapters, like the ones that must be chosen by serious lens testers, have light baffling similar to those in SLR mirror boxes. In many cases their walls are more distant from the light path than the average internal wall of an SLR. The idea of “diffraction due to an extra lens flange” may arise from someone that never used such adapters, nor had the chance to inspect one closely. One more mythological entity created from the lack of knowledge.
– There are so many adapters on the market, not only for Sony but for all mirrorless cameras, from M43 to medium format! Are all those manufacturers stupid or crooks? Just for Sony APSC I found 141 models in B&H. So let’s name some of the brands that are in this so said “suspicious” game:
= Leica has the “Leica S-Adapter L for SL Camera”, “allowing you to mount a Leica S-system lens (DSLR MF) to an SL (Typ 601) mirrorless digital camera body” (full frame).
= Hasselblad has the “Hasselblad XH Lens Adapter ” so that you can “use any of your current H system lenses (DSLR MF) on a Hasselblad X system camera”, which is, by the way, mirrorless MF.
= Sony has its own line for mounting A series lenses. Sigma designs its own adapter to use Canon or Nikon mount Sigma lenses on Sony E mount cameras. More significant for those that place Canon as the supreme deity in camera and lens manufactures, this brand has now a mirrorless APSC camera system for which you may presently choose from just 6 native lenses… or you may get “the Canon EF-M Lens Adapter Kit for Canon EF / EF-S Lenses”, which “enables you to mount (any of the 82, as counted on B&H site) Canon EF and EF-S lenses onto the Canon EOS-M mirrorless digital camera”.
To finish this chapter, below is a list of adapters that I have used on the last five years and the lenses/cameras combination that they allowed me, with satisfactory results (no visible flaws, just the limitations of each lens).
– Mirex Tilt and Shift Mamiya 645 to A mount: 500mm f/4.5 (sometimes with a Mamiya 2x converter) on A77; 35mm f/3.5 on A900 and A99; 80mm f/4 Macro, 120mm f/4 Macro and 150mm f/2.8 on A99.
– Sony LAEA-1 (A to E mount): Sony A 50mm f/1.4 and Sony ZA 16-35mm f/2.8 on NEX7, Rokkinon 8mm f/3.5 Fisheye on A6000.
Sony LAEA-3 (A to FE mount): Minolta HS 300mm f/2.8 to A6000 and A7RII, Sigma Art 35mm f/1.4, Sony ZA 24mm f/2 on A7RII
– Novoflex Nikon to E mount: Nikon 10.5mm f/2.8 Fisheye, Nikon TS 24mm f/3.5 on NEX7; Nikkor non-AI 105mm f/2.5 on A6000.
– Novoflex T to A mount: Rikkenon 800mm f/8 on A99, A77 and A6000 (via an additional LAEA-1).
Now I am considering buying the new Sigma EF 85mm f/1.4 with the EF-E adapter.
m hart ·
This is a bit distressing if you’re on a B&H wait list for this lens. Have been very pleased with the GM 85 and the 24-70. Was looking forward to the 70-200 GM. Do I bail?
bdbender4 ·
I think some of the comments miss the point. The only formal test was MTF for the new Nikon 70-200. Joey took some informal shots to follow up. I draw two conclusions:
1. If I was in the market for the Nikon lens (I’m not) I would buy it.
2. If I was in the market for the Sony lens (I’m not) I would wait a while until what seems to be early teething QC problems get worked out.
For the record, I am just a private citizen with no affiliation with any camera company. Or with LensRentals, for that matter.
Roger Cicala ·
May I remind everyone that on this blog we discuss facts and do not make personal aspersions about other posters please. I know sometimes things are meant to be funny, but it doesn’t always come across that way.
Oleg ·
Well, it’s an ok lens, assuming the price is right. IMO the right price for this lens should be well under $2k.
Albert ·
I looked at the images at 100% and it looks like some decentering problem. The top right corners of the Nikon and Sony are nearly identical (close enough that no one would be complaining about the Sony), but the top left is night and day, with the Sony terribly soft, and the Nikon consistently sharp. There is no reason for the lens to be sharp in only one corner, and very soft in the other.
Munchma Quchi ·
I love these tests where ‘these leaves look sharper than those leaves’. They always greatly influence my buying decisions especially when large amounts of money are involved. Numbers schmumbers…..
Ben Nieves ·
These results lead to the question: should Sony users rely on the 70-200 F4 by Sony instead?
Carleton Foxx ·
Joey, if you really love old film cameras and film, here’s a cool idea…. Put your head together with the other great minds at LR and figure out a way to do the definitive test of film vs. digital.
But the criteria has to go beyond the boring line pairs per whatever. You need to create criteria that measure the magic of film—the gentle handling of highlights, the mysterious ability to do soft-sharp, the velvety nature of film shadows, and the thousand other ways that film is better than digital.
Keep it simple by pitting your favorite film camera and lens combo (perhaps medium or large format or even just a Nikon F) against a contemporary camera….
Who benefits? The LensRental bottom line. Visitors to the site would skyrocket which, of course, translates to higher rental rates which means more money for the owners. Everyone wins.
Plus it would be cool, and there’s literally no group of people on Planet Earth more qualified to perform such a daring and important experiment.
Lotus Eater ·
Interesting: https://www.dxomark.com/Reviews/Sony-FE-70-200mm-f-2.8-GM-OSS-lens-review
So who’s right?
Photographer100 ·
!*!*! ~ LAW OF LENS REVIEWS: Whatever DXOmark says…….the OPPOSITE is true *!*!*
Lens rentals provided TEST SHOTS…………DXOmark NEVER PROVIDES ANYTHING
Lensrentals tested MULTIPLE copies.
Thinkinginpictures ·
Well, this should be interesting….
http://www.sonyalpharumors.com/dxo-tests-new-70-200mm-f2-8-fe-gm-lens-best-performance-class/#disqus_thread
Photographer100 ·
thats based on the nonsense filth of DXOmark….BOGUS
Thinkinginpictures ·
DXO has a proper place if you understand their methodology. Have to take it in context. Besides, plenty of independent reviewers have stated the same misgivings over lensrentals findings. I’m sure Roger will follow up at some point.
Photographer100 ·
WRONG, there is no such nonsense as a “perceptual megapixel”
Lens rentals provided the PROOF above from MULTIPLE lens copies
DXO mark provides NOTHING……..100% NOTHING but their flapping lips
hey ·
lol never realize that Lensrental has issue with testing lenses?
EcoR1 ·
Now the official dxoMark results are out and they indicate problems with Lensrentals test methods. According to dxoMark the Sony 70-200 GM is sharpest 70-200 lens ever tested from any brand. https://www.dxomark.com/Reviews/Sony-FE-70-200mm-f-2.8-GM-OSS-lens-review
Brandon Dube ·
They didn’t test Nikon’s new 70-200. There is also no context for their measurements. What color light is used? What focusing distance? There is no way to predict performance at one focusing distance from information about another. Sony’s 90mm macro is kind of crap at infinity, but a lot better at macro distances. Does that make a test that reports the performance as crap at infinity wrong?
Photographer100 ·
DXOmark is NOT trusted by even a SINGLE professional photog…..theyre worse than awful
Photographer100 ·
BS……lens rentals tested multiple copies WITH TEST PICS above
DXO mark does nothing but flap their lips and has test shots of NOTHING……EVER
Roger Cicala ·
DxO Mark results are out. But they don’t indicate problems with our testing methods. We’ve redone them three times now and discussed them with Sony engineers at length.
What DxO Mark’s results probably indicates is this lens does much better close up than at infinity, where we test. Or that they got a really good copy. If I publish the MTFs for the best copy at each focal length, we’ll they’d be outstanding, too. Which is why I don’t test 10 lenses and then give you the best result out of 10.
Photographer100 ·
AHEM! Lensrentals said the Sony was BAD…..
Lens rentals provided TEST SHOTS…………DXOmark NEVER PROVIDES ANYTHING
Lensrentals tested MULTIPLE copies.
DXOmark is bogus filth
!*LAW OF LENS REVIEWS: Whatever DXOmark says…….the OPPOSITE is true !*
bbflynt ·
ePhotozine has posted their updated, very favorable review today. High sharpness marks on their MTF charts.
Sample pictures are odd in that some of them aren’t focused well. (e.g. the men bowling on the lawn — focus plane looks to be behind the guy rolling his ball.)
https://www.ephotozine.com/article/sony-fe-70-200mm-f-2-8-g-master-oss-review-30557
Piero Monteverde ·
Did you disable the OS when putting the GMaster in the tripod? It seems not to affect the Nikon but made a huge difference with the Sony. Once I disabled the OS, both lenses tested very close side by side, even having the Nikon adapted to the a7rII.
lwestfall ·
Clearly that Sony is heavily decentered with a sharp right side (at least as good as the Nikon, maybe a bit better) and not so good on the left (actually neck and neck with the Canon in most areas). How about you show the right-side MTF and not just the left-side MTF for the Sony? And have you since tested more copies of the Sony? It would be great to see how they all look in terms of MTF but also centering. It does concern me if a lot of the Sony lenses have this decentering issue. Thank you for the great work!
Danno ·
Sony published a vastly different MTF chart on their product page for the lens: https://www.sony.net/Products/di/en-us/products/lenses/lineup/detail/sel70200gm.html
Have you re-run the tests on any later copies from them? Did they ever offer a comment regarding these results?
Roger Cicala ·
Sony published computer generated MTF charts, just like all manufacturers do. Our results on actual measurements differ, like they always do. We have now run MTF tests on several different sets of 70-200 GM MTF charts and the results stand. It’s a good lens. It’s not the best in class.
Danno ·
Thanks. Just trying to understand what was up with their numbers. Do the computer generated ones represent a “theoretical” level of performance, basically?
Brandon Dube ·
Nominal or best case, yes.
Jim Heal ·
When I saw this I was rather shocked. I have just come from the DXOmark site and they have the Sony 70-200 GM lens rated as the best of the genre. So my question is…
Who is getting it wrong?
I was just about to place my order, but now I’m running scared!
Regards, Jim