LensRentals.com

Roger Buys a Camera System: A 24-70mm System Comparison

Posted by

Note: I'm going to bore people for a week or two while I decide on a new camera system for myself. To alert those who are going to be bored by posts about "Roger Buys a Camera System"; I'll put that in the title for the rest of this series. 

I don't own an SLR - I go check one out for 'testing' when I need one. But I've moved out to the country and I want a camera at the house. I can't really justify to management that I need to test a camera and some lenses for a year or two.

I know what I want: the Canon T4i's touch screen, the Canon 6D's Wi-Fi, and the Canon 5D Mk III's autofocus built around the Nikon D800E sensor, Nikon's flash systemPentax's user interface (I'd take their sensor too, if I went crop frame), and be able to mount lenses from all manufacturers. But given a far-less-than-unlimited budget, I'll be making some compromises, like everyone else. In order to make comparisons, I want to take a look at exactly how some systems differ.

Most of that doesn't involve geek stuff like this, but geek stuff is what I know best so that's where I'll start. Resolution isn't the end-all point for deciding on a camera system. It isn't even the most important point in my decision about a camera system, and I'm a resolution nut. But it is a thing I want to know about.

A Resolution Comparison

One of the things I constantly harp on is that people should not compare Imatest or DxO results on two different cameras. You can't look at the results of a lens on a crop sensor and a full-frame, for example. You can't look at results of a lens test on a Canon 5D and make good predictions of how it will behave on a Canon 5D Mk III. We've even found lately that you can't take the results on a Sony NEX-7 and extrapolate to a Sony NEX-6.

But there is one thing you can do fairly reasonably. You can compare two systems (camera and lens) to each other and determine the overall resolution of each system. I had some pretty self-centered reasons for doing just that. I, the ultimate camera system commitophobe, am going to have to buy, with my own money, a camera system. I hate when that happens.

Lensrentals.com, 2013

One of the first compromise questions I had involves shooting with 24-70mm f/2.8 lenses; this is my most commonly used lens. The highest resolving 24-70mm f/2.8 lens is the Canon 24-70 f/2.8L II. The highest resolving camera is the Nikon D800E. Since I can't mount the best lens to the best camera, I thought I'd look into how the two systems compare in final resolution.

I was fairly certain the D800E with a good Nikon lens is going to be better than the 5D Mk III with the great Canon lens. But I wasn't sure by how much. The other nice thing about working at this focal length is we have a similar lens we can mount to either camera, the Tamron 24-70 f/2.8 VC, to get a little further comparison.

Let's Look at Just the Lenses

Before we begin, I know there are some Fanboys somewhere who have stopped payment on their reality check and are stabbing pins in their Roger Effigy Doll because I said the Canon 24-70 II is the highest resolving f/2.8 lens. So let's take the camera out of the equation and compare just the lenses on our Well's Optical Bench. This means no camera mount, just evaluating the lens itself.

The following are MTF vs frequency plots for the center of the lenses in question - again, this is not Imatest data using camera images, this is purely assessment of the lenses (at infinity focus). The separation of the two graph lines shows the astigmatism of the lens. Almost all lenses have some; the Canon is truly unique in having so little. (These graphs courtesy of Aaron Closz who still gets nervous when I play with the optical bench. It's nice and predictable, though. If I want him to run some tests all I have to do is sit down and say, "where's that 70 micron reticle?" and here's there like magic.)

 

MTF (vertical) versus frequency (horizontal) of Canon 24-70 f/2.8 Mk II @ 50mm

 

MTF (vertical) versus frequency (horizontal) of Nikon 24-70 f/2.8 @ 50mm

 

MTF (vertical) versus frequency (horizontal) of Tamron 24-70 f/2.8 VC @ 50mm

 

From direct comparisons we've known the Canon 24-70 II had a higher MTF 50 than the Tamron on Canon cameras, and that the Tamron was nearly as good as the Nikon on Nikon cameras. The optical bench shows a bit more differentiation between the Nikon and the Tamron than I expected, but otherwise clearly demonstrates what we already knew. I should mention we tested 2 copies of each, all of which had already been tested using Imatest and shown to be good copies.

System Testing

Now let's add the camera systems into the mix, something Imatest is perfectly set up to do. We're going to measure Imatest MTF in line pairs / image height, as always. Since the D800E has 4912 pixels of image height compared to the Canon 5D III's 3840 pixels the Nikon should resolve somewhere around 1.2 to 1.3 X the Canon's resolution if the lenses were equal. (Several other factors, including that the Nikon does not have an AA filter, lenses aren't perfect, and the math is more complex than a simple ratio, make this a very rough estimate.)

Let's start by comparing the Tamron 24-70 f/2.8 VC on the two different cameras. We shot two copies on two bodies and averaged the results (which, btw, were nearly identical) to show MTF 50 in the center, averaged across the entire lens, and averaged in the 4 corners at f/2.8 and f/4.

These tests are all done at 50mm. I just didn't have time to set up at multiple focal lengths and 50mm is a strong area for all 3 lenses. I wanted to compare them at their best.

          

Tamron 24-70mm f/2.8 VC on Both Cameras

Center MTF50Average MTF50Corner Avg. MTF50
Canon 5DIII f/2.8810665350
Nikon D800e f/2.81085855445
Canon 5DIII f/4940710445
Nikon D800e f/41225955560

 

The MTF50 difference between the two cameras shooting the same lens is quite apparent. The difference is a bit greater in the center and a bit smaller in the corners but it is quite significant - as we knew it would be.

Canon 24-70mm f/2.8 II vs. Nikon 24-70mm f/2.8

Now let's compare the Canon camera with the Canon 24-70 f/2.8 Mk II to the Nikon camera with the Nikon 24-70 f/2.8 AF-S. The better Canon lens should offset some of the Nikon camera's superior resolution. That is exactly what happened.

Center MTF50Average MTF50Avg. Corner MTF50
Canon @ f/2.81000860450
Nikon @ f/2.81170945500
Canon @ f/41060910505
Nikon @ f/412401000570

 

The higher resolution of the D800E makes the resolution of the Nikon system superior to the Canon system, although the difference isn't as great as it was when we compared identical Tamron lenses. No real surprise here. Also not surprising, the Nikon lens is slightly better than the Tamron, although this is fairly close.

The real bottom line here is that there are no losers. The resolution numbers all of these combinations show are nothing short of amazing. For example, all three zooms are equal to, or slightly better than, the superb Zeiss 50mm f/2 Makro Planar at equal apertures on the same camera.

I'll show the f/2.8 data as our usual graph with center resolution on the horizontal axis, average on thevertical, all in line pairs / image height. I think this shows fairly well the actual resolution difference between the cameras (compare the two Tamron results) and the degree to which a better lens closes the gap.

 

MTF50 (LP/IH) at f/2.8

 

There's one other aside that is probably worth mentioning; the test fairly well confirms common wisdom. If we run SQF numbers on these resolution differences, it suggests we'd need a print size of about 11 X 16 to detect this resolution difference. At that print size we should be able to tell the Tamron mounted to 5DIII (worst performer) from the Nikon 24-70 f/2.8 on D800E (best performer) pretty clearly. We might detect the difference between the Canon 24-70 Mk II on the 5DIII and the Nikon on the D800e. On a 16 X 20 print the Canon - Nikon difference would probably be clearly apparent.

So What Did I Learn Today?

Not too much. Like everyone else I already knew the D800E with a good lens was going to out-resolve the 5D III with a great lens, but that I'd need a reasonably large print to see the difference.

It also demonstrates another thing I mention a lot: the value of any third party lens varies according to what camera you shoot. The resolution difference between the Canon and Tamron 24-70 lenses is greater than that  between the Nikon and Tamron. Right now, the price difference reflects that: the Nikon costs $600 more than the Tamron, the Canon $900.

But if you want to look at it another way, the Tamron on a D800E is about the resolution equal of a Canon Mk II on a 5DIII -- a bit sharper in the center, not quite as sharp in the corners, but pretty even. The Tamron-Nikon combination (for a guy like me looking at shelling out some major bucks soon) is $1,000 cheaper than the Canon-Canon system.

Of course, all of those prices are going to settle a bit differently in a couple of months. This is just where they are right now. And resolution is just one factor that goes in to choosing a piece of kit.

As to my ongoing search for which camera system I'm buying into, this just answers one tiny question. I've got a lot more research to do. I expect you'll be reading more about that soon.

 

Roger Cicala and Aaron Closz

Lensrentals.com

January, 2013

 

 

 

 

109 Responses to “Roger Buys a Camera System: A 24-70mm System Comparison”

Tony Bologna said:

While i too like the Tamron, i can only trust it with VC off. There have been a lot of inconsistencies with my copy at "middle" shutter speeds. If i shoot at 1/125th or faster (usually something i avoid with all VR/VC lenses)it's fine, and when i push it to the limits, say 1/10th at 70mm it's great too. But if i shoot in between it's a horrible mess. Have you noticed this issue?

KK said:

D600 vs. 6D in Low Light

Roger,

I absolutely love your analysis. Crisp, to the point, with a great level of precision. Love it. It also comes at a great time for me as I’m about to make a decision between those exact same three lenses and between the Nikon D600 and Canon 6D. I love both of those cameras. They are also newer than the 5D Mark III and the D800E, and a lot cheaper. I wish there was no AA filter in them. Perhaps the next generation of cheaper full frames…

I shoot a lot in low light (at parties and fashion shows mostly.) For that reason, I’m leaning towards the Tamron given its image stabilization. It’s also a great feature for video.

That leaves the question open between the 6D and the D600. (If I go D600, I’ll probably go with the Tamron lens. if I go 6D, I’ll consider the Canon Mark II lens for its optical supremacy, but am still leaning towards the Tamron for that image stabilization.) Any views on how their low light performance compares please? I’ve been reading conflicting views on that. Of course, this is a question to anybody who is reading this.

Thank guys!

@Bruce: I do own the Sony RX100, and absolutely love it for its low light capabilities. It’s image quality is so superior to all other pocketable cameras I‘ve ever owned. Sadly, it still cannot be a substitute for a full frame camera. I wish it was. Who wants to carry a big camera around? It’s lens is also only really fast at the wide end and goes up to f4.9 (still pretty decent at that focal length.)

KK said:

D600 vs. 6D in Low Light

Roger,

I absolutely love your analysis. Crisp, to the point, with a great level of precision. Love it. It also comes at a great time for me as I’m about to make a decision between those exact same three lenses and between the Nikon D600 and Canon 6D. I love both of those cameras. They are also newer than the 5D Mark III and the D800E, and a lot cheaper. I wish there was no AA filter in them. Perhaps the next generation of cheaper full frames…

I shoot a lot in low light (at parties and fashion shows mostly.) For that reason, I’m leaning towards the Tamron given its image stabilization. It’s also a great feature for video.

That leaves the question open between the 6D and the D600. (If I go D600, I’ll probably go with the Tamron lens. if I go 6D, I’ll consider the Canon Mark II lens for its optical supremacy, but am still leaning towards the Tamron for that image stabilization.) Any views on how their low light performance compares please? I’ve been reading conflicting views on that. Of course, this is a question to anybody who is reading this.

Thank guys!

KK

@Bruce: I do own the Sony RX100, and absolutely love it for its low light capabilities. It’s image quality is so superior to all other pocketable cameras I‘ve ever owned. Sadly, it still cannot be a substitute for a full frame camera. I wish it was. Who wants to carry a big camera around? It’s lens is also only really fast at the wide end and goes up to f4.9 (still pretty decent at that focal length.)

Eric said:

Hi Roger,
If a combination of the Canon 24-70 f/2.8 Mk II with a Nikon D800E was possible, where do you think it would land on that MTF50 graph?

LensRentals Employee

Roger Cicala said:

Derek,
In testing the Sigma comes out ahead of the Zeiss, at least at f/1.4 a bit in resolution. But the lenses have a different look and I can certainly see why some would prefer the Zeiss. For me right now, looking at buying into an entire system, the price of the Sigma means I'll probably be heading in that direction, whichever system I go with.

Roger

LensRentals Employee

Roger Cicala said:

Joh, no disagreement at all. My own particular workflows often include a good bit of image manipulation in post, all of which (including sharpening) are going to reduce resolution (meaning actual resolution, not acutance) so I find at least 16 Mpix for my work and 20 can make a difference. Of course, that doesn't stop me from lusting after 36 :-)

LensRentals Employee

Roger Cicala said:

Bill, to a limited degree, but at those apertures I think the difference in lenses would be minimized - how well the camera is going to perform with diffraction limited resolution will probably be more important. But I haven't actually tested to see.

roger

joh said:

Resolution tests such as these would have been more useful in the days when 12MP bodies were considered flagship cameras. You know, back when every little factor counted.

Clients don't compare photos from different body/lens combos for resolution. Any of the above combos can easily produce files that will satisfy clients, even when printed to 16x24 (assuming good photographic technique and post processing). And by clients, I am referring to photographers' clients, not (pixel-peeping) photographer-clients. :-)

Regardless, thanks for the test. For me, noise performance and dynamic range are far more important factors in considering a camera. My resolution needs were met back in 2008, and I don't think I'll need more than ~20MP for years to come.

LensRentals Employee

Roger Cicala said:

Michael,

It would be very much a feature for me, too, because I don't have steady hands and then compound the problem with caffeine. Although I spend all day doing geek stuff like this, when choosing a camera things like ergonomics, reliability, vibration control, accessory selection are probably more important to me than absolute resolution. But I wanted to start my search with a bit of 'see what's out there and how much difference it makes'.

LensRentals Employee

Roger Cicala said:

Hi Timmi,

You might try this: http://www.lensrentals.com/blog/2009/06/have-you-seen-my-acutance It's a starting guide to what mtf (Modulation Transfer Function) is all about.

And I apologize - I should have put some links in for that and written it out the first time.

Roger

derek said:

and one more question:
what is you consider the sharpest 35mm prime in any FF DSLR mount?
the Sigma 35mmf1.4 or the Zeiss T*1.4/35mm ZF ZE or what?

thanks in advance.

derek said:

thanks Roger , you are always the best.
after reading your test here,I just decided to go for the mount with the best 24-70f2.8 zoom or the best 85mmf1.2 or f1.4 prime(I think the Sony 85mmf1.4ZA is the best 85mm prime for SLR).
I guess it is Canon or Sony for me.
BTW, I am a Nikon shooter with a D600, a D800E and a D7000. But I have been debating selling my D800 and D7000 to get the A99v or the 5D3 since last Dec.
I have never been happy about my D800E and Nikon AFS85mmf1.4G, I'm quite fond of my new D600 for its small form factor and all around-ness , though.
I hate the ergonomics of the D800E(especially the grip), and unlike many many others , I consider my D800E as an experimental body(not very matured product).

I know many die-hard Nikon fans disagree but I really feel its lowlight AF sucks and its OVF really not as good as the OVF of the A900.
Plus, I much prefer the great EVF of the A99v ,with great focus peaking feature.
I guess my suggestion here is you might want to compare Sony vs Nikon vs Canon 85mm primes and 135mm primes.
I am quite sure as an EX-Sony shooter there is no better 135mm prime than the Sony 135mm f1.8ZA in any mount and the Sony 85mmf1.4ZA is quite amazing as well.

Alan said:

A few months ago AP magazine here in the UK ran a test of three 24-70 f2.8 lenses - Canon, Nikon and Zeiss (for Sony). Guess which won?

brent said:

have you thought about testing the sony a99 and the zeiss 24-70 against the nikon 800e and canon 5dM3?

CarVac said:

@Ralf C. Kolrausch: For non unit-focusing lenses, adapter tolerances are irritating and he would have to try a whole bunch of adapters to find one that had the right thickness for that particular Nikon 24-70 on that particular 5D3.

Joseph said:

What confuses me slightly is, if it's the body that is the limiting factor, why is it that stopping down sees such a drastic improvement? For example, in the case of the Canon, the sensor is what is limiting the numbers compared to the Nikon, however, when stopped down to F4 the number improves greatly for Canon, if the Lens' was better from the start, why wasn't it on more equal footing?

Kjeld Olesen said:

Hi

Great and interesting blog. You mention that you may venture into the comparison of a mirrorless system to one of the dSLR systems. Although not performed to the same scientific accuracy as your analysis here, you may find some inspiration in my test here http://www.acapixus.dk/photography/m43_v_Canon/
Here the 13 megapixel full frame camera performs well against the 16 megapixel MFT camera.

Rgds/Kjeld

Reinhard said:

I would like to see a best as best can comparison. A german magazine tested the D800E against the 40MP Hasselblad and said, that there is only a small difference. But only if you use the best lenses available, not the Nikon lenses...

The best result was by using the stellar 100 Macro from Zeiss. This lens is available for Canon and Nikon and if it´s really the best lens available today, this would be an interesting test. This also could be a good benchmark to compare different lenses on the same camera.

A direct comparison between Canon and Nikon lenses gives you an overview, the test with the best available lens gives you the maximum possible.

Bill said:

Can these results be extrapolated to those of us who shoot a smaller apertures? Landscape photography is often done at f/8-22 (though I'm still trying to find a lens that is sharp below f/11) due to depth of field needs.

Also how about shooting at significantly closer than infinity where in any real world image, the atmosphere will be the limiting factor? I have a view of 14,000 ft Long's Peak from my porch and there is no resolution differentiation based on camera/lens resolution. For me, I care about the 1-3 meter range which is where I do 90% of my shooting with my Sigma 180 macro and Nikon 80-200 f/2.8D. At these distances, resolution differences are very obvious. My guess is that the Zeiss 50mm macro would quite noticeably outperform the lenses in this test at a 1 meter shooting distance.

Many thanks for a thought provoking article a great feedback as well.
Bill

Timmi said:

OK, my first time here... I have no idea what MTF means. Perhaps a mention of what it is?
I tried saving you the trouble by doing a search... but I can't figure out if you refer to the lense's Manual Transmission Fluid, Make Trade Fair, Musculoskeletal Transplant Foundation, or what...

Ralf C. Kohlrausch said:

Hi Roger,

while you can't put the Canon lens on the Nikon Camera, you can do it the other way round and adapt the Nikon lens to the Canon. That would rule out differences in sensor-technique and in-camera-processing. C'mon, you're a geek, are you not?

Greets
Ralf C.

Michael said:

The Tamron is listed as having VC. Is this a feature you would use in your decision making? I don't have a steady hand so have always relied on VR/VC/OIS ;)

Mark said:

I mean who cares which one is marginally sharper? Do you want a 21mp file or a 36mp file? That's the question you have to ask. Of course the D800E is sharper but so is my 8x10 camera. I'm not advising anyone looking at a 5D3 to consider a Deardorff becasue one might be sharper. It's just different. I'm sure the D800e creates a great file but as a wedding photographer I'd never use one. What am I gonna do with all that data? Sharpness is such a meaningless metric.

Jeremy said:

"If I can get 90% of the best for 60% of the price, I’ll probably go that way. But I have no idea if that’s possible yet."

This bang for the buck statement almost always leads to Pentax, currently the Pentax k-01 w/16-50/2.8 at 30% price. At this point the resolution across systems is the minor point of system selection, and other factors (e.g. ergonomics, reliability, portability) begin to dominate the process.

Otherwise based on resolution, system selection is solely based on which canikon system has the newest combo. Thanks for the analysis, interesting to see that these lenses were on par with the prime for resolution.

AJ said:

Hi Roger,
A simple, concise and practical series of tests.
Lensrentals is my premier source of information on lenses.
Just wish you had the time to do more comparisons like this ...
I appreciate the technical aspects of the lenses but it would be appreciated if there were some subjective comments as well on intangibles like 'drawing style' and bokeh - but I guess that's a tall order by it's very nature.
You had briefly commented on the Zeiss 50 mm being some-what 'inferior' to the zooms IRO MTF but that's not the full IQ story I imagine.
Anyhow, the information you provide on your site, in addition to the evaluations/comparisons et al, is very much appreciated.
Warm regards.

Mark said:

Could you explain the resolution of lenses? Is this just a function of the quality of the build/glass, or is there more to it?

LensRentals Employee

Roger Cicala said:

Frans, that is wonderful work. Very interesting to know.

Francois said:

I am always very supicious when comparisons in between MTF50 numbers are made.

A comparaison of different lenses on the same body, using the same RAW conversion is usually quite good.

When you compare two different bodies, it becomes very dangerous. First, you need to use the same RAW converter. I assume that you use Imatest that uses dcraw internally and does not sharpen the images at all. When you use that process and you compare a D800 and a D800E, you will see that the D800E comes ahead by 10%. But it seems the information in the D800 file is still there, and can be recovered by deconvolution. At the end of the day, what counts is what you can get from an image, not what it is when everything is at 0.

There is a huge difference in between the blur produced by a weak lens and the blur produced by an anti-aliasing filter. For the one introduced by the first one, there is no way to recover the information in post-processing. Although it might still be there, as nobody knows the point spread function of the lens, you are dead. The blur introduced by an anti-aliasing filter is very regular, and some sharpening algorithms almost recover it.

So MTF50 results for D800 and D800E are quite different, but the information contained in the files is almost the same. That's one of the reason I sill have doubts about what this kind of experiment means.

Leave a Reply