Resolution Tests

Roger Buys a Camera System: A 24-70mm System Comparison

Published January 29, 2013

Note: I’m going to bore people for a week or two while I decide on a new camera system for myself. To alert those who are going to be bored by posts about “Roger Buys a Camera System”; I’ll put that in the title for the rest of this series. 

I don’t own an SLR – I go check one out for ‘testing’ when I need one. But I’ve moved out to the country and I want a camera at the house. I can’t really justify to management that I need to test a camera and some lenses for a year or two.

I know what I want: the Canon T4i‘s touch screen, the Canon 6D‘s Wi-Fi, and the Canon 5D Mk III‘s autofocus built around the Nikon D800E sensor, Nikon’s flash systemPentax’s user interface (I’d take their sensor too, if I went crop frame), and be able to mount lenses from all manufacturers. But given a far-less-than-unlimited budget, I’ll be making some compromises, like everyone else. In order to make comparisons, I want to take a look at exactly how some systems differ.

Most of that doesn’t involve geek stuff like this, but geek stuff is what I know best so that’s where I’ll start. Resolution isn’t the end-all point for deciding on a camera system. It isn’t even the most important point in my decision about a camera system, and I’m a resolution nut. But it is a thing I want to know about.

A Resolution Comparison

One of the things I constantly harp on is that people should not compare Imatest or DxO results on two different cameras. You can’t look at the results of a lens on a crop sensor and a full-frame, for example. You can’t look at results of a lens test on a Canon 5D and make good predictions of how it will behave on a Canon 5D Mk III. We’ve even found lately that you can’t take the results on a Sony NEX-7 and extrapolate to a Sony NEX-6.

But there is one thing you can do fairly reasonably. You can compare two systems (camera and lens) to each other and determine the overall resolution of each system. I had some pretty self-centered reasons for doing just that. I, the ultimate camera system commitophobe, am going to have to buy, with my own money, a camera system. I hate when that happens.

Lensrentals.com, 2013

One of the first compromise questions I had involves shooting with 24-70mm f/2.8 lenses; this is my most commonly used lens. The highest resolving 24-70mm f/2.8 lens is the Canon 24-70 f/2.8L II. The highest resolving camera is the Nikon D800E. Since I can’t mount the best lens to the best camera, I thought I’d look into how the two systems compare in final resolution.

I was fairly certain the D800E with a good Nikon lens is going to be better than the 5D Mk III with the great Canon lens. But I wasn’t sure by how much. The other nice thing about working at this focal length is we have a similar lens we can mount to either camera, the Tamron 24-70 f/2.8 VC, to get a little further comparison.

Let’s Look at Just the Lenses

Before we begin, I know there are some Fanboys somewhere who have stopped payment on their reality check and are stabbing pins in their Roger Effigy Doll because I said the Canon 24-70 II is the highest resolving f/2.8 lens. So let’s take the camera out of the equation and compare just the lenses on our Well’s Optical Bench. This means no camera mount, just evaluating the lens itself.

The following are MTF vs frequency plots for the center of the lenses in question – again, this is not Imatest data using camera images, this is purely assessment of the lenses (at infinity focus). The separation of the two graph lines shows the astigmatism of the lens. Almost all lenses have some; the Canon is truly unique in having so little. (These graphs courtesy of Aaron Closz who still gets nervous when I play with the optical bench. It’s nice and predictable, though. If I want him to run some tests all I have to do is sit down and say, “where’s that 70 micron reticle?” and here’s there like magic.)

 

MTF (vertical) versus frequency (horizontal) of Canon 24-70 f/2.8 Mk II @ 50mm

 

MTF (vertical) versus frequency (horizontal) of Nikon 24-70 f/2.8 @ 50mm

 

MTF (vertical) versus frequency (horizontal) of Tamron 24-70 f/2.8 VC @ 50mm

 

From direct comparisons we’ve known the Canon 24-70 II had a higher MTF 50 than the Tamron on Canon cameras, and that the Tamron was nearly as good as the Nikon on Nikon cameras. The optical bench shows a bit more differentiation between the Nikon and the Tamron than I expected, but otherwise clearly demonstrates what we already knew. I should mention we tested 2 copies of each, all of which had already been tested using Imatest and shown to be good copies.

System Testing

Now let’s add the camera systems into the mix, something Imatest is perfectly set up to do. We’re going to measure Imatest MTF in line pairs / image height, as always. Since the D800E has 4912 pixels of image height compared to the Canon 5D III’s 3840 pixels the Nikon should resolve somewhere around 1.2 to 1.3 X the Canon’s resolution if the lenses were equal. (Several other factors, including that the Nikon does not have an AA filter, lenses aren’t perfect, and the math is more complex than a simple ratio, make this a very rough estimate.)

Let’s start by comparing the Tamron 24-70 f/2.8 VC on the two different cameras. We shot two copies on two bodies and averaged the results (which, btw, were nearly identical) to show MTF 50 in the center, averaged across the entire lens, and averaged in the 4 corners at f/2.8 and f/4.

These tests are all done at 50mm. I just didn’t have time to set up at multiple focal lengths and 50mm is a strong area for all 3 lenses. I wanted to compare them at their best.

          

Tamron 24-70mm f/2.8 VC on Both Cameras

 Center MTF50Average MTF50Corner Avg. MTF50
Canon 5DIII f/2.8810665350
Nikon D800e f/2.81085855445
Canon 5DIII f/4940710445
Nikon D800e f/41225955560

 

The MTF50 difference between the two cameras shooting the same lens is quite apparent. The difference is a bit greater in the center and a bit smaller in the corners but it is quite significant – as we knew it would be.

Canon 24-70mm f/2.8 II vs. Nikon 24-70mm f/2.8

Now let’s compare the Canon camera with the Canon 24-70 f/2.8 Mk II to the Nikon camera with the Nikon 24-70 f/2.8 AF-S. The better Canon lens should offset some of the Nikon camera’s superior resolution. That is exactly what happened.

 Center MTF50Average MTF50Avg. Corner MTF50
Canon @ f/2.81000860450
Nikon @ f/2.81170945500
Canon @ f/41060910505
Nikon @ f/412401000570

 

The higher resolution of the D800E makes the resolution of the Nikon system superior to the Canon system, although the difference isn’t as great as it was when we compared identical Tamron lenses. No real surprise here. Also not surprising, the Nikon lens is slightly better than the Tamron, although this is fairly close.

The real bottom line here is that there are no losers. The resolution numbers all of these combinations show are nothing short of amazing. For example, all three zooms are equal to, or slightly better than, the superb Zeiss 50mm f/2 Makro Planar at equal apertures on the same camera.

I’ll show the f/2.8 data as our usual graph with center resolution on the horizontal axis, average on thevertical, all in line pairs / image height. I think this shows fairly well the actual resolution difference between the cameras (compare the two Tamron results) and the degree to which a better lens closes the gap.

 

MTF50 (LP/IH) at f/2.8

 

There’s one other aside that is probably worth mentioning; the test fairly well confirms common wisdom. If we run SQF numbers on these resolution differences, it suggests we’d need a print size of about 11 X 16 to detect this resolution difference. At that print size we should be able to tell the Tamron mounted to 5DIII (worst performer) from the Nikon 24-70 f/2.8 on D800E (best performer) pretty clearly. We might detect the difference between the Canon 24-70 Mk II on the 5DIII and the Nikon on the D800e. On a 16 X 20 print the Canon – Nikon difference would probably be clearly apparent.

So What Did I Learn Today?

Not too much. Like everyone else I already knew the D800E with a good lens was going to out-resolve the 5D III with a great lens, but that I’d need a reasonably large print to see the difference.

It also demonstrates another thing I mention a lot: the value of any third party lens varies according to what camera you shoot. The resolution difference between the Canon and Tamron 24-70 lenses is greater than that  between the Nikon and Tamron. Right now, the price difference reflects that: the Nikon costs $600 more than the Tamron, the Canon $900.

But if you want to look at it another way, the Tamron on a D800E is about the resolution equal of a Canon Mk II on a 5DIII — a bit sharper in the center, not quite as sharp in the corners, but pretty even. The Tamron-Nikon combination (for a guy like me looking at shelling out some major bucks soon) is $1,000 cheaper than the Canon-Canon system.

Of course, all of those prices are going to settle a bit differently in a couple of months. This is just where they are right now. And resolution is just one factor that goes in to choosing a piece of kit.

As to my ongoing search for which camera system I’m buying into, this just answers one tiny question. I’ve got a lot more research to do. I expect you’ll be reading more about that soon.

 

Roger Cicala and Aaron Closz

Lensrentals.com

January, 2013

 

 

 

 

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Resolution Tests
  • Roger Cicala

    Derrick,

    Momma said if I can’t say something nice . . . .

    Just kidding. Actually it’s not a bad lens at all, but I don’t have a nice comparison point since I don’t have the Tamron in Alpha mount. From the optical bench I’d say it’s nearly as good as the Tamron, which is quite good. That doesn’t help factor in the Sony sensor, which is also superb so I wouldn’t be surprised if the Sony combo was at least as good as the Tamron-Canon. But I’ll try to get them done one of these days.

    Roger

  • Roger Cicala

    Tristan, from some other testing we’ve done, it appears roughly a 10% difference.

  • Tristan

    Hi Roger
    I love the articles you write for your blog always a great read.
    This article has left me with a question though.
    How does a D800 system compare to a D800e system?
    or, I guess you could ask. How much resolution do you give up with the AA filter?
    Hypothetically speaking.

  • Derrick

    Can we get some Sony CZ 24-70 love up in here!!! Sony always gets over looked!

  • Roger Cicala

    Thank you – I’ve corrected it. The D800e was the Nikon camera in all tests. I have a tendency to think I can telepathically project what I mean without typing things properly.

  • Bruce

    Sony DSC-RX100? Fuji X100S?

    Do your homework, by all means, and keep up the great posts, but then make life easy on yourself and pick something like one of the above! 😉

    (The devil made me do it.)

  • drgradus

    Does the VC add any value to the kit for you? It’s easily over a stop, so the resolution increase from 2.8 to 4.0 wipes out any difference at 2.8, for most intents and purposes.

  • n/a

    I strongly suggest to reevaluate yours perception about things you can or can not live with out. I have to many friends who went mirrorless route and found they can not live with out optical viewfinder. I have to many friends who bought in all holy trinity and no matter how good these lenses are – they gather dust, because they are big pigs. Not so long a go, I was sure fx is not for me, I was in love with 18-200 versatility (28-300 not an option – to big) – and now am on d600 covered with 1.8g primes wondering why I waited for so long (gorgeous viewfinder, bright and razor sharp lenses, lightweight setup)…

    You are in unique position to extensively test all possible combos you might like – do so / combat all perceptions 🙂

  • MFG

    Hey Roger, you constantly mix up the D800 and the D800E in this article. And in the first paragraph you tell us that you CAN’T make these predictions(eg. Comparing two differnt cameras).

    Did you forgot the “E”?

    D800 and D800E is not the same camera; or is it?

  • Aaron

    @Roger, Frans van den Bergh,

    Thanks for the information, that’s good to know. So Imatest likely won’t be a real significant piece of information for how a system performs at the super-high ISO’s available these days. I’ll just have to figure out another way to see if I should stick with the Tamron 24-70, or hold out for the Canon 24-70. What a horrible choice to have to make 🙂

  • Sounds great. Eager to read your thoughts along the journey!

  • James Scholz

    Roger,I love your tests and comments and look forward to each new entry.

    Relative to how prints look I have a comment. I shoot architecture for a living and generally give the client a large print, normally 20×30, with the completion of each job. I have used the same Nikor 14-24 2.8 wide angle, and PC lenses with 3 full frame cameras over the years, the D3, D3x, and now D800e. I always shoot with sturdy tripod, mirrow lock up, cable release, etc. With out a doubt the clarity and dynamic range of the images has improved dramatically with each new system. Sensors make a big difference in large print appearance as your tests indicate.

  • Roger Cicala

    Samuel,

    Next up, I think, is a bit of price comparison – what a full-frame SLR, crop sensor SLR, and mirrorless system is going to run to meet my needs, along with which systems just don’t have lenses or other aspects that do what i want. I’ve can do the lens selections and camera aspects from memory pretty much, but honestly I’ve lost touch with what a system is going to cost. Other than more than I want it to.

    Then sort of screening out cameras for things like dynamic range, ISO performance, etc. – looking at what I stand to gain or loose with various systems and sensor sizes.

    Hopefully after that I’ll have a couple of candidate systems in full-frame, crop-sensor, and mirrorless and some ideas about cost. Then I’ll start doing the things you suggest. And finally take a couple of finalists out shooting.

  • Oh, and if you’re considering crop cameras too, a fullframe vs APS-C comparison should be around the corner, right?

    I ran my own tests and I’m pretty sure FF is going to end up sharper, even if you keep pixel count about the same, but I’ve had some people complain that my lenses are weird and the “you only use the sweet spot of the lens” argument is bigger than my “you’re magnifying all the issues” one (they point to slrgear results, which I guess are relative but they say they’re absolute).
    http://www.similaar.com/foto/lenstestsff/lenstestsff.html

  • We’re all very lucky that you need to buy into a camera system, we stand to learn a lot here 🙂

    So, next up, AF tests comparing Canon-Canon vs Nikon-Nikon vs Nikon-Tamron, right?

  • Samuel

    Frans, thanks for the insight. This leads to a new question – how much less reliable are the MTF50 values at ISO 400 vs 100? How many sets of measurements should one make and average to reduce the error?

  • @Samuel: Changing ISO has less effect on Imatest scores than you might expect. The Slanted Edge algorithm used to measure resolution averages quite a few pixels (following lines parallel to the edge being tested) for each edge profile sample, i.e., there is significant high-frequency noise reduction built into the algorithm. Because of this, the MTF50 values do not respond to per-pixel noise in the same way a human observer would.

    You are more likely to see an increase in the standard deviation of MTF50 values (over repeated images of the same location on test chart) with increasing ISO, rather than a clear trend in the mean MTF50 value (over repeated images of the same location on the test chart). In plain English: MTF50 values do not necessarily decrease with increasing ISO (up to a point); they just become more unreliable.

  • Samuel

    Roger, how significant are the differences in the resolution figures if the testing was done at ISO 100 instead of 400?

  • Gav

    “On a 16 X 20 print the Canon – Nikon difference would probably be clearly apparent.”

    Does anyone seriously believe this? There may be detectable differences there, but I doubt a human could do it.

  • Benny

    All I see is that Tamron, Canon and Nikon 24-70 are all underutilized even on a 22 and 36mp sensors.

  • Tim Glaser

    *sigh*

    3 reasons I disagree with that. I have trouble counting, and I don’t see an edit button.

  • Tim Glaser

    I’m curious how the D800(E) vs 5dmkIII would compare using the new Sigma 35mm 1.4. I think if I was in the market to splash $$ around like that, that lens would probably be the first place for me to go.

    Secondly, to address a comment above, where it was suggested to go with Canon because the 24-70mm lens is the best lens, there are 2 reasons I disagree with that:

    1) Canon and Nikon both just released new bodies [it will be quite some time ‘hoping’ that Canon releases a better camera.]

    2) Using that reasoning you could say that since Nikon glass is 6 years old or so now, there are more likely to bring out an update to that lens which will substantially improve it.

    3) Obviously no camera system choice will be made on just one lens. For example someone planning on buying mostly used lenses will probably prefer Canon because there is always a better selection of used Canon glass. But someone else who wants the 14-24 f/2.8 Nikkor isn’t going to find much compelling at all about the Canon system. Same could be said for some one wanting the 50 1.4L, which is clearly better than the 50 1.4G.

  • Eric Lew

    Roger – I just wanted to say I absolutely LOVE your blog. I the tests you run, the caveats you include – everything! Amazing work! What a contribution to the internet!!

  • Roger Cicala

    Aaron,

    We actually test at ISO 400. We thought that was close to real world, but once we crank up ISO we’re testing the cameras way more than the lens.

    Lighting is also critical – the chart has to be lit with a maximum 1 lumen variation.

    So I’m afraid we have to extrapolate Imatest results to more real world settings. But you’re welcome to come play with our toys if you like. You’d be just like us, though: grabbing a couple of hours of testing time before the backlog of rental lenses needing routine testing makes you go back to work :=)

  • Marty

    Both the Nikon and Canon sensors out-resolve the lenses at the center so the Nikon pair has higher central resolution than the Canon. Assuming that the sensor resolution is constant from center to edge both sensors would appear to out-resolve the lenses at the corners (e.g. resolution on the Canon drops from 1000 to 450, and Nikon from 1170 to 500 so the lenses are the limiting component, than why does the Nikon pair still have higher resolution at the corners even though the Canon lens has a higher corner MTF?

  • Aaron

    I’m guessing that the Imatest is done at ISO100. What would happen if you were to go up to the highest ISO? In poor (but still usable) lighting? And then resize both for 8×10 or 11×16 prints?

    I know it’s a ton of work (I’ve got some vacation time…can I come up and play with your tools for a bit?), but while resolution and performance in good light is valuable, I think most of us shoot a bit higher than ISO100, and I’m regularly going up to 3200+, and often 6400+ for some of my shoots (live events).

    I’ve used the Tamron 24-70 for once, and the 2.8 with VC was great, and I’m renting it again, but given how every single shoot is quite a bit different, it’d be hard to rent the Canon 24-70 v2 and do exact apple-to-apple comparison of resolution/quality, I’d love it if someone who has the tools and the itch to gather lots of data could do it. How about it? Pretty please?

  • Roger Cicala

    David I totally agree with the adage. But I’m just beginning this decision making process. A resolution reality check was step 1.

    I’m also cheap 🙂 I’m still considering both full-frame and crop-sensor cameras for my needs. I guess the process in my mind starts with “here’s the best”, now how close to the best can I get for how much less than that price. If I can get 90% of the best for 60% of the price, I’ll probably go that way. But I have no idea if that’s possible yet.

    Also, I’ve learned being in this business for years. I don’t do hope.

  • So if you were going to put down several thousand $’s wouldn’t you be better of with the canon lens and hope canon releases a higher resolving camera in the future? you know, the old adage to invest in your lens’s? I would think the canon 24-7 f2.8 with canon 6d would be an interesting compromise?

  • Roger Cicala

    Patrick,

    I don’t think there’s as much difference between the 70-200 f2.8 lenses, so I expect the resolution difference would be larger in favor of the D800e. But I’ll double check.

  • Patrick

    Great analysis, thanks! I was wondering how the Canon and Nikon 24-70 f/2.8 compare.

    Now if you could only test the 70-200mm f/2.8 . . .

Follow on Feedly