It’s impossible to accurately compare lenses from different mounts using any form of Computerized Target Analysis testing (methods like DxO or Imatest). Target analysis tests an entire system (camera and lens). People try to, of course, but it’s not accurate since you always have the added variables of camera sensor, microlenses, in-camera image processing, etc. Some people try to use adapters to test different lenses on the same camera, but then you have added variables from the adapter and sometimes from sensor stack thickness.
In general, it doesn’t really matter. If you shoot Nikon you aren’t really interested in Canon lenses, and vice-versa. Still, directly comparing the lenses without any camera involved is interesting to some people, I think. Some people are thinking of changing brands – they know the other brand’s camera has higher resolution, but aren’t sure if the other brand’s lenses are as good. More people than ever are shooting lenses across brands using adapters, and they’re sometimes curious, too.
It is, of course, possible that nobody but me is interested in direct cross-brand lens comparisons. But since I am interested and since I have to test all of the lenses in our inventory anyway, I thought I’d show some of that data. If others find it interesting, too, I’ll write up some more comparisons like this.
I chose these specific lenses because I get asked every so often whether a 24-70 f/2.8 lenses at 70mm is sharper than the 70-200 f/2.8 lenses at 70mm. This again is a bit of a moot point, since there’s actually a focal length difference (the 24-70s are about 67mm when the lens says 70, while the 70-200 lenses are closer to 73mm).
But hey, I usually find the opportunity to do a meaningless test hard to resist.
We tested 7 copies each of the Canon 24-70 f/2.8 Mk II, Canon 70-200 f/2.8 IS II, Nikon 24-70 f/2.8G AF-S, and Nikon 70-200 f/2.8G AF-S VR II lenses on our Trioptics Imagemaster optical bench. All copies had been through our routine optical and Imatest screening and passed with flying colors. These are all pretty modern designs (the Nikon 24-70 is the oldest, released in 2007) and I don’t think we’ll be seeing a replacement for any of them any time soon. All were tested at their 70mm settings according to the zoom ring, at f/2.8, and at infinity focus setting. (The actual focal lengths were about 67mm for both the 24-70 lenses, 73mm for both of the 70-200 lenses.)
One thing about looking at MTF charts that people either lover or hate is that it gives a lot more data than just Imatest resolution numbers. It’s so easy to look at Imatest numbers and say, “A is 875 and B is 820, so A is better.” MTF charts give you a lot more information. That generally makes the summary a little more complex, which some people hate. But it also makes the summary a lot more useful.
If you don’t speak MTF, don’t worry. It’s not hard. Higher on the vertical axis is better. Dotted and solid lines of the same color close together are better (far apart is astigmatism). The horizontal axis goes from the center of the lens at “0″ to the edge of the lens at “20″.
Legend for all the MTF graphs
Painting with a broad brush, the MTF graphs show that one thing that most people assumed is accurate. At 70mm, the Nikon 24-70 f/2.8 G isn’t quite as good as the other three lenses. The Nikon 24-70 is a good lens, but the other 3 are absolutely awesome lenses.
The MTF charts showed several things that I didn’t expect, though. First among those is that the Canon 70-200 f/2.8 IS II is actually a bit better at 70mm than the Canon 24-70 f/2.8 II. Imatest results on a large number of copies gave a slight edge to the 24-70.
There are a number of reasons for the difference. First, optical bench tests are done at infinity focusing distance, while the Imatest results were at about 15 feet. Second, Imatest shows only the MTF50 — a snapshot along the 0.5 line on the vertical axis of the MTF chart. Finally, Imatest results show a lot of averaging that the MTF graphs don’t do. I usually lump the readings for center, average of the entire lens, and average of the corners, for example. Additionally, Imatest doesn’t separate out astigmatism (the difference between sagittal and tangential lines). On the other hand, Imatest can look a bit further out into the corner than on the optical bench does.
Going into this test I expected the Nikon 70-200 f/2.8 VR II was as good in the center as the Canon lenses, which it is. I thought it would be a bit weaker off center, but it’s not that simple. If you look at just the sagittal numbers (solid lines) the Nikon has a steady fall-off, while the Canon gets softer, then sharper. So in the area from about halfway to 2/3 of the way to the edge of the image the Nikon is better, while the Canon gets better out on the edges. The Nikon also has less astigmatism.
We also ran field curvature tests on all the lenses. If you haven’t seen these graphs before, they’re a bit different than the MTF curves. (You can read more about them here.) Most people think of field curvature in kind of general terms, but tend to forget that the tangential and sagittal fields can have very different curvature and that the curves can be complex or simple. A perfectly flat field is actually rather rare.
The field curvature graphs give us a lot of information about the lens that we can’t really get any other way. Basically these show what the area of sharpest focus across the lens looks like, assuming you had carefully focused on the center point.
Notice the Nikon 24-70 has some significant field curvature. It’s going to be very difficult to get the center and corners both in sharpest focus without stopping down quite a bit. The Canon 70-200 has a more complex ‘sombrero’ type sagittal curvature with a simpler tangential curvature. The Canon 24-70 Mk II and Nikon 70-200 VR II both have impressively flat fields of focus.
The sagittal field curvature on the Canon 70-200 f/2.8 IS II explains a bit about its MTF chart. If you look at the sagittal field curvature graph notice how the area of sharpest focus has curved away from the “0″ line between 10mm and 15mm from the center, then curves back so it’s in sharp focus again at 18mm. Now go back and look at the sagittal MTF curves for the lens and you get some idea as to why it gets softer, then sharper, then softer again.
You might have noticed a bit of tilt in a couple of the field curvature examples. These are minor and would be impossible to detect in optical testing. Just to reassure you, below are the tangential and sagittal graphs for a Nikon 24-70 f/2.8 that was tilted enough to be obvious in an image (it is not one included in the testing).
Field curvature graphs for a decentered and tilted lens.
The Best Lens?
Life was nice and simple when we averaged some Imatest numbers and easily declared a lens ‘better’ or ‘sharper’ than another. We can still do that, of course. When there’s a big difference between lenses it’s just as apparent on the MTF bench as it was with Imatest. When we compare a couple of good lenses, though, the information is not so much ‘better and worse lens’, but rather ’stronger and weaker points’ of each lens.
These lenses provide a good example. When I just provided MTF numbers, I found the Canon 24-70 f/2.8 a bit better than the Canon 70-200 f/2.8 IS II at 70mm. Very close, but a bit better. Comparing them here my original thought was things might be different at infinity focus compared to the closer focus used for Imatest. But the additional information we get with bench testing shows us more than just better or worse. The 70-200 is a bit sharper in the center while the 24-70 has a much flatter field.
If you look at the MTF chart there’s not much question the Canon 24-70 has a higher MTF than the Nikon 24-70 in most locations and at most frequencies. However, the Nikon has less astigmatism in the outer areas, and that may give it a ‘look’ that some prefer over the absolute resolution of the Canon.
Before doing this test, if you had asked me I would have told you I thought the Canon 70-200 was just a bit sharper than the Nikon. It might be, by just a tiny bit in the center. Away from center, though, the Nikon 70-200 is better at higher frequencies. The Nikon 70-200 also has less astigmatism in the outer edges and a flatter field than the Canon.
The field curvature in the edges of the 70-200 f/2.8 IS II at 70mm would make some people (including me) prefer the very flat field of the Canon 24-70 f/2.8 Mk II, at least at 70mm. It depends what you use the lens for. For portraiture or action sports, it would be completely meaningless but I do some architectural shooting at 70mm.
Saying one of these three lenses is clearly better than the other at 70mm is rather silly. They have some different characteristics; some slightly different strengths and weaknesses. The reality is if I shot Nikon and carried these two zooms, I’d use the 70-200 for work around 70mm. If I shot Canon and carried these two, I’d prefer the flat field of the 24-70 once in a while, need the IS of the 70-200 once in a while, but generally wouldn’t worry about changing lenses to get a 70mm shot.
Still, I find the comparisons interesting, even if not very useful, and we’ll be doing some more. And who knows — someday when I’m shooting my interchangeable mount 50mpix SLR I might actually be making a decision as to whether I want the Zeiss, Canon, or Nikon lens for it in a given focal length.
I haven’t posted very much lately. We’ve had some new equipment installed and we’ve been doing a LOT of testing as we develop our new database of lenses on the optical bench. As the database fills out I’ll be posting more than ever, just because a lot of this stuff is just fun. Today’s post is largely for fun, but will have some additional interest for those who shoot Leica or shoot M-mount lenses adapted to other cameras.
One thing that optical bench testing gives us that is hard to find elsewhere is a clear map of field curvature. We had a client interested in determining field curvatures for a several M-mount lenses and thought there would be a few among you who also wanted to see them.
What These Graphs Are
The graphs are pretty simple: the machine finds the best focus point in the center of the lens (“0″ on the vertical axis). It then measures 20 other points from one side to the other of the field, finding the best focus and highest MTF at each point. The relative MTF is shown by color (white>red>orange>yellow, etc.). The focus position compared to best center focus is shown on the vertical axis. The horizontal axis shows position from the left side of an APS-C size sensor to the right.
These lenses were all tested at f/4 to level the playing field for the wider aperture lenses. But this means the field curvature wide open would probably be larger than what you see here. They were done at infinity focus, so the field curvature might be a bit different at shorter focal lengths.
One thing these graphs will show (that you probably don’t really want to know) is that a lot of lenses have a very slight bit of tilt to the field. These are all good copies, tested multiple times. The tilt that is noticeable on these graphs isn’t noticeable in real-world photography, at least not without a great degree of pixel peeping.
The other thing that you may not have thought of is that the sagittal and tangential fields often have different field curvature.
Does this have real-world implications? Yes. The lens with wicked field curvature may give amazingly sharp portraits, but not sharp landscapes or architectural shots, for example. I’m sure someone is going to ask something like, “Well, now many feet does a 100 micron focusing distance equal at infinity?” I don’t have the math to answer that question and don’t have time to go look it all up, but if one of you wants to we’d welcome your input.
Some wide-angle M-mount lenses.
First we’ll show 4 wide-angle lenses. You may notice the Leica 18mm is very mildly tilted, although this is not something you’d notice in a photograph. The Voigtlander 21mm is a good example of a lens with quite different sagittal and tangential curvatures.
A few that are not quite that wide.
The Leica 28mm f/2.8 gives us a nice example, at least in the sagittal field, of a lens with double (sometimes called Sombrero) curvature.
And lastly some 35mm lenses
The Voigtlander 35mm f/1.4 (and remember, this is stopped down to f/4) shows some pretty wicked curvature. Because I know some Voigt fanboy is going to tell me his 35mm has no field curvature I’ll go ahead and tell you that I tested 5 copies and they were all identical. The double field curvature seems to be pretty much standard for these M-mount 35mm lenses.
I don’t have any dramatic conclusions to add, other than I think this is a very useful tool. We’ll be presenting field curvature graphs on all of our lens reports going forward. I’ll also apologize in advance to all of you who want to see the curvature of some specific lens or other. We have over 150 more lenses that need to be tested, minimum of 8 copies of each one, and the zooms at 3 different focal lengths minimum. I’m just not in a position to take requests right now. But we’ll be publishing more of them soon.
I try to start these articles by putting my preconceptions out there first. Every reviewer or blogger has them, they affect our opinions, and you have a right to know them. So I’m writing this introduction the day before our first copies arrive.
The HandeVision IBELUX 40mm f/0.85 is designed by IB/E Optics GmbH in Germany and manufactured by Kipon (aka Shanghai Transvision Photographic Equipment Co. Ltd). IB/E has developed a number of lenses and adapters for the Cinema world and other optics, so I figured the design would be good; probably a telecentric lens with a built-in Speedbooster-type element or group. Kipon is known as a lens adapter company, although Shanghai Transvision has also manufactured and distributed video and photo accessories. They are rumored to manufacture lenses for other brand names, so they have some lens manufacturing experience. But, I have to say, my expectations for build quality weren’t great. I expected a lot of variation between copies. I don’t know if I even had any expectations regarding image quality.
Okay, so much for what I expected. There are now five new copies sitting on my desk so let’s take a look. Continue reading →
Way back in the old days, before everything was plug and play, you could buy a computer and accessories from one manufacturer and be certain it would all work together. The cool kids, though, would mix-and-match different pieces and end up with a better system that could do more things than any one manufacturer’s system would, and for less money. They realized they might have to spend some time getting this to work with that, and once in a while things wouldn’t work at all. That’s the price you pay for being a cool kid and getting more for less money.
The people who had problems bought the same thing the cool kids did, but just got irritated when things didn’t all work together right out of the box. The cool kids laughed, felt even cooler, and made some money debugging the other kids’ systems for them.
We do the same thing today with cameras. It’s simple to use a rig, lenses, monitors, whatever of the same brand. It will probably cost a little more and you may have a few less capabilities, but it’s just about guaranteed everything will work fine with everything else. But if you want to save some money and expand your capabilities, you mix-and-match systems, often using an adapter or two. The problems come when people don’t realize that not everything is going to play nice with everything else. The problem is compounded because the people who make the widgets just don’t have the capabilities of testing everything with everything else.
A few weeks ago Metabones sent me one of their new EF lens to Black Magic Pocket Camera Speedboosters to play with. I did the usual optical testing and let some of the video guys shoot with it, like all the other bloggers do. I planned on writing a piece saying how awesome the optics were (they are) and how many cool things it lets you do with a Pocket Camera (it does) like all the other bloggers do. But in our testing we found a few things that didn’t work well together, and it occurred to me that rather than adding yet another “The EF to Pocket Camera Speedbooster is Really Great” blog post, I could do something useful and actually list what it works well with and what it doesn’t.
This is a Geek Article. Many of you don’t understand the term ‘Geek’ properly, so perhaps this will help. As the graph shows, if you aren’t both intelligent and obsessed with photo gear, you won’t enjoy this article.
I’ve tried hard to find whom to credit for this, but haven’t been able to. If you know, please let me know so I can credit this brilliant work.
Left to right: Canon 16-35 f/2.8 II, 16-35 f/4 IS, 17-40 f/4. Can you spot the one with the wrong hood? The intern obviously couldn’t.
As is so often the case, I bit off more than I wanted to chew when I came back from vacation. The Canon 16-35mm f/4 IS lens had just been released, a few copies were in stock, and I thought I’d do a nice quick test. But one of the reasons I’d wanted an optical bench was because I don’t trust Imatest results with wide-angle lenses. At 16mm, even with the very largest test charts, we’re testing at about 4 feet shooting distance.
So I after I did our standard Imatest on the 16-35 f/4 IS, I wanted to repeat the results on our optical bench. Of course, I don’t have a big database of optical bench results to compare against like I do with Imatest. So I had to do optical bench tests on some other wide zooms for comparison purposes. Then I had to do some more comparison with other lenses to see if the variations we were seeing on the optical bench were simply a new, higher resolution testing method, or if they were telling me something about variation with wide-angle zoom lenses. (Both things were true.) Anyway, the testing I thought would take a week has taken three.
I realize some of you just want to see the usual Imatest results on a group of these lenses since that’s what you’re used to seeing. Others are also interested in the optical bench results showing how the lenses resolve at infinity, rather than just close up. And of course there are a few of you who want all the gory details of Geekiness that the optical bench reveals. So I’ll try to present this in three parts: the Imatest results first, the optical bench optical test results second, and the geeky stuff third. It’s a buffet; just grab what appeals to you.
Like a lot of photo history buffs, I’ve been quite excited about Lomography’s new iteration of the Petzval lens in 85mm focal length. For those of you who don’t know about the Petzval lens, I wrote about it a few years ago. It really has a rather a fascinating story.
Since writing that article, I’ve been rather obsessed by this lens. I own several of them, made in the late 1800s, but I haven’t been able to adapt them to work on a modern camera. Now Lomography has reproduced the Petzval lens in a nice brass housing, for either Canon or Nikon mounts. Our first copies arrived yesterday and I grabbed them for a bit before they headed out the door.
Well, I have to admit this has been a fun series. I’ve learned a whole lot. That’s what makes this so fun — I get some results I don’t understand, get some help figuring out what is going on, and before I know it, I’ve learned something that explains other things I haven’t been able to understand.
In the second part of this series, we started a database of sensor stack thickness and exit pupil distances, hoping that it would help people decide which lenses would adapt best to which cameras. (And, of course, determine which lenses would not adapt well to which cameras.) A number of people have added information to the database since it was first posted — enough to make it pretty useful.
Since the database is now large enough to be useful, I thought it would be a good idea to make a summary of what we know about lenses and sensor stacks. The best thing about all this, for me at least, is that it lets us make some generalizations about which lenses would be expected to have problems on which cameras.
A couple of weeks ago I got an email asking if we would be willing to take some lenses, remove the electronics, fix the aperture wide-open, and permanently lock them at infinity focus. It seems the person who needed this done was having trouble finding a legitimate repair shop or service center that was willing to do it.
Well, illegitimate is our specialty, so I started negotiations about just how exorbitant a fee we would charge for this work. We quickly arrived at a fair price (no money, but we get to take pictures) and yesterday received brand new copies of the Canon 100mm f/2 and Sigma 35mm f/1.4 Art to work on. If you’re the kind of person who slows down to view car wrecks or spent $200 on fireworks for the 4th of July holiday, you might like this.
(For those of you who aren’t American, the 4th of July is when we celebrate our Independence by getting sunburned, making burnt offerings of animal parts in our backyards, and then eating said offerings. During the entire day, we drink massive quantities of American beer and once it gets dark we shoot off massive quantities of Chinese fireworks. All too often, the results of mixing alcohol and explosives prove that Darwin was correct — but hey, that’s what celebrating is all about, right?)
If torn apart camera lenses make you squeamish, then you won’t like this, and I suggest you not read further. You won’t miss learning anything; it’s just for fun. As best I can determine, this post has absolutely no practical use whatsoever. It’s just something to amuse and entertain those of you who are amused and entertained by such things.