There's been a lot of interest in the newly released Tamron 150-600mm f/5-6.3 SP Di VC USD lens. (For those wondering what the initials mean, VC means vibration compensation, the others all read 'marketing drivel'.) But meaningless initials or not, with a price under $1,100, a superior range, and vibration compensation, this lens has created a lot of excitement.
Recently, the good people at Imatest have developed an ultra-high resolution, backlit chart printed on photographic film that is perfect for testing long telephoto lenses in the lab. The combination of a new, cool Imatest setup and a new, cool lens proved irresistible, so we decided to compare the new Tamron with the older Tamron 200-500mm, Sigma 50-500mm OS, and Canon 100-400mm IS lenses.
One important note: this is a higher resolution type of chart than the printed charts generally used for Imatest work (if you're into geeky stuff, there's an addendum about this at the end of the article). We haven't done enough comparisons to say exactly how much it increases Imatest MTF50 numbers (it's not a linear kind of thing) but it does to some degree. So don't take today's numbers, compare them to one of our previous tests, and say something dumb like "Roger showed the Tamron 150-600 is sharper than the Canon 70-200 f/2.8". Different tests mean different data ranges.
One other note: I am totally aware that 713 of you have suggestions for further testing. Heck, I have suggestions for further testing. But we won't be able to do further testing on this group anytime soon. This kind of stuff is incredibly time consuming and right now our repair department has larger priorities.
Meet the Contestants
Tale of the Tape:
Tamron 200-500 Tamron 150-600 Canon 100-400 Sigma 50-500
Price $950 $1070 $1700 $1510
Weight (lb.) 2.72 4.3 3.04 4.33
Length (inches) 8.9 10.15 7.4 8.6
Filter Size mm 86 95 77 95
min. focus dist. (ft) 8.2 8.9 5.9 1.64
Telephoto zooms are not small lenses, so size and weight are important considerations. Two lenses, the Canon 100-400 IS and Tamron 200-500, weigh about 3 pounds while the Sigma and Tamron 150-600 are well over 4 pounds. That's critical for some people.
For those who might want to shoot with a filter on, the Sigma and the Tamron 150-600 require expensive 95mm filters, while the Canon uses more reasonably priced 77mm filters. Probably the most important difference to most people, however, is the price, and the Tamrons are clearly less expensive.
We only received three copies of the Tamron 150-600mm today. Quick testing showed they were all well centered with little sample variation. We tested two of those lenses against two copies of each of the other lenses (which had previously been tested and found to be well-centered copies well within the 'normal' range for those lenses). We followed our usual Imatest protocols with the exception that the new backlit film chart was used in this test.
One word about focal length scales: when we set up Imatest we set the lens to a focal length, then position the lens-tripod combination to a distance that fills the camera frame with the test chart. At 200mm and 400mm three of the lenses agreed they were at 200 and 400mm, but the Sigma 50-500 read as 210mm and 420mm at those same shooting distances.
I had planned to just test the lenses wide open, but that, too, presents a problem. At 200mm, for example, the Canon and both Tamrons are at f/5 wide open, while the Sigma is f/5.6. At 400mm the Sigma is at f/6.3 wide open, while the others are all f/5.6. To level the playing field, we repeated the tests with the other lenses stopped down to the same aperture as the Sigma. The half stop does make a difference in MTF50 readings.
MTF 50 at 200mm
Center MTF 50 Avg MTF 50
Tamron 150-600 (f/5) 895 745
Tamron 200-500 (f/5) 715 510
Canon 100-400 (f/5) 980 780
Sigma 50-500 (f/5.6) 980 780
Tamron 150-600 (f/5.6) 980 810
Canon 100-400 (f/5.6) 1010 850
The takeaway message is the Tamron 200-500mm lens clearly doesn't resolve as well as the other three lenses at 200mm. That's not shocking, it's 1) a much older design and 2) working at its widest focal length while none of the others are at the extreme end of their zoom range here.
The other three lenses are in the same ballpark. The new Tamron 150-600 may not be quite as sharp at pixel-peeping resolutions when all are wide open, but put them all at f/5.6 (where the Sigma is wide open) the three lenses are nearly identical. The Canon's numbers are a bit higher, but the difference is probably not significant enough to notice in a photograph.
MTF 50 at 400mm
Center MTF 50 Avg MTF 50
Tamron 150-600 (f/5.6) 945 840
Tamron 200-500 (f/5.6) 795 665
Canon 100-400 (f/5.6) 945 835
Sigma 50-500 (f/6.3) 780 640
Tamron 150-600 (f/6.3) 975 840
Canon 100-400 (f/6.3) 1000 870
At this focal length the Sigma 50-500 OS appears to be fading compared to the other lenses, which surprised me at first. We repeated the test with several more copies of the Sigma, though, and the results were consistent.
I had not seen this drop off at 400mm in previous Sigma tests, although it was clearly present at 500mm. I think the reason for the difference is when previously testing the Sigma, we tested at 400mm according to the lens. In this test, the Sigma was set at 420mm so that it matched the field of view that the other lenses have at 400mm. I assume the fall off occurs right after 400mm on the Sigma lens.
(Each Imatest run at a given focal length requires a complete teardown and new setup, so it's a very time consuming matter to check at 400mm, 420mm, 450mm, etc.)
The bottom line, though, is at 400mm the Tamron 150-600 VC and Canon 100-400 IS are virtually identical, with the Tamron 200-500 and Sigma 50-500 a bit behind. Let's keep it in perspective, though, both the Sigma and Tamron 200-500 are still very good at 400mm.
As an aside, because I expect it will come up in discussions, there are a number of people on the internet who say their Canon 100-400 isn't as sharp at 400mm as it is at shorter focal lengths. My experience is this usually means a slightly decentered front element. Good copies are equally sharp throughout the zoom range.
MTF 50 at 600mm
There's only one contestant here, obviously -- we just wanted to compare the Tamron's results at the extreme end to the rest of the range. And let's be realistic: at 600mm and f/8 technique is going to have a lot more to do with how good the images look than the MTF 50 data generated in a lab. Yeah, I know you have a great image taken hand-held at 600mm and f/8 with 1/150 shutter speeds. One time I dropped my camera, the shutter went off when it hit, and it made a great macro of the bottom of a dandelion.
There's no question that resolution drops off at the extreme telephoto end of the zoom range, as shown below. A number of photographers have noted the same thing, but have shown 600mm resolution is much better at f/8.
Realistically, though, the fact that a zoom reaches a true 600mm focal length at all is rather amazing. Sure, the resolution drops off a bit there, but no other zoom gets there at all.
I don't do resolution testing at telephoto range very often. The new Imatest equipment certainly makes it possible, but in general I don't find it particularly useful. Working at these focal lengths the lens' MTF just isn't as important as real-world variables (atmosphere, tripod support, photographic technique, etc.). Additionally, testing the lenses at lab distances (13 to 30 feet depending upon focal length) may give slightly different results than would be seen if we could test the lenses at longer distances.
On the other hand, those real world variables tend to make looking at posted images confusing. Each of the lenses we tested today have many soft and many sharp images posted online. Technique, lighting, atmospheric conditions and a host of other variables create a lot of variation. So I think lab testing does give some worthwhile information and confirmation.
My summary would be that the selection between a Tamron 150-600, Canon 100-400 IS, and Sigma 50-500 OS should be made on criteria other than MTF 50. There are some minor differences in resolution, but nothing that makes one clearly better than another. Price, weight, autofocus accuracy, effectiveness of vibration compensation, and a number of other factors (did I mention price?) are more important considerations when choosing among these lenses.
It's pretty obvious that the Tamron has both 600mm range and the lowest price. These tests, and everything I see from photographers using the lens in the field, support that it's of at least equal image quality. Some people will prefer the extra wide range of the Sigma, others the lighter weight of the Canon. But for a lot of people, the Tamron is going to be the best bang for the buck.
One note for Nikon shooters: I'm sorry I won't have time to run the same tests on Nikon cameras when the Nikon mount is released. However, the new Nikon 80-400 AF-S zoom is, as near as we can tell, equivalent to the Canon 100-400 IS as far as resolution goes, so you should be able to extrapolate pretty easily.
Roger Cicala and Aaron Closz
Addendum: About Imatest Charts
I've been using Imatest for several years. Lots of reviewers and testers do, too. I've written before about why Imatest numbers are different among different testers. In the last year, or so, though, I've become more aware that there's another difference I hadn't appreciated before -- the quality of the charts used when obtaining the images for Imatest analysis.
Back when I started using Imatest a 12-megapixel camera was the norm, and 16 megapixels nearly the maximum. Now we're testing 36 megapixel cameras on lenses that are immensely better than anything we had a decade ago. But a lot of people are still testing on charts printed on 300 DPI inkjet printers while others test using higher resolution, professionally printed charts. More recently, Imatest has developed even higher resolution transmissive film charts, although they aren't being used by many testers.
The chart type doesn't make much difference when testing average quality lenses on 16 megapixel cameras. It probably does make a difference testing highest quality lenses on 36 megapixel cameras.
I should mention, too, that Imatest is sensitive to the lighting used - it's important to keep the lighting similar with every test run if you want equivalent results. Transparency charts obviously have different lighting than reflective charts.
For all of these reasons, results testing a lens using a reflective chart and a transmissive film are going to be slightly different. The difference should be greater with higher resolution cameras and the best lenses, but there will be some difference with even more routine equipment.
We have to test hundreds of lenses on both types of charts before we will feel we can convert between the two charts with some degree of accuracy. Until we do that, we can't directly compare numbers from a test run on a film transparency chart with numbers from a test run with a reflective chart.