I'm going to open a can of worms today. I've been getting more and more emails from people telling me the same story that goes like this:
I've got this lens. It's awful. I've sent it in for adjustment and the service center tells me it's 'in spec' and nothing is wrong with it. Am I crazy?
Second only to the dreaded 'impact damage - warranty void' statement, the 'lens is in spec' statement seems to be some factory service center's answer to far too many complaints. Don't get me wrong, sometimes there is impact damage and the warranty should be void. But I can't think of any reason why this seems to happen only to certain brands and never to others. Similarly, lenses a customer thinks are bad can be 'in spec'. The problem is, since the factory service center doesn't have to tell us what 'in spec' means, it's open to a lot of abuse.
Now before I go further, let me say that I feel the service center's pain, too. Most of those emails I get are accompanied by some useless JPEGs. Maybe 1 person in 10 who emails me actually has taken images that would indicate if there was a problem or not. So I'm certain the techs at the service center become a bit cynical over time.
So if you think a hand-held autofocus image, an ISO 6400 image, or a picture of some isolated object right in the center of the image is helpful in deciding if a lens is OK, then you aren't allowed to read this post. Instead read about how to test a lens, how to check for decentering, or what a lens with a tilted element looks like. Then you can come back and read this.
First of all, I'm not going to mention what lens or company this example is taken from. The reason is simple - I've got plenty of similar examples (over 100 actually) with other lenses and other factory service centers.
What we have here is a high-quality, fairly expensive zoom lens. It was sent to factory service because it had much lower resolution on the left side than the right. It came back looking the same. The second time it came back better at the wide end, but worse at the long end along with a note saying it had been tested and was in spec. The third time it came back no better with a note that it had been tested and was in spec. We didn't see much sense in trying a 4th time.
I won't bore you with optical bench, lens test projector, or Imatest results for the lens (we did them, but I want to keep this simple and they aren't necessary). I'll just show you images taken of an ISO12233 test chart. I like this chart because it lets us look at resolution in the center and all 4 corners in one shot, if done correctly.
Here's a picture of the chart, in case you aren't familiar with it:
The lens that came back from factory service for the 3rd time was OK in the center, so I'll just show you close-ups of the 4 cross-shaped resolution charts in the corners. Here are the corners at the wide & zoom ends of the zoom range.
To make sure you have an idea of how bad this 'in spec' lens is, in the image below I've put the top two corners at the long end from the 'factory in spec' lens with the top two corners of another copy of the lens taken off our shelf. Again, because someone who doesn't read the whole article is bound to say it - these shots are all focus bracketed and the best image used. Autofocus has nothing to do with it - this is purely optics.
Like we all learned from Sesame Street, one of these things is not like the other. According to the factory service center, they are both fine. But if I sent that 'in spec' lens out to one of my customers, I would have one less customer.
I talked Aaron into opening this lens up and adjusting it in-house. Adjusting it involved removing the mount and making some 17 trial-and-error adjustments to the rear group, testing after each adjustment. Then we removed the front barrel and made a half dozen adjustments at the front group, followed by taking the mount off again and making some more tweaking adjustments to the rear element. It was a several-hours long process. But at the end of the process, the lens was now within my 'spec' as you can see below.
It's not absolutely perfect. If you noticed, the left lower corner at the wide end still has some astigmatism with the vertical lines not resolving nearly as well as the horizontal lines. But even that corner is much better than when it came back from the factory in spec (and is better at infinity than in this fairly close-up shot). The other 7 corners are all excellent. Maybe we could have gotten that last corner better, but my experience is that most of these lenses have a slightly weak corner at one end of the zoom range.
It's the same lens, no new parts, just a very lengthy optical adjustment.
My point is not that we're so great. We're not. We're just a couple of self-taught guys trying to figure stuff out mostly by trial and error. My point is that if some half-trained guys with a few screwdrivers and some optical testing equipment could do this, then the trained technicians at the factory service center could have done it, too. They certainly could have done it faster. In theory they could have done it better.
But in three tries they didn't do it at all. They just told me, in writing, that the lens was 'in spec'. They won't share exactly what those 'specs' are, but in this case the picture says it all. Their specs are pretty damn low.
Let me repeat for you fanboys who are certain your brand doesn't do this - chances are that it does. I just chose this lens because it happened to come back yesterday. I have this kind of result several times a week (out of dozens of repairs a week, mind you).
So Why Doesn't the Service Center Do It?
I'm doing some educated guessing here, and not all reasons would apply all of the time. But here are some of them.
1. Some lenses can't be optically adjusted. The factory has the choice of giving you a new one, telling you the one you have is OK, or randomly replacing some of the lens elements and groups and hoping that makes it better.
2. Service centers don't have nearly as much optical testing equipment as you think they do. None of them talk openly about what equipment they have, but I've had a lot of conversations with techs that let slip that they don't have certain capabilities. They have lots and lots of computerized adjustment software. But most don't have optical benches, collimators, or even high-quality test targets. Some use lens-test projectors, which aren't particularly accurate, but are better than nothing. Some don't even have those. My favorite anecdote is a lens we fought with the service center over for 3 months. The last time it came back we were told their chief of optical testing had adjusted it himself. Unfortunately the chief left his memory card in the camera that we'd sent along with the lens: 62 images of a bookshelf were used to make the optical adjustments. Even when looking at a bookshelf, though, you could tell it still wasn't right.
3. Some service centers apparently can only adjust optics for the center of the lens. They apparently don't have equipment for off-axis adjustments or assessment. (That's may be the case with this brand; they always seem to get the center perfectly adjusted but often send lenses back with horrid corners.)
4. These adjustments can be really, really difficult and time consuming. The techs are on a clock, expected to do a repair in a given amount of time, so they must dread optical adjustments. Or maybe 'in spec' just means time was up.
5. When a customer says their lens is soft, the most common problem is that they can't tell backfocus from their back end. So the service center tends to 'electronically adjust' the lenses. That is fast, computerized, and works great when the problem really is backfocus or frontfocus -- and that's usually the problem. But eventually, if all you have is a hammer, everything starts looking like a nail.
6. So many people claim their lens is bad when it really isn't, that the techs get rather cynical. I've gone up the food chain often enough, and have heard the surprise in someone's voice when they say, "Oh, you really do know what you're talking about," to know this is true.
So What to Do?
I don't expect my writing a blog post is going to make any factory service center rush out and make changes. But it may help some of you who are fighting these battles to know it isn't just you. Just because you're paranoid doesn't mean they aren't out to get you.
I also think that maybe, just maybe, if people ask politely with their wallets, some manufacturer will decide it will be to their competitive advantage to offer better optical services. Sigma recently got a ton of positive press when they showed the equipment they had placed in the assembly factory to test every lens before it was boxed for shipment. People want that kind of assurance.
If you run into this problem, you might ask to see the optical test results. They aren't going to show them to you. Looking at the example above (and lots of similar examples like the ones above), I doubt they even have any. But maybe if enough people ask, in these days of lowered sales expectations, one or two companies might decide enough customers are interested in that service to provide it. Actually, we only need one company to do that. The others will have little choice but to follow along.
I think that a service center that actually sent you a printout of their optical tests, showing that your lens really did (or didn't) meet the expected test results, would win a lot of loyal customers. Heck, the oil change place always shows me the old filter they replaced just to make sure I know they really replaced it. The service center could at least show me that they'd actually tested the lens.
Actually, I suspect there are one or two companies that already do have optical testing equipment available in their service centers, and who use it to make accurate optical adjustments. If they'd like to show it off, I'd be happy to write about it for them.
A Note About Who Shot John:
I knew it would happen - that people would be angry that I don't name names. Let me be clear as to why.
First and foremost, I haven't gone back, gathered the data on 10,000 repairs, hand sorted which ones were repeat optical repairs (because that parts not a sortable field in our database), compared that to all optical repairs for that brand, number of copies and rental weeks for each lens, etc. That's a 60 hour job and I'm busy at the moment. But without that it would be totally unfair for me to put up a note saying 'this manufacturer has more problems', etc. Do I have some ideas in my head? Sure. But I'm the analytical type. I want numbers and facts before I spew off negativity.
Second, I'd really like to see changes made. Backing someone into a corner and thrashing them is certainly fun, but it rarely results in change. It usually just results in damage control and denial. If I did that today I already know what the response would be - 'we have far superior methods of testing our own lenses, but they are secret and proprietary and we can't show them to the public'. Given the way most camera manufacturers seem to work these days, they might change things quietly and secretly. A few years from now things might be better (or they might not). But they would never, ever let us know there had been changes, because there was never a problem.
I'd much rather that one company, having not been backed into a corner, decides it will be great marketing to show the purchasing public how they test, to provide test results when you send a lens in for optical adjustment, to get lavish praise in all the forums for that, and watch the serious photographers, who buy lots of lenses, flock to their brand. Is it going to happen? Nah, probably not. But I'm an optimist.
But for those that want to investigate what companies might be better at optical adjustments, here's a good place to start. Look up what companies make both lenses and optical testing equipment. You'd kind of figure they'd have their own machines in their own service center, wouldn't you?