The vast majority of my blog posts are about things other than Lensrentals, but a couple of times a year I put out an update about the things we’re doing in-house. Those who like a ‘behind the scenes’ look usually enjoy finding out, well, what we’re doing behind the scenes. This year, since most of our Winter efforts focused on testing and quality assurance, we’ve also learned some interesting things.
Lensrentals has roughly doubled in size every year for the last 5 years. That sort of growth means it’s not “Roger in command 24/7” anymore. We’ve added people who do various things better than I do, and who have time to do those things. That has allowed me to go do the things that I really enjoy doing the most (playing with gear) and that I think are the most important (quality assurance).
Quality assurance is something that we’ve always prided ourselves on and I’ve always felt comfortable saying “no one takes better care of their equipment than Lensrentals”. But, being human, we didn’t do it perfectly, and if we aren’t perfect then there’s room for improvement. I’m comfortable that in 2012 we will deliver a level of equipment excellence that we’ve never attained before (and that no one else is close to).
We’ve always inspected everything, tested everything, and been very open about exactly what tests and procedures we use on every piece of equipment. I’m not talking about “we took some shots of the building next door” or “the person across the room” testing. Every item is tested by a tech when it comes in and again before it goes out. You can hardly walk anywhere in our offices without seeing specific test charts, focus charts, and and right-angle grids set up for just that purpose.
But despite thousands of dollars invested in charts and testing equipment, and 6 people who’s only job here is to test equipment, the inevitable happens: we occasionally miss something. Sometimes it’s simply human error: someone made a mistake. Sometimes everything is done properly, but testing with human vision (show the same images to two people and ask them to pick which lines are clear and which aren’t and you get two slightly different answers) has limitations. No matter how carefully we tested, we still occasionally miss things in a 15 minute inspection that a photographer might see in days of use.
In an effort to standardize our testing and make it more accurate we invested in an Imatest computerized testing system in in September. Since then we’ve been testing lenses. Every. Single. Lens. Do you know how many lenses that is? I would if I could count that high.
Imatest let us eliminate the human, judgement-call part of testing that occurs whenever a person evaluates images from a test chart. Now we get actual MTF numbers for each lens, not just in the center but across the entire surface. Suddenly, instead of just arguing about whether a lens was softer in the right lower corner, we could see it.
As we began testing all of our lenses, including the ones that had passed our visual inspections, we saw that a few really weren’t as OK as we thought they were (like the one in the example above). And as we tested more more lenses, the difference between the good lenses and the outliers became readily apparent (even though they’d all been tested by our technicians and found acceptable).
Over the winter we’ve tested almost all of our lenses, sent the ones that weren’t OK off to repair and have developed the range of acceptable results for each Canon, Nikon, and Zeiss lens (we don’t have enough copies of other brands to feel comfortable that we’ve clearly defined the acceptable range).
Is all of this critical? Not everyone thinks so. Our accountants certainly don’t. They, and many of our competitors, believe if a lens is bad the renters will let you know without going to all this time and expense. But I think it is worthwhile. Our renters are often trying a lens to determine if the lens they’re renting is something they might want to buy. They can’t make an informed decision unless they absolutely know the lens they’re shooting with is a good copy. Which group of Canon 35mm L lenses would you rather rent from, the upper right (our lens fleet as tested September, 2010) or the lower left graph (the current fleet)? Me too.
The Bad Thing about more Accurate Tests
Well, if you ask the business people they’ll tell you it’s the six plus figures Roger spent on all this testing stuff. For me, though, the bad thing was testing lenses when they returned from repair. The vast majority came back just perfect. But a few came back just as bad as when we sent them. So we sent them back. And they came back again without any improvement.
Pretty soon a pattern started to emerge: lenses that were broken, that were soft in general, or that had focusing issues got fixed quickly and accurately. But when a lens was reasonably sharp in the middle, but soft on one side, or soft in all corners, it was likely to come back just like we’d sent it. Those lenses, the ones that had decentered or tilted elements, were likely to not get fixed at the service center. So I started bothering and alienating people until I started getting some answers about why that was. I got some answers. Mostly off-the-record answers, but answers.
I was aware, of course, that modern factory repair centers use computerized programs and test targets to make optical adjustments on lenses. I wasn’t aware that for most lenses, from most manufacturers, that procedure maximizes center sharpness but can’t do a damn thing about a lens that has a tilted or decentered element. Some repair centers were pretty honest: either the adjustments were impossible to do or they were soooo time consuming that it wasn’t economically feasible to make them. It depends largely upon the lens in question and its design.
Other repair centers simply made ridiculous statements, like the factory service center that told me the lens with the two corners below was “perfectly within specifications”. Of course, when I asked what their corner resolution standards were I got disconnected somehow. When I called back I was told they didn’t release that information to the public. I can see why: if your standards say the image below is acceptable, I wouldn’t release those standards to the public either. I don’t know about you, but to me one of these things is not like the other.
So what do we do? The lens isn’t acceptable to rent like that, but we can’t find a repair center to fix it. All of you who said “Set up an anonymous eBay account and sell it” shame on you. Plus you failed the gear-head test of the day. Aaron, Tim and I actually got pretty excited: if it can’t be fixed, and it’s not OK to rent, then it should be taken apart.
We knew a bit of optical theory, have a lot of hand’s on experience, and now we had lots of lenses to experiment on — it’s not like we were going to try to tell our customers what the factory told us – we knew the lens wasn’t OK and we weren’t about to tell anyone it was. So over the Winter we’ve invested in a Century Lens Test Projector , various lens mounts, some tools, and many hours of trial and error. Surprisingly, we learned that (in some cases at least) we can do what the repair center couldn’t do. It’s not that we’re so smart. Probably more like we’re really stubborn.
Just in case you have the idea that “they could do this if they wanted to at the repair center”, as best I can tell that’s not true. It takes a long time to optically adjust a lens that’s decentered or has a tilted element, if it’s possible at all. (When you look at the images below, do the math and figure out how many in-between images there were – and remember each image was taken after a session of adjustment on the lens projector.). If you asked us to do it for you, we’d tell you it’s not worth it, we’d have to charge a fortune. It’s only worth it to us because the alternative is writing off a $2,000 lens.
But for lenses we have in stock, at least, we’re able to make changes like the before and after below. The graphs are a bit confusing at first, but simply look at the numbers in the yellow boxes, those are the MTF50 of the lens at that location. It’s obvious the right side has been markedly improved after we’ve finished adjusting it.
And in case you’re wondering, you can tell the difference in real world, too. Here are some corner crops at 100% after the adjustment was done (This is the lens from the Imatest charts above, not the same lens as the previous corner crops way above. But they both looked similar.). It was “within specifications” according to the factory before we adjusted it (image 3394 above). I guess we’ll say that it wasn’t until image 3411 above that I’d say it was “within Lensrentals specifications”.
Of course, there are a few among you who remembered that I said we can only optically adjust some lenses and are thinking ‘so you sell the rest on eBay, right’? Yep, right. Except we sell them as parts. Broken lenses often sell for nearly as much as working lenses (something to consider if you have some broken lenses or cameras around). The parts in them are worth more than the lens itself in most cases. If Nikon goes through with their new idea to not sell parts I may start a new business buying new Nikon lenses, taking them apart, and selling the parts. There’s good money to be made doing that. If I don’t do it, I bet someone else will.
Oh, One Last Thing
We’re really proud of the improvement we’ve made in our lens fleet over the Winter. We’ve repaired or eliminated over 150 lenses that weren’t up to our new standards. I’m absolutely comfortable your odds of getting an excellent lens are better with our rental lenses than if you bought one new. (I’m comfortable saying that because I test lenses new out of the box before we put them in stock. The failure rate of those lenses optically, is about 1.5%. Nobody at the camera store opens the box and tests the lens.)
But we also know that shipping plays havoc with lenses and every year we look at ways to improve our preventative packing. We’ve tried hard cases (transmits too much shock and adds to shipping costs) and numerous different types of packing material. Last fall we started using heavy, padded cases to ship most lenses in, and used a heavy foam packing around those. It worked very well and we had far fewer lenses damaged in transit.
But the foam, while recyclable to some degree, only lasted for a few shipments. Unwrapping layers of foam, and then remembering how to repack it when you sent the lenses back got confusing sometimes. So over the Winter, Tyler negotiated with some of our supplies to custom make foam inserts for our commonly used box sizes.
Now it’s easy as can be: put the lens in the case, put the case in the padded box,
put the top on, seal the box. It’s faster, simpler, provides more consistent protection, and the foam inserts can be reused dozens of times, so it’s friendly on the environment too.
This was probably more than you wanted to know, but that should bring you up to date on what we’ve done over the Winter. We want to start every year knowing we’re doing an even better job than we did the year before. I’m comfortable that’s where we are as the 2012 rental season kicks off.