Roger's Corner

What I Did Over Winter Vacation

Published March 14, 2012

The vast majority of my blog posts are about things other than Lensrentals, but a couple of times a year I put out an update about the things we’re doing in-house. Those who like a ‘behind the scenes’ look usually enjoy finding out, well, what we’re doing behind the scenes. This year, since most of our Winter efforts focused on testing and quality assurance, we’ve also learned some interesting things.

Lensrentals has roughly doubled in size every year for the last 5 years. That sort of growth means it’s not “Roger in command 24/7” anymore. We’ve added people who do various things better than I do, and who have time to do those things. That has allowed me to go do the things that I really enjoy doing the most (playing with gear) and that I think are the most important (quality assurance).

Quality assurance is something that we’ve always prided ourselves on and I’ve always felt comfortable saying “no one takes better care of their equipment than Lensrentals”. But, being human, we didn’t do it perfectly, and if we aren’t perfect then there’s room for improvement. I’m comfortable that in 2012 we will deliver a level of equipment excellence that we’ve never attained before (and that no one else is close to).

Improved Testing

We’ve always inspected everything, tested everything, and been very open about exactly what tests and procedures we use on every piece of equipment.  I’m not talking about “we took some shots of the building next door” or “the person across the room” testing. Every item is tested by a tech when it comes in and again before it goes out. You can hardly walk anywhere in our offices without seeing specific test charts, focus charts, and and right-angle grids set up for just that purpose.

But despite thousands of dollars invested in charts and testing equipment, and 6 people who’s only job here is to test equipment, the inevitable happens: we occasionally miss something. Sometimes it’s simply human error: someone made a mistake. Sometimes everything is done properly, but testing with human vision (show the same images to two people and ask them to pick which lines are clear and which aren’t and you get two slightly different answers) has limitations. No matter how carefully we tested, we still occasionally miss things in a 15 minute inspection that a photographer might see in days of use.

In an effort to standardize our testing and make it more accurate we invested in an Imatest computerized testing system in in September. Since then we’ve been testing lenses. Every. Single. Lens. Do you know how many lenses that is? I would if I could count that high.

Imatest let us eliminate the human, judgement-call part of testing that occurs whenever a person evaluates images from a test chart. Now we get actual MTF numbers for each lens, not just in the center but across the entire surface. Suddenly, instead of just arguing about whether a lens was softer in the right lower corner, we could see it.

Imatest printout for a lens with a slightly tilted element.  Most interesting is that the lower right corner still resolves well enough that it passes our most stringent optical testing. But a photographer making large prints would notice the right lower corner was softer.

As we began testing all of our lenses, including the ones that had passed our visual  inspections, we saw that a few really weren’t as OK as we thought they were (like the one in the example above). And as we tested more more lenses, the difference between the good lenses and the outliers became readily apparent (even though they’d all been tested by our technicians and found acceptable).

Results for the first (left) and first two (right) test sessions of Canon 35mm f/1.4 lenses. Three lenses that had passed optical inspection weren't as OK as we thought they were.

Over the winter we’ve tested almost all of our lenses, sent the ones that weren’t OK off to repair and have developed the range of acceptable results for each Canon, Nikon, and Zeiss lens (we don’t have enough copies of other brands to feel comfortable that we’ve clearly defined the acceptable range).

Imatest results for all of our current Canon 35mm f/1.4 lenses (left) and our ‘acceptable results’ standards (shaded area on right) for that lens for all tests going forward.

Is all of this critical? Not everyone thinks so. Our accountants certainly don’t. They, and many of our competitors, believe if a lens is bad the renters will let you know without going to all this time and expense. But I think it is worthwhile. Our renters are often trying a lens to determine if the lens they’re renting is something they might want to buy. They can’t make an informed decision unless they absolutely know the lens they’re shooting with is a good copy. Which group of Canon 35mm L lenses would you rather rent from, the upper right (our lens fleet as tested September, 2010) or the lower left graph (the current fleet)? Me too.

The Bad Thing about more Accurate Tests

Well, if you ask the business people they’ll tell you it’s the six plus figures Roger spent on all this testing stuff. For me, though, the bad thing was testing lenses when they returned from repair. The vast majority came back just perfect. But a few came back just as bad as when we sent them. So we sent them back. And they came back again without any improvement.

Pretty soon a pattern started to emerge: lenses that were broken, that were soft in general, or that had focusing issues got fixed quickly and accurately. But when a lens was reasonably sharp in the middle, but soft on one side, or soft in all corners, it was likely to come back just like we’d sent it. Those lenses, the ones that had decentered or tilted elements, were likely to not get fixed at the service center. So I started bothering and alienating people until I started getting some answers about why that was. I got some answers. Mostly off-the-record answers, but answers.

I was aware, of course, that modern factory repair centers use computerized programs and test targets to make optical adjustments on lenses. I wasn’t aware that for most lenses, from most manufacturers, that procedure maximizes center sharpness but can’t do a damn thing about a lens that has a tilted or decentered element. Some repair centers were pretty honest: either the adjustments were impossible to do or they were soooo time consuming that it wasn’t economically feasible to make them. It depends largely upon the lens in question and its design.

Other repair centers simply made ridiculous statements, like the factory service center that told me the lens with the two corners below was “perfectly within specifications”. Of course, when I asked what their corner resolution standards were I got disconnected somehow. When I called back I was told they didn’t release that information to the public. I can see why: if your standards say the image below is acceptable, I wouldn’t release those standards to the public either. I don’t know about you, but to me one of these things is not like the other.

Opposite corners from a lens that was "within specifications" after repair.

So what do we do?  The lens isn’t acceptable to rent like that, but we can’t find a repair center to fix it. All of you who said “Set up an anonymous eBay account and sell it” shame on you. Plus you failed the gear-head test of the day. Aaron, Tim and I actually got pretty excited: if it can’t be fixed, and it’s not OK to rent, then it should be taken apart.

We knew a bit of optical theory, have a lot of hand’s on experience, and now we had lots of lenses to experiment on — it’s not like we were going to try to tell our customers what the factory told us – we knew the lens wasn’t OK and we weren’t about to tell anyone it was. So over the Winter we’ve invested in a Century Lens Test Projector , various lens mounts, some tools, and many hours of trial and error. Surprisingly, we learned that (in some cases at least) we can do what the repair center couldn’t do. It’s not that we’re so smart. Probably more like we’re really stubborn.

Adjusting resolution on a Lens Projector

Just in case you have the idea that “they could do this if they wanted to at the repair center”, as best I can tell that’s not true. It takes a long time to optically adjust a lens that’s decentered or has a tilted element, if it’s possible at all. (When you look at the images below, do the math and figure out how many in-between images there were – and remember each image was taken after a session of adjustment on the lens projector.). If you asked us to do it for you, we’d tell you it’s not worth it, we’d have to charge a fortune. It’s only worth it to us because the alternative is writing off  a $2,000 lens.

But for lenses we have in stock, at least, we’re able to make changes like the before and after below. The graphs are a bit confusing at first, but simply look at the numbers in the yellow boxes, those are the MTF50 of the lens at that location. It’s obvious the right side has been markedly improved after we’ve finished adjusting it.

MTF 50 numbers (yellow boxes) before (above) and after (below) adjustment.

And in case you’re wondering, you can tell the difference in real world, too. Here are some corner crops at 100% after the adjustment was done (This is the lens from the Imatest charts above, not the same lens as the previous corner crops way above. But they both looked similar.). It was “within specifications” according to the factory before we adjusted it (image 3394 above). I guess we’ll say that it wasn’t until image 3411 above that I’d say it was “within Lensrentals specifications”.

Opposite corners after centering adjustments.

 

Of course, there are a few among you who remembered that I said we can only optically adjust some lenses and are thinking ‘so you sell the rest on eBay, right’? Yep, right. Except we sell them as parts. Broken lenses often sell for nearly as much as working lenses (something to consider if you have some broken lenses or cameras around). The parts in them are worth more than the lens itself in most cases. If Nikon goes through with their new idea to not sell parts I may start a new business buying new Nikon lenses, taking them apart, and selling the parts. There’s good money to be made doing that. If I don’t do it, I bet someone else will.

Oh, One Last Thing

We’re really proud of the improvement we’ve made in our lens fleet over the Winter. We’ve repaired or eliminated over 150 lenses that weren’t up to our new standards. I’m absolutely comfortable your odds of getting an excellent lens are better with our rental lenses than if you bought one new. (I’m comfortable saying that because I test lenses new out of the box before we put them in stock. The failure rate of those lenses optically, is about 1.5%. Nobody at the camera store opens the box and tests the lens.)

But we also know that shipping plays havoc with lenses and every year we look at ways to improve our preventative packing. We’ve tried hard cases (transmits too much shock and adds to shipping costs) and numerous different types of packing material. Last fall we started using heavy, padded cases to ship most lenses in, and used a heavy foam packing around those. It worked very well and we had far fewer lenses damaged in transit.

But the foam, while recyclable to some degree, only lasted for a few shipments. Unwrapping layers of foam, and then remembering how to repack it when you sent the lenses back got confusing sometimes. So over the Winter, Tyler negotiated with some of our supplies to custom make foam inserts for our commonly used box sizes.

Now it’s easy as can be: put the lens in the case, put the case in the padded box,

put the top on, seal the box. It’s faster, simpler, provides more consistent protection, and the foam inserts can be reused dozens of times, so it’s friendly on the environment too.

 

This was probably more than you wanted to know, but that should bring you up to date on what we’ve done over the Winter. We want to start every year knowing we’re doing an even better job than we did the year before. I’m comfortable that’s where we are as the 2012 rental season kicks off.

 

Roger Cicala

Lensrentals.com

March, 2012

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Roger's Corner
  • Ken

    Looks like your packaging solution is pretty good but have you ever looked at something like these?

    http://www.betashell.com/

  • Wow! Roger, you guys are amazing. I bought my Canon 10-22 from you guys this past thanksgiving and have loved it, and never doubted the purchase at all. It was an amazing deal.

    I feel even better about my purchase now than I did before, even if my lens probably did’t go through this extra testing.

    P.S. I’ve reused the box my 10-22 came in too! Great packing.

  • TM

    A German branch would be awesome^10!

  • Bryan Conner

    If only you had a facility in Germany………just dreaming.

  • Roger Cicala

    Ralf and Hakon,

    We’d love to, but we’d need European partners: we don’t know shipping, taxes, collections and all the other things that would go with running a business in Europe. All we know about is gear 🙂

  • Hi Roger,

    I second Hakon. Please do open a branch in Europe. Most rental-companies around here stock old stuff, so if you wanted a 2,8/300 it likely would be one without VR. If you want to go beyond 300, your are lost at most places. Rental conditions demand you to pick stuff up in person and/or you pay for the shipping days as well.

    Thanks for the informative and entertainings lines,

    greets
    Ralf C.

  • Randy

    Roger,

    It’s a shame you left medicine, maybe you could have had an influence on the quality of medical care. Oh well, I guess you found another “calling.” lol.
    Keep up the good work!

  • Roger Cicala

    I wouldn’t mind showing it as far as business purposes, but unless you have a lens test projector, Imatest, or a collimator there’s no way to do it at home. Plus you can really, really screw things up amazingly.

  • Roger Cicala

    Infinity focus is a combination of lens-to-flange distance and within lens distances. That’s why it’s really common with adapters: if they’re off just a bit, some lenses won’t focus to infinity. But it can be a problem without adapters. If you send me your order number (email me) I’ll check how that first lens did when it came back. My guess would be it didn’t show up on any of our test cameras since we routinely check for that.

  • CarVac

    I’m not sure if I should bring this up here, but I rented Canon mount Tokina 11-16 for a trip last week and it could not focus all of the way to infinity in the wideangle range. Of course, I was sent a replacement immediately which, while it exhibits the same focus shifting tendencies when zooming, can focus (if just barely) to infinity. Do you have any way (or plan) to test that sort of thing before sending out lenses?

    I was surprised that this problem existed at all, since it was a mechanical and not electrical issue. Its sharpness was perfectly acceptable, but the problem probably wouldn’t show up in the resolution or focusing accuracy tests (it autofocused alright up close).

    Also, just wondering: is that a typical problem with 11-16’s (in Canon mount)? That the problem existed to a nonzero extent in both lenses worries me, especially since I was considering purchasing a copy of the lens.

  • George

    It would be amazing if, when you figure out how to fix decentering in certain lenses, you published how you went about doing it–it’d obviously make more business sense to keep that info to yourself, though. It’s a massive problem, especially on IS lenses. 🙂 GREAT article.

  • Roger Cicala

    Allan,

    It’s really hard to generalize, all lenses are different and different methods are used. Most have at least two elements / groups that are adjustable and many have more than two. Sometimes one group can be adjusted many ways. For example the front element on some lenses has a sliding ramp that lets it move forward and backwards (in relation to the rear of the lens). It may also be able to move side to side in order to help center the lens. It may also have eccentric adjustable screws that allow you to adjust tilt. And that’s just on one element.

    For another lens, centering may be done by a rear element and distance may be adjusted by shimming an air space between elements within the lens. Or infinity focus adjusted by shimming the lens mount, or even by having lens mounts of different thickness. Another lens may not use eccentric collars to adjust tilt, they may have small shims of variable thickness under three screws to tilt an element this way or that. And there are a few lenses (usually less expensive primes) that have no adjustments at all. You get what you get.

    Some adjustments are very convenient to reach (like the one in the photo). Others require partial or near complete disassembly of the lens to make an adjustment and some require a special tool or jig to adjust or center an element.

    The bottom line is every lens is pretty complex, especially when trying to reverse engineer like we have to (repair manuals are very proprietary and manufacturers guard them jealously). Currently we’re very comfortable with our adjustment abilities for some lenses, can do certain things with others, and have no idea what to do for some lenses.

  • Allan Sheppard

    Hi Roger, another very informative post opening up the darker side of lens manufacturer performance.

    Could you expand the mechanics of adjusting the lens/lens groups. Zoom lenses can have many lenses, yet from your picture above it seems that the access hole would only enable 1 or 2 lenses to be adjusted.

    If only a couple of lenses/lens groups are able to be adjusted does this mean that all the other lenses are fixed or have less/mimimal affect on focussing?

  • Another great post Roger, fascinating but directly relevant to our day-to-day photography. As always your approach does you enormous credit. I don’t think you are neglecting the business side at all, to me it makes absolute business sense to maintain and be known for your standards, especially at the top end of the lens market. What a contrast to certain companies – http://www.nytimes.com/2012/03/14/opinion/why-i-am-leaving-goldman-sachs.html?_r=2&pagewanted=all

  • Roger Cicala

    We are looking to start AF adjustment and perhaps offer testing when I get a bit more equipment (Imatest has some limitations). Bad pixel mapping, right now at least, requires the Canon Factory Service software. They won’t let anyone but an authorized service center have that. And if we became and ASC I’d be under nondisclosure so I couldn’t write about all the stuff I find out when I take them apart 🙂

Follow on Feedly