Roger's Corner

What I Did Over Winter Vacation

Published March 14, 2012

The vast majority of my blog posts are about things other than Lensrentals, but a couple of times a year I put out an update about the things we’re doing in-house. Those who like a ‘behind the scenes’ look usually enjoy finding out, well, what we’re doing behind the scenes. This year, since most of our Winter efforts focused on testing and quality assurance, we’ve also learned some interesting things.

Lensrentals has roughly doubled in size every year for the last 5 years. That sort of growth means it’s not “Roger in command 24/7” anymore. We’ve added people who do various things better than I do, and who have time to do those things. That has allowed me to go do the things that I really enjoy doing the most (playing with gear) and that I think are the most important (quality assurance).

Quality assurance is something that we’ve always prided ourselves on and I’ve always felt comfortable saying “no one takes better care of their equipment than Lensrentals”. But, being human, we didn’t do it perfectly, and if we aren’t perfect then there’s room for improvement. I’m comfortable that in 2012 we will deliver a level of equipment excellence that we’ve never attained before (and that no one else is close to).

Improved Testing

We’ve always inspected everything, tested everything, and been very open about exactly what tests and procedures we use on every piece of equipment.  I’m not talking about “we took some shots of the building next door” or “the person across the room” testing. Every item is tested by a tech when it comes in and again before it goes out. You can hardly walk anywhere in our offices without seeing specific test charts, focus charts, and and right-angle grids set up for just that purpose.

But despite thousands of dollars invested in charts and testing equipment, and 6 people who’s only job here is to test equipment, the inevitable happens: we occasionally miss something. Sometimes it’s simply human error: someone made a mistake. Sometimes everything is done properly, but testing with human vision (show the same images to two people and ask them to pick which lines are clear and which aren’t and you get two slightly different answers) has limitations. No matter how carefully we tested, we still occasionally miss things in a 15 minute inspection that a photographer might see in days of use.

In an effort to standardize our testing and make it more accurate we invested in an Imatest computerized testing system in in September. Since then we’ve been testing lenses. Every. Single. Lens. Do you know how many lenses that is? I would if I could count that high.

Imatest let us eliminate the human, judgement-call part of testing that occurs whenever a person evaluates images from a test chart. Now we get actual MTF numbers for each lens, not just in the center but across the entire surface. Suddenly, instead of just arguing about whether a lens was softer in the right lower corner, we could see it.

Imatest printout for a lens with a slightly tilted element.  Most interesting is that the lower right corner still resolves well enough that it passes our most stringent optical testing. But a photographer making large prints would notice the right lower corner was softer.

As we began testing all of our lenses, including the ones that had passed our visual  inspections, we saw that a few really weren’t as OK as we thought they were (like the one in the example above). And as we tested more more lenses, the difference between the good lenses and the outliers became readily apparent (even though they’d all been tested by our technicians and found acceptable).

Results for the first (left) and first two (right) test sessions of Canon 35mm f/1.4 lenses. Three lenses that had passed optical inspection weren't as OK as we thought they were.

Over the winter we’ve tested almost all of our lenses, sent the ones that weren’t OK off to repair and have developed the range of acceptable results for each Canon, Nikon, and Zeiss lens (we don’t have enough copies of other brands to feel comfortable that we’ve clearly defined the acceptable range).

Imatest results for all of our current Canon 35mm f/1.4 lenses (left) and our ‘acceptable results’ standards (shaded area on right) for that lens for all tests going forward.

Is all of this critical? Not everyone thinks so. Our accountants certainly don’t. They, and many of our competitors, believe if a lens is bad the renters will let you know without going to all this time and expense. But I think it is worthwhile. Our renters are often trying a lens to determine if the lens they’re renting is something they might want to buy. They can’t make an informed decision unless they absolutely know the lens they’re shooting with is a good copy. Which group of Canon 35mm L lenses would you rather rent from, the upper right (our lens fleet as tested September, 2010) or the lower left graph (the current fleet)? Me too.

The Bad Thing about more Accurate Tests

Well, if you ask the business people they’ll tell you it’s the six plus figures Roger spent on all this testing stuff. For me, though, the bad thing was testing lenses when they returned from repair. The vast majority came back just perfect. But a few came back just as bad as when we sent them. So we sent them back. And they came back again without any improvement.

Pretty soon a pattern started to emerge: lenses that were broken, that were soft in general, or that had focusing issues got fixed quickly and accurately. But when a lens was reasonably sharp in the middle, but soft on one side, or soft in all corners, it was likely to come back just like we’d sent it. Those lenses, the ones that had decentered or tilted elements, were likely to not get fixed at the service center. So I started bothering and alienating people until I started getting some answers about why that was. I got some answers. Mostly off-the-record answers, but answers.

I was aware, of course, that modern factory repair centers use computerized programs and test targets to make optical adjustments on lenses. I wasn’t aware that for most lenses, from most manufacturers, that procedure maximizes center sharpness but can’t do a damn thing about a lens that has a tilted or decentered element. Some repair centers were pretty honest: either the adjustments were impossible to do or they were soooo time consuming that it wasn’t economically feasible to make them. It depends largely upon the lens in question and its design.

Other repair centers simply made ridiculous statements, like the factory service center that told me the lens with the two corners below was “perfectly within specifications”. Of course, when I asked what their corner resolution standards were I got disconnected somehow. When I called back I was told they didn’t release that information to the public. I can see why: if your standards say the image below is acceptable, I wouldn’t release those standards to the public either. I don’t know about you, but to me one of these things is not like the other.

Opposite corners from a lens that was "within specifications" after repair.

So what do we do?  The lens isn’t acceptable to rent like that, but we can’t find a repair center to fix it. All of you who said “Set up an anonymous eBay account and sell it” shame on you. Plus you failed the gear-head test of the day. Aaron, Tim and I actually got pretty excited: if it can’t be fixed, and it’s not OK to rent, then it should be taken apart.

We knew a bit of optical theory, have a lot of hand’s on experience, and now we had lots of lenses to experiment on — it’s not like we were going to try to tell our customers what the factory told us – we knew the lens wasn’t OK and we weren’t about to tell anyone it was. So over the Winter we’ve invested in a Century Lens Test Projector , various lens mounts, some tools, and many hours of trial and error. Surprisingly, we learned that (in some cases at least) we can do what the repair center couldn’t do. It’s not that we’re so smart. Probably more like we’re really stubborn.

Adjusting resolution on a Lens Projector

Just in case you have the idea that “they could do this if they wanted to at the repair center”, as best I can tell that’s not true. It takes a long time to optically adjust a lens that’s decentered or has a tilted element, if it’s possible at all. (When you look at the images below, do the math and figure out how many in-between images there were – and remember each image was taken after a session of adjustment on the lens projector.). If you asked us to do it for you, we’d tell you it’s not worth it, we’d have to charge a fortune. It’s only worth it to us because the alternative is writing off  a $2,000 lens.

But for lenses we have in stock, at least, we’re able to make changes like the before and after below. The graphs are a bit confusing at first, but simply look at the numbers in the yellow boxes, those are the MTF50 of the lens at that location. It’s obvious the right side has been markedly improved after we’ve finished adjusting it.

MTF 50 numbers (yellow boxes) before (above) and after (below) adjustment.

And in case you’re wondering, you can tell the difference in real world, too. Here are some corner crops at 100% after the adjustment was done (This is the lens from the Imatest charts above, not the same lens as the previous corner crops way above. But they both looked similar.). It was “within specifications” according to the factory before we adjusted it (image 3394 above). I guess we’ll say that it wasn’t until image 3411 above that I’d say it was “within Lensrentals specifications”.

Opposite corners after centering adjustments.

 

Of course, there are a few among you who remembered that I said we can only optically adjust some lenses and are thinking ‘so you sell the rest on eBay, right’? Yep, right. Except we sell them as parts. Broken lenses often sell for nearly as much as working lenses (something to consider if you have some broken lenses or cameras around). The parts in them are worth more than the lens itself in most cases. If Nikon goes through with their new idea to not sell parts I may start a new business buying new Nikon lenses, taking them apart, and selling the parts. There’s good money to be made doing that. If I don’t do it, I bet someone else will.

Oh, One Last Thing

We’re really proud of the improvement we’ve made in our lens fleet over the Winter. We’ve repaired or eliminated over 150 lenses that weren’t up to our new standards. I’m absolutely comfortable your odds of getting an excellent lens are better with our rental lenses than if you bought one new. (I’m comfortable saying that because I test lenses new out of the box before we put them in stock. The failure rate of those lenses optically, is about 1.5%. Nobody at the camera store opens the box and tests the lens.)

But we also know that shipping plays havoc with lenses and every year we look at ways to improve our preventative packing. We’ve tried hard cases (transmits too much shock and adds to shipping costs) and numerous different types of packing material. Last fall we started using heavy, padded cases to ship most lenses in, and used a heavy foam packing around those. It worked very well and we had far fewer lenses damaged in transit.

But the foam, while recyclable to some degree, only lasted for a few shipments. Unwrapping layers of foam, and then remembering how to repack it when you sent the lenses back got confusing sometimes. So over the Winter, Tyler negotiated with some of our supplies to custom make foam inserts for our commonly used box sizes.

Now it’s easy as can be: put the lens in the case, put the case in the padded box,

put the top on, seal the box. It’s faster, simpler, provides more consistent protection, and the foam inserts can be reused dozens of times, so it’s friendly on the environment too.

 

This was probably more than you wanted to know, but that should bring you up to date on what we’ve done over the Winter. We want to start every year knowing we’re doing an even better job than we did the year before. I’m comfortable that’s where we are as the 2012 rental season kicks off.

 

Roger Cicala

Lensrentals.com

March, 2012

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Roger's Corner
  • intrnst

    LR is not a lens rental store, Roger is not a name.
    They became a standard. LR is a valuable brand. Already.
    If you want, you have a gold mine.

    My sinceres congratulations, you and your company deserve it.
    Carlos

  • Craig

    I can see why the adjustment biz is a bad choice for you. Have you ever thought of offering bad pixel remapping, AF adjustment and such?

  • Ditto what others said that I think there is a business niche for you to sell “new” lenses Roger … and I think you could easily command a price premium that it has been “Lens Rental tested” … and include a copy of your test data with the lens. With higher resolving sensors, tight QA is only going to become more and more important

    My wife works in the adventure travel biz, so I fortunately will probably go to Africa with her this summer. I’d love a 500/F4 to compliment my 70-200ISv2 … but realistically, I don’t think I’m going to win the lottery and be able to get … but if I did and was spending that much dough, I’d want some re-assurance that I’m getting a “good” copy.

    BTW, if I were the CEO of Canon (or Nikon, etc), I’d be sending around your blog post to my QA people as an example of the type of stuff we should be talking about … and more importantly, DOING to insure that 99+% of ours lens ship in top-notch shape.

  • Roger Cicala

    Samuel,

    Unfortunately there isn’t enough margin in Canon or Nikon lenses to do that unless you’re a huge company like B&H or Adorama. Most photography shops are pleased to break even on lenses. They make a bit on cameras and a lot on accessories. The difference in what we pay Canon for a lens and what B&H charges is usually 2-3%. By the time we depreciate the equipment and pay a tech for testing we’d be selling most lenses at a loss.

    Roger

  • Charging a fee to test or calibrate other people’s lenses doesn’t make sense.

    But selling new, tested lense makes a lot of sense.

    You get the lens and test it. If it’s bad, you send it back. If it’s good, you classify it as either “good” or “outstanding” (based on which half of that acceptance zone it falls into), then sell the “good” ones at the regular price, and the “outstanding” ones at a premium.

    You get the standard margin on the “good” lenses, and a bigger one on the “outstanding” ones. Buyers know they’re never getting a bad lens, and if they want a great one they can get it for a bit more.

  • Ed

    Roger,

    If you expanded your business to sell lenses which include a printout of their “measured MTF50 zones” like your IMG_3411 above, I bet it would be a runaway success. Or how about charging a fee for the service of measuring lenses sent to you?
    You’re a real breath of fresh air; keep doing your wonderful job.

    Ed

  • Scott

    Roger,

    Thanks for replying. Really enjoy the blog too, though it can’t last, straight talking common sense with no brand loyalty just isn’t confrontational enough to make the headlines 🙂 Please, don’t change a thing…….

  • Roger Cicala

    Scott,

    I’m afraid I don’t know about the 16-35 Mk I. We only have the Mk II version and Canon has been pretty good with that one.

  • Gerry

    Roger, the integrity with which you operate your business is outstanding. It is an obvious differentiator between LR and other rental companies. I only wish all businesses worked the way you do. Keep up the good work!
    Cheers.. Gerry.

  • Scott

    Hi Roger,

    I have a Canon 16-35 f2.8 MkI with severe softness on the right side, is this one of the lenses that can be adjusted? Already been to Canon twice and whilst I am a long time Canon user they just seem to make this lens worse.

  • +1 to what Budi said — I would consider buying lenses from Lens Rental if I knew it’s been tested.

  • Tom

    What would you charge to calibrate my Nikon 70-200 vr2?

  • Joe Towner

    Damn it Roger, quit goofing around. 🙂 Thank you for all that you do to push the edge of perfection. I pity the camera rep that says ‘it’s good enough’ to you.

    +1 on dealing in new ‘tested by LensRentals’ lenses. I promise no one will ask you to price match another vendor. Anyone who does will get all their gear taken away.

  • Roger Cicala

    Jerry,

    I really hope Nikon reverses themselves, but I have a feeling they won’t: Sony, Sigma, Tamron, and Tokina are already in the “no parts sold” camp. It’s a Sony lens, BTW, that I have up there with the example of “it’s within specifications” in the article. We tried sending it to third party repair, who sent it back to Sony because they couldn’t get parts, who repeated it was within specifications. Monopolies have no need to improve, they can be ‘take it or leave it’.

    My remaining hope, and I think it’s reasonable, is that Canon is going to say “here’s a chance to have obviously better customer service” and not follow the others. As someone in charge of 300 to 400 repairs a month, I can say I agree with your decision: Canon repairs are much faster and usually less expensive than any of the others.

    When lenses are bad out of the box we do return them to the retailer or manufacturer. I can’t say what’s done with them then. Hopefully they all get refurbished.

    Roger

  • Kudos! It seems then that we would be wiser to purchase a used lens from you than a new copy from our local dealer, amazing.

    THANK YOU for doing business right.
    I don’t rent too often, but I am happy to say that when I do, I’ll always use you guys.
    p

  • Jerry Russell

    I’m curious: if you reject a lens with tilted or off-center elements and poor edge resolution back to the manufacturer right after it arrives in your shop for initial testing, do the manufacturers routinely accept those lenses back, no questions asked? And if so, does that force them to resell the lenses through the “reconditioned” channel, if at all?

    Also, are you able to test camera bodies for things like optical alignment and focusing accuracy? Do you find a similar lack of consistency in products from the factory?

    +1 to what Budi said — I would consider buying new stuff from you if I know it’s been tested.

    About spare parts (especially for Nikon) I would imagine that some parts would be much more in demand than others. For example, front lens elements and autofocus motors for lenses, or shutters for cameras. Hopefully there aren’t enough dead cameras and lenses to form an adequate supply of those most commonly needed replacement parts. So I think photographers would be smart to keep up the pressure on Nikon, to reverse their decision not to sell parts. If we let them get away with it, it may be just a matter of time before Canon follows suit. And, the D800 was tempting me to switch to Nikon — but not since I heard about their new parts policy.

  • Like Budi said above. It would be interesting to see the response to selling lenses with a Lens Rentals Gold seal of approval on them. I think it would be a great benefit to know you didn’t get a lemon. It is difficult to determine just by looking at my random photos. I have a 24-70L which is good, but sometimes even in ideal conditions, I think maybe it is a little inconsistent. I can’t put my finger on it though. I would love to send my camera and a couple of lenses in to you and see if you could calibrate them together. I am sure you could do a much better job then I can with the Micro Focus Adjustment, and now it seems like you can do an even better job than Canon. Thanks for the article.

    Tom

  • Joe Sankey

    Bravo, Roger. Raising the standards of excellence yet again. (Not that any of us who know you are surprised.)

  • Amazing.

    And +1 for opening up shop in Europe!

  • Roger Cicala

    Hi Chris,

    Live view focusing and focus bracketing are the key. Autofocus is unreliable for critical testing.

  • Roger Cicala

    Charles, what lens is it? I may be able to make some suggestions.

  • Christoph Breitkopf

    Hello Roger,

    how do you ensure accurate focus when shooting test charts? Do you use focus bracketing? I’m just trying to find out whether my bunch of old MF Nikkors are “good enough” to get a D800 instead of D700 and my results depend very much on focusing.

    – chris

  • I just recently started following your blog, but you keep impressing me. Again a really interesting topic, with useful information.

    When do you establish a shop here in Europe? I would love a place where I could rent lenses before I buy them, and to rent that special one when I really need it.

    Keep writing about how you work at lensrentals.

  • Charles

    What do you suggest I do with a lens that has a tilted element? To me, that lens is unreliable enough that I can’t use it at wider apertures anymore—which is a shame, since I paid more to get the 2.8 version of that zoom.

    Am I wasting time/money if I send it for repair, if (according to your experience) it will simply come back the same? Am I stuck forever with a “defective” lens? (I would be tempted to ask you guys to repair it!)

  • Budi

    Roger, personally I’d buy new lenses from you exclusively, if you run the same tests on them.
    🙂

  • A

    Excellent and informative as ever Roger. Care and attention to detail like this is rare, and it is appreciated!

  • Mike

    Roger,

    You certainly run a first class operation! It’s quite refreshing that you are so uncompromising on quality and customer satisfaction when I’m sure you could easily have kept your QA process the same and increased you bottom line. I’m sure the decision will pay dividends for you in the long run. Cheers!

  • Saumene

    So much testing and attention to detail, and expertise for a rental house. You guys should just start your own camera company

  • Roger Cicala

    Andre,

    They do, and those that don’t pass don’t get sold. But we consider that just reasonable. If I was buying a rental lens, I’d want assurance that it was in good condition.

  • Andre

    Thanks for an interesting post as always Roger!

    And I hate to say this, because this might cause you guys to raise your used prices, but do the used lenses that you sell go through this kind of QA? If so, that would seem be a pretty valuable benefit of buying from you guys.

Follow on Feedly