LensRentals.com

What I Did Over Winter Vacation

Posted by

The vast majority of my blog posts are about things other than Lensrentals, but a couple of times a year I put out an update about the things we're doing in-house. Those who like a 'behind the scenes' look usually enjoy finding out, well, what we're doing behind the scenes. This year, since most of our Winter efforts focused on testing and quality assurance, we've also learned some interesting things.

Lensrentals has roughly doubled in size every year for the last 5 years. That sort of growth means it's not "Roger in command 24/7" anymore. We've added people who do various things better than I do, and who have time to do those things. That has allowed me to go do the things that I really enjoy doing the most (playing with gear) and that I think are the most important (quality assurance).

Quality assurance is something that we've always prided ourselves on and I've always felt comfortable saying "no one takes better care of their equipment than Lensrentals". But, being human, we didn't do it perfectly, and if we aren't perfect then there's room for improvement. I'm comfortable that in 2012 we will deliver a level of equipment excellence that we've never attained before (and that no one else is close to).

Improved Testing

We've always inspected everything, tested everything, and been very open about exactly what tests and procedures we use on every piece of equipment.  I'm not talking about "we took some shots of the building next door" or "the person across the room" testing. Every item is tested by a tech when it comes in and again before it goes out. You can hardly walk anywhere in our offices without seeing specific test charts, focus charts, and and right-angle grids set up for just that purpose.

But despite thousands of dollars invested in charts and testing equipment, and 6 people who's only job here is to test equipment, the inevitable happens: we occasionally miss something. Sometimes it's simply human error: someone made a mistake. Sometimes everything is done properly, but testing with human vision (show the same images to two people and ask them to pick which lines are clear and which aren't and you get two slightly different answers) has limitations. No matter how carefully we tested, we still occasionally miss things in a 15 minute inspection that a photographer might see in days of use.

In an effort to standardize our testing and make it more accurate we invested in an Imatest computerized testing system in in September. Since then we've been testing lenses. Every. Single. Lens. Do you know how many lenses that is? I would if I could count that high.

Imatest let us eliminate the human, judgement-call part of testing that occurs whenever a person evaluates images from a test chart. Now we get actual MTF numbers for each lens, not just in the center but across the entire surface. Suddenly, instead of just arguing about whether a lens was softer in the right lower corner, we could see it.

Imatest printout for a lens with a slightly tilted element.  Most interesting is that the lower right corner still resolves well enough that it passes our most stringent optical testing. But a photographer making large prints would notice the right lower corner was softer.

As we began testing all of our lenses, including the ones that had passed our visual  inspections, we saw that a few really weren't as OK as we thought they were (like the one in the example above). And as we tested more more lenses, the difference between the good lenses and the outliers became readily apparent (even though they'd all been tested by our technicians and found acceptable).

Results for the first (left) and first two (right) test sessions of Canon 35mm f/1.4 lenses. Three lenses that had passed optical inspection weren't as OK as we thought they were.

Over the winter we've tested almost all of our lenses, sent the ones that weren't OK off to repair and have developed the range of acceptable results for each Canon, Nikon, and Zeiss lens (we don't have enough copies of other brands to feel comfortable that we've clearly defined the acceptable range).

Imatest results for all of our current Canon 35mm f/1.4 lenses (left) and our 'acceptable results' standards (shaded area on right) for that lens for all tests going forward.

Is all of this critical? Not everyone thinks so. Our accountants certainly don't. They, and many of our competitors, believe if a lens is bad the renters will let you know without going to all this time and expense. But I think it is worthwhile. Our renters are often trying a lens to determine if the lens they're renting is something they might want to buy. They can't make an informed decision unless they absolutely know the lens they're shooting with is a good copy. Which group of Canon 35mm L lenses would you rather rent from, the upper right (our lens fleet as tested September, 2010) or the lower left graph (the current fleet)? Me too.

The Bad Thing about more Accurate Tests

Well, if you ask the business people they'll tell you it's the six plus figures Roger spent on all this testing stuff. For me, though, the bad thing was testing lenses when they returned from repair. The vast majority came back just perfect. But a few came back just as bad as when we sent them. So we sent them back. And they came back again without any improvement.

Pretty soon a pattern started to emerge: lenses that were broken, that were soft in general, or that had focusing issues got fixed quickly and accurately. But when a lens was reasonably sharp in the middle, but soft on one side, or soft in all corners, it was likely to come back just like we'd sent it. Those lenses, the ones that had decentered or tilted elements, were likely to not get fixed at the service center. So I started bothering and alienating people until I started getting some answers about why that was. I got some answers. Mostly off-the-record answers, but answers.

I was aware, of course, that modern factory repair centers use computerized programs and test targets to make optical adjustments on lenses. I wasn't aware that for most lenses, from most manufacturers, that procedure maximizes center sharpness but can't do a damn thing about a lens that has a tilted or decentered element. Some repair centers were pretty honest: either the adjustments were impossible to do or they were soooo time consuming that it wasn't economically feasible to make them. It depends largely upon the lens in question and its design.

Other repair centers simply made ridiculous statements, like the factory service center that told me the lens with the two corners below was "perfectly within specifications". Of course, when I asked what their corner resolution standards were I got disconnected somehow. When I called back I was told they didn't release that information to the public. I can see why: if your standards say the image below is acceptable, I wouldn't release those standards to the public either. I don't know about you, but to me one of these things is not like the other.

Opposite corners from a lens that was "within specifications" after repair.

So what do we do?  The lens isn't acceptable to rent like that, but we can't find a repair center to fix it. All of you who said "Set up an anonymous eBay account and sell it" shame on you. Plus you failed the gear-head test of the day. Aaron, Tim and I actually got pretty excited: if it can't be fixed, and it's not OK to rent, then it should be taken apart.

We knew a bit of optical theory, have a lot of hand's on experience, and now we had lots of lenses to experiment on -- it's not like we were going to try to tell our customers what the factory told us - we knew the lens wasn't OK and we weren't about to tell anyone it was. So over the Winter we've invested in a Century Lens Test Projector , various lens mounts, some tools, and many hours of trial and error. Surprisingly, we learned that (in some cases at least) we can do what the repair center couldn't do. It's not that we're so smart. Probably more like we're really stubborn.

Adjusting resolution on a Lens Projector

Just in case you have the idea that "they could do this if they wanted to at the repair center", as best I can tell that's not true. It takes a long time to optically adjust a lens that's decentered or has a tilted element, if it's possible at all. (When you look at the images below, do the math and figure out how many in-between images there were - and remember each image was taken after a session of adjustment on the lens projector.). If you asked us to do it for you, we'd tell you it's not worth it, we'd have to charge a fortune. It's only worth it to us because the alternative is writing off  a $2,000 lens.

But for lenses we have in stock, at least, we're able to make changes like the before and after below. The graphs are a bit confusing at first, but simply look at the numbers in the yellow boxes, those are the MTF50 of the lens at that location. It's obvious the right side has been markedly improved after we've finished adjusting it.

MTF 50 numbers (yellow boxes) before (above) and after (below) adjustment.

And in case you're wondering, you can tell the difference in real world, too. Here are some corner crops at 100% after the adjustment was done (This is the lens from the Imatest charts above, not the same lens as the previous corner crops way above. But they both looked similar.). It was "within specifications" according to the factory before we adjusted it (image 3394 above). I guess we'll say that it wasn't until image 3411 above that I'd say it was "within Lensrentals specifications".

Opposite corners after centering adjustments.

 

Of course, there are a few among you who remembered that I said we can only optically adjust some lenses and are thinking 'so you sell the rest on eBay, right'? Yep, right. Except we sell them as parts. Broken lenses often sell for nearly as much as working lenses (something to consider if you have some broken lenses or cameras around). The parts in them are worth more than the lens itself in most cases. If Nikon goes through with their new idea to not sell parts I may start a new business buying new Nikon lenses, taking them apart, and selling the parts. There's good money to be made doing that. If I don't do it, I bet someone else will.

Oh, One Last Thing

We're really proud of the improvement we've made in our lens fleet over the Winter. We've repaired or eliminated over 150 lenses that weren't up to our new standards. I'm absolutely comfortable your odds of getting an excellent lens are better with our rental lenses than if you bought one new. (I'm comfortable saying that because I test lenses new out of the box before we put them in stock. The failure rate of those lenses optically, is about 1.5%. Nobody at the camera store opens the box and tests the lens.)

But we also know that shipping plays havoc with lenses and every year we look at ways to improve our preventative packing. We've tried hard cases (transmits too much shock and adds to shipping costs) and numerous different types of packing material. Last fall we started using heavy, padded cases to ship most lenses in, and used a heavy foam packing around those. It worked very well and we had far fewer lenses damaged in transit.

But the foam, while recyclable to some degree, only lasted for a few shipments. Unwrapping layers of foam, and then remembering how to repack it when you sent the lenses back got confusing sometimes. So over the Winter, Tyler negotiated with some of our supplies to custom make foam inserts for our commonly used box sizes.

Now it's easy as can be: put the lens in the case, put the case in the padded box,

put the top on, seal the box. It's faster, simpler, provides more consistent protection, and the foam inserts can be reused dozens of times, so it's friendly on the environment too.

 

This was probably more than you wanted to know, but that should bring you up to date on what we've done over the Winter. We want to start every year knowing we're doing an even better job than we did the year before. I'm comfortable that's where we are as the 2012 rental season kicks off.

 

Roger Cicala

Lensrentals.com

March, 2012

45 Responses to “What I Did Over Winter Vacation”

Andre said:

Thanks for an interesting post as always Roger!

And I hate to say this, because this might cause you guys to raise your used prices, but do the used lenses that you sell go through this kind of QA? If so, that would seem be a pretty valuable benefit of buying from you guys.

LensRentals Employee

Roger Cicala said:

Andre,

They do, and those that don't pass don't get sold. But we consider that just reasonable. If I was buying a rental lens, I'd want assurance that it was in good condition.

Saumene said:

So much testing and attention to detail, and expertise for a rental house. You guys should just start your own camera company

Mike said:

Roger,

You certainly run a first class operation! It's quite refreshing that you are so uncompromising on quality and customer satisfaction when I'm sure you could easily have kept your QA process the same and increased you bottom line. I'm sure the decision will pay dividends for you in the long run. Cheers!

A said:

Excellent and informative as ever Roger. Care and attention to detail like this is rare, and it is appreciated!

Budi said:

Roger, personally I'd buy new lenses from you exclusively, if you run the same tests on them.
:)

Charles said:

What do you suggest I do with a lens that has a tilted element? To me, that lens is unreliable enough that I can't use it at wider apertures anymore—which is a shame, since I paid more to get the 2.8 version of that zoom.

Am I wasting time/money if I send it for repair, if (according to your experience) it will simply come back the same? Am I stuck forever with a "defective" lens? (I would be tempted to ask you guys to repair it!)

Håkon K. Olafsen said:

I just recently started following your blog, but you keep impressing me. Again a really interesting topic, with useful information.

When do you establish a shop here in Europe? I would love a place where I could rent lenses before I buy them, and to rent that special one when I really need it.

Keep writing about how you work at lensrentals.

Christoph Breitkopf said:

Hello Roger,

how do you ensure accurate focus when shooting test charts? Do you use focus bracketing? I'm just trying to find out whether my bunch of old MF Nikkors are "good enough" to get a D800 instead of D700 and my results depend very much on focusing.

- chris

LensRentals Employee

Roger Cicala said:

Charles, what lens is it? I may be able to make some suggestions.

LensRentals Employee

Roger Cicala said:

Hi Chris,

Live view focusing and focus bracketing are the key. Autofocus is unreliable for critical testing.

Samuel Hurtado said:

Amazing.

And +1 for opening up shop in Europe!

Joe Sankey said:

Bravo, Roger. Raising the standards of excellence yet again. (Not that any of us who know you are surprised.)

tom alicoate said:

Like Budi said above. It would be interesting to see the response to selling lenses with a Lens Rentals Gold seal of approval on them. I think it would be a great benefit to know you didn't get a lemon. It is difficult to determine just by looking at my random photos. I have a 24-70L which is good, but sometimes even in ideal conditions, I think maybe it is a little inconsistent. I can't put my finger on it though. I would love to send my camera and a couple of lenses in to you and see if you could calibrate them together. I am sure you could do a much better job then I can with the Micro Focus Adjustment, and now it seems like you can do an even better job than Canon. Thanks for the article.

Tom

Jerry Russell said:

I'm curious: if you reject a lens with tilted or off-center elements and poor edge resolution back to the manufacturer right after it arrives in your shop for initial testing, do the manufacturers routinely accept those lenses back, no questions asked? And if so, does that force them to resell the lenses through the "reconditioned" channel, if at all?

Also, are you able to test camera bodies for things like optical alignment and focusing accuracy? Do you find a similar lack of consistency in products from the factory?

+1 to what Budi said -- I would consider buying new stuff from you if I know it's been tested.

About spare parts (especially for Nikon) I would imagine that some parts would be much more in demand than others. For example, front lens elements and autofocus motors for lenses, or shutters for cameras. Hopefully there aren't enough dead cameras and lenses to form an adequate supply of those most commonly needed replacement parts. So I think photographers would be smart to keep up the pressure on Nikon, to reverse their decision not to sell parts. If we let them get away with it, it may be just a matter of time before Canon follows suit. And, the D800 was tempting me to switch to Nikon -- but not since I heard about their new parts policy.

Phillip Palacios said:

Kudos! It seems then that we would be wiser to purchase a used lens from you than a new copy from our local dealer, amazing.

THANK YOU for doing business right.
I don't rent too often, but I am happy to say that when I do, I'll always use you guys.
p

LensRentals Employee

Roger Cicala said:

Jerry,

I really hope Nikon reverses themselves, but I have a feeling they won't: Sony, Sigma, Tamron, and Tokina are already in the "no parts sold" camp. It's a Sony lens, BTW, that I have up there with the example of "it's within specifications" in the article. We tried sending it to third party repair, who sent it back to Sony because they couldn't get parts, who repeated it was within specifications. Monopolies have no need to improve, they can be 'take it or leave it'.

My remaining hope, and I think it's reasonable, is that Canon is going to say "here's a chance to have obviously better customer service" and not follow the others. As someone in charge of 300 to 400 repairs a month, I can say I agree with your decision: Canon repairs are much faster and usually less expensive than any of the others.

When lenses are bad out of the box we do return them to the retailer or manufacturer. I can't say what's done with them then. Hopefully they all get refurbished.

Roger

Joe Towner said:

Damn it Roger, quit goofing around. :) Thank you for all that you do to push the edge of perfection. I pity the camera rep that says 'it's good enough' to you.

+1 on dealing in new 'tested by LensRentals' lenses. I promise no one will ask you to price match another vendor. Anyone who does will get all their gear taken away.

Tom said:

What would you charge to calibrate my Nikon 70-200 vr2?

Jeff said:

+1 to what Budi said — I would consider buying lenses from Lens Rental if I knew it’s been tested.

Scott said:

Hi Roger,

I have a Canon 16-35 f2.8 MkI with severe softness on the right side, is this one of the lenses that can be adjusted? Already been to Canon twice and whilst I am a long time Canon user they just seem to make this lens worse.

Gerry said:

Roger, the integrity with which you operate your business is outstanding. It is an obvious differentiator between LR and other rental companies. I only wish all businesses worked the way you do. Keep up the good work!
Cheers.. Gerry.

LensRentals Employee

Roger Cicala said:

Scott,

I'm afraid I don't know about the 16-35 Mk I. We only have the Mk II version and Canon has been pretty good with that one.

Scott said:

Roger,

Thanks for replying. Really enjoy the blog too, though it can't last, straight talking common sense with no brand loyalty just isn't confrontational enough to make the headlines :-) Please, don't change a thing.......

Ed said:

Roger,

If you expanded your business to sell lenses which include a printout of their "measured MTF50 zones" like your IMG_3411 above, I bet it would be a runaway success. Or how about charging a fee for the service of measuring lenses sent to you?
You're a real breath of fresh air; keep doing your wonderful job.

Ed

Samuel H said:

Charging a fee to test or calibrate other people's lenses doesn't make sense.

But selling new, tested lense makes a lot of sense.

You get the lens and test it. If it's bad, you send it back. If it's good, you classify it as either "good" or "outstanding" (based on which half of that acceptance zone it falls into), then sell the "good" ones at the regular price, and the "outstanding" ones at a premium.

You get the standard margin on the "good" lenses, and a bigger one on the "outstanding" ones. Buyers know they're never getting a bad lens, and if they want a great one they can get it for a bit more.

LensRentals Employee

Roger Cicala said:

Samuel,

Unfortunately there isn't enough margin in Canon or Nikon lenses to do that unless you're a huge company like B&H or Adorama. Most photography shops are pleased to break even on lenses. They make a bit on cameras and a lot on accessories. The difference in what we pay Canon for a lens and what B&H charges is usually 2-3%. By the time we depreciate the equipment and pay a tech for testing we'd be selling most lenses at a loss.

Roger

alek said:

Ditto what others said that I think there is a business niche for you to sell "new" lenses Roger ... and I think you could easily command a price premium that it has been "Lens Rental tested" ... and include a copy of your test data with the lens. With higher resolving sensors, tight QA is only going to become more and more important

My wife works in the adventure travel biz, so I fortunately will probably go to Africa with her this summer. I'd love a 500/F4 to compliment my 70-200ISv2 ... but realistically, I don't think I'm going to win the lottery and be able to get ... but if I did and was spending that much dough, I'd want some re-assurance that I'm getting a "good" copy.

BTW, if I were the CEO of Canon (or Nikon, etc), I'd be sending around your blog post to my QA people as an example of the type of stuff we should be talking about ... and more importantly, DOING to insure that 99+% of ours lens ship in top-notch shape.

Craig said:

I can see why the adjustment biz is a bad choice for you. Have you ever thought of offering bad pixel remapping, AF adjustment and such?

intrnst said:

LR is not a lens rental store, Roger is not a name.
They became a standard. LR is a valuable brand. Already.
If you want, you have a gold mine.

My sinceres congratulations, you and your company deserve it.
Carlos

LensRentals Employee

Roger Cicala said:

We are looking to start AF adjustment and perhaps offer testing when I get a bit more equipment (Imatest has some limitations). Bad pixel mapping, right now at least, requires the Canon Factory Service software. They won't let anyone but an authorized service center have that. And if we became and ASC I'd be under nondisclosure so I couldn't write about all the stuff I find out when I take them apart :-)

Richard said:

Another great post Roger, fascinating but directly relevant to our day-to-day photography. As always your approach does you enormous credit. I don't think you are neglecting the business side at all, to me it makes absolute business sense to maintain and be known for your standards, especially at the top end of the lens market. What a contrast to certain companies - http://www.nytimes.com/2012/03/14/opinion/why-i-am-leaving-goldman-sachs.html?_r=2&pagewanted=all

Allan Sheppard said:

Hi Roger, another very informative post opening up the darker side of lens manufacturer performance.

Could you expand the mechanics of adjusting the lens/lens groups. Zoom lenses can have many lenses, yet from your picture above it seems that the access hole would only enable 1 or 2 lenses to be adjusted.

If only a couple of lenses/lens groups are able to be adjusted does this mean that all the other lenses are fixed or have less/mimimal affect on focussing?

LensRentals Employee

Roger Cicala said:

Allan,

It's really hard to generalize, all lenses are different and different methods are used. Most have at least two elements / groups that are adjustable and many have more than two. Sometimes one group can be adjusted many ways. For example the front element on some lenses has a sliding ramp that lets it move forward and backwards (in relation to the rear of the lens). It may also be able to move side to side in order to help center the lens. It may also have eccentric adjustable screws that allow you to adjust tilt. And that's just on one element.

For another lens, centering may be done by a rear element and distance may be adjusted by shimming an air space between elements within the lens. Or infinity focus adjusted by shimming the lens mount, or even by having lens mounts of different thickness. Another lens may not use eccentric collars to adjust tilt, they may have small shims of variable thickness under three screws to tilt an element this way or that. And there are a few lenses (usually less expensive primes) that have no adjustments at all. You get what you get.

Some adjustments are very convenient to reach (like the one in the photo). Others require partial or near complete disassembly of the lens to make an adjustment and some require a special tool or jig to adjust or center an element.

The bottom line is every lens is pretty complex, especially when trying to reverse engineer like we have to (repair manuals are very proprietary and manufacturers guard them jealously). Currently we're very comfortable with our adjustment abilities for some lenses, can do certain things with others, and have no idea what to do for some lenses.

George said:

It would be amazing if, when you figure out how to fix decentering in certain lenses, you published how you went about doing it--it'd obviously make more business sense to keep that info to yourself, though. It's a massive problem, especially on IS lenses. :) GREAT article.

CarVac said:

I'm not sure if I should bring this up here, but I rented Canon mount Tokina 11-16 for a trip last week and it could not focus all of the way to infinity in the wideangle range. Of course, I was sent a replacement immediately which, while it exhibits the same focus shifting tendencies when zooming, can focus (if just barely) to infinity. Do you have any way (or plan) to test that sort of thing before sending out lenses?

I was surprised that this problem existed at all, since it was a mechanical and not electrical issue. Its sharpness was perfectly acceptable, but the problem probably wouldn't show up in the resolution or focusing accuracy tests (it autofocused alright up close).

Also, just wondering: is that a typical problem with 11-16's (in Canon mount)? That the problem existed to a nonzero extent in both lenses worries me, especially since I was considering purchasing a copy of the lens.

LensRentals Employee

Roger Cicala said:

Infinity focus is a combination of lens-to-flange distance and within lens distances. That's why it's really common with adapters: if they're off just a bit, some lenses won't focus to infinity. But it can be a problem without adapters. If you send me your order number (email me) I'll check how that first lens did when it came back. My guess would be it didn't show up on any of our test cameras since we routinely check for that.

LensRentals Employee

Roger Cicala said:

I wouldn't mind showing it as far as business purposes, but unless you have a lens test projector, Imatest, or a collimator there's no way to do it at home. Plus you can really, really screw things up amazingly.

Randy said:

Roger,

It's a shame you left medicine, maybe you could have had an influence on the quality of medical care. Oh well, I guess you found another "calling." lol.
Keep up the good work!

Ralf C. Kohlrausch said:

Hi Roger,

I second Hakon. Please do open a branch in Europe. Most rental-companies around here stock old stuff, so if you wanted a 2,8/300 it likely would be one without VR. If you want to go beyond 300, your are lost at most places. Rental conditions demand you to pick stuff up in person and/or you pay for the shipping days as well.

Thanks for the informative and entertainings lines,

greets
Ralf C.

LensRentals Employee

Roger Cicala said:

Ralf and Hakon,

We'd love to, but we'd need European partners: we don't know shipping, taxes, collections and all the other things that would go with running a business in Europe. All we know about is gear :-)

Bryan Conner said:

If only you had a facility in Germany.........just dreaming.

TM said:

A German branch would be awesome^10!

Michael said:

Wow! Roger, you guys are amazing. I bought my Canon 10-22 from you guys this past thanksgiving and have loved it, and never doubted the purchase at all. It was an amazing deal.

I feel even better about my purchase now than I did before, even if my lens probably did't go through this extra testing.

P.S. I've reused the box my 10-22 came in too! Great packing.

Ken said:

Looks like your packaging solution is pretty good but have you ever looked at something like these?

http://www.betashell.com/

Leave a Reply