Equipment

Optical Quality Assurance

Published February 5, 2016

 

Pinhole images from new, but optically decentered lens. Olaf Optical Testing, 2016

 

After we published a number of posts about copy-to-copy variation, people were quick to say that this company or that needs to ‘just’ improve quality control. I totally agree but realize most people don’t have a clue what ‘improving quality control’ would really look like. I think they have some vague idea of hiring a guy named Joe to sit at the end of the assembly line, check all the lenses, and reject all the bad ones. Optical quality control for lenses is way, way more complicated than that.

Before we start, let’s get a couple of things out of the way.

  1.  If you expect that the lens you buy has the exact same MTF that the manufacturer published, then don’t read this. You’ve got too much reeducation to do.
  2.  If you ever say, “for that kind of money I expect perfect” please don’t read further. Perfect doesn’t exist. For $20,000 and up you can get pretty close. For $2,000 you should be able to get reasonably close.

On the other hand, a reasonably expensive lens should have reasonable quality control and minimal copy-to-copy variation.  We know it can be done; some manufacturers are already doing it. Others totally suck. But when they suck, sticking Joe at the end of the assembly line won’t really fix things. If you want to read this pretty long article, I’ll explain why, and also explain some of what can be done.

A Few Basics First

To Some Degree, Consistency Costs Money.

Here’s a quick, relatively cheap example. I need to buy a fairly large, optically flat piece of glass to calibrate our MTF bench. But how accurate of an optical flat do I need? A flat accurate to 1/4 wavelength of visible light should be flat enough, right? That costs $450. But 1/20 wavelength is more accurate. That one costs $825. Is it worth the difference? Well, it depends on what you’re doing with it. For my purposes, 1/4 wavelength is plenty good.

 

Please don’t respond with, ‘well there will be discounts for quantity’. Sure there are, but better still costs more. Want a million 1mm screws toleranced to be between 0.96 and 1.04mm? They’re really cheap. Want them to be between 0.99 and 1.01mm? The price is going up. It’s cheaper per screw to buy a million, but the tighter tolerance is still more expensive.

Optical Testing May Be Cheap. Or Not.

People love to say testing won’t add much cost to a lens.  ‘Not much’ is a pretty variable term, though. A state-of-the-art MTF bench costs between $250,000 and $500,000. And it takes the time to test a lens. A single test run across the lens takes a minute or two. We need at least two angles (4 would be better) for each lens; plus, mounting, unmounting, and other things take time. So we can test a lens in 5 minutes; 12 lenses an hour. Let’s work 12 hour days and we can test 144 lenses a day. Unless it’s a zoom lens. Then you must test each lens at multiple focal lengths (3 or 4) so things down to maybe 3 or 4 lenses an hour.

 

If your assembly line is churning out 5,000 lenses a week you will need a lot of benches. Any way you slice it, the equipment and the operators to run it are going to add something to the price of the lens. I think a company being really efficient could do it for an additional $20 per prime lens, $60 per zoom. (I couldn’t break even at that cost, but I’m assuming they have economies of scale that I don’t have.)

 

Of course, if you want the best, state of the art testing, like interferometry, things could get really pricey. Interferometry requires different reference elements for each type of lens, and reference elements alone cost $25,000 to $50,000 each.

You can set up an Imatest-type program or even high-quality optical targets on a rail system to keep things properly square and get good results, especially if the images are computer analyzed.  That’s still very good and certainly less expensive. So there are lots of good ways to check optical performance to a high level, but they are going to cost a bit of money.

 

 

Or, you can make some ink-jet printed, thick-bar test charts, hook a camera to a computer, and let the computer say ‘pass/fail’ and poorly test a lot of lenses for almost no cost at all. This is actually what many manufacturers do. It’s quick, cheap, and has the added advantage of setting the failure bar really low so there aren’t many rejections. They have to do something like this anyway, in order to set autofocus and back focus for most lenses.

Some manufacturers do have high-level test equipment and test every lens. More often, they just do this type of testing on a percentage of lenses. (BTW – this sure doesn’t mean that type of equipment is available in your country’s repair center; it may only exist at the corporate manufacturing facility.) Many manufacturers do have some type of testing other than the ‘computer takes images of a few thick, ink-jet bars’. Then again, many do not.

But the Most Expensive Thing is Failing

But if you’re thinking, “OK, so they all need to have better testing and that doesn’t cost too much,” well, no there’s a lot more to it that that. The worst thing that can happen from the manufacturer’s standpoint is for a lens to be completely assembled and then fail quality control at the end of the assembly line. Let’s say that lens you pay $1,000 for costs $500 to manufacture and assemble and then fails a quality check.

What do you do? Throw it in the trash and take a loss? That’s actually fairly likely if it’s a consumer-grade lens. Disassembling and testing each component to salvage the good parts may be more expensive than throwing it away. You could just disassemble it, reuse all the parts without testing them, and hope it was just an assembly error and not a bad part. (This is pretty common among small manufacturers, and actually works pretty well.) But it still means the assembly cost of the lens is more than doubled. Now we assembled, disassembled, and reassembled and still have just one lens to sell.

Whatever you do, you will have to account for the cost of failed lenses. If you have to manufacture 1,100 lenses to sell 1,000, then each of those 1,000 lenses has to include the cost of those 100 failures. If you had to disassemble the 100 but were able to reuse the parts that still costs money and the cost has to be accounted for. One way or another, failure is always expensive. So good quality control needs to be done before and during lens assembly, not just afterwards.

And, just in case you haven’t thought about it everyone, from the owner of the company to the manager of the assembly line, at some point has to think, “well, just how bad did those 100 lenses fail? Are we maybe being too picky?”

Optical Quality Assurance is a Series of Feedback Loops

Design Feedback Loops

You may not have thought of it, but the biggest part of quality assurance probably occurs during design and prototyping.

Optical Design

The lens designer is given a set of goals and constraints; price, size, necessary features, release date, etc. The designer usually tries to avoid really expensive glass types and extreme lens shapes; these can cost 10X to 30X more than common glass.

Prototype elements for a future lens. Image courtesy C-4 Optics.

 

After the initial design is completed there are a series of redesigns to refine the lens. A lot of this refinement has to do with ‘tolerancing’. Each element in the lens is evaluated given the known accuracy to which it can be made, to see what effect that variation will have on actual performance of the assembled lens.  The lens may have to be redesigned several times so that inevitable variation in glass element production doesn’t degrade performance too severely.

There are tradeoffs for every change. We may get better consistency by changing this, but it might cause a slight aberration or increase the cost significantly. Other people are sitting in these meetings along with the optical designers. The marketing guys may want the best MTF curves possible, even if that may mean the lens is going to have more variance. The financial people may veto a more expensive glass. Different companies may give different emphasis to different things. The same company may give different emphasis with different lenses.

Mechanical Design

The mechanical design of the lens is usually far more complex than the optical design, and the same steps have to be made during this design. How accurately can each element be placed in the lens barrel? How much variation will occur during focusing or zooming? The optical designer will have identified if there are critical elements that may need to be adjusted in the lens so that the mechanical designer can add shims, eccentric collars, centering rings, or other forms of adjustment to his design.

The complexity here has to be seen to be believed. I’m working with another company designing some simple, manual-focus prime lenses. The designs for the mechanical parts run well over a hundred printed pages. That’s just mechanical parts; no autofocus, no zoom mechanisms, no image stabilization, and no optics. Every single part has to be sourced to certain specifications.

Image Courtesy C-4 Optics

 

Just like the optics, the mechanical design has to be toleranced. This piece may need to be accurate within a few microns, this other piece may be ok if it’s within 0.1mm. Even things like thermal expansion of parts need to be considered. The same type of compromises has to be made here. This part may be more durable made out of this substance, but that will have worse thermal characteristics and cost more. It’s endless.

Not to mention, the mechanical designer has to let the optical designer know these things. For example, if the mechanical design can only place an element within X microns, but the optical formula will be altered if the element is moved by 1/2X microns, there’s a problem and we have to go back to square one. The optical designer might have to change his formula again, perhaps giving up a little sharpness or adding a slight aberration to obtain more consistency.

Assembly Design

The designers and the assemblers also need to give feedback to each other. Can the manufacturing department put all the pieces together accurately and quickly? What type of automated machinery does the assembly line have and how accurate is it? Using existing machinery is always going to be preferred over buying new types of equipment just to assemble one specific lens out of the dozens of types that must be assembled each year.

It isn’t surprising that when you view our teardowns that you see different brands of lenses tend to have a similar look inside. The mechanical designer would much rather reuse or simply resize parts from drawings and mechanisms that already exist instead of redesigning hundreds of pieces from scratch. The assembly line will be more efficient if the process is similar for many lenses, not different for each specific type of lens.

Assembly variations must be considered in the optical design, too. A lot of lenses contain groups of cemented elements, for example. How is the cementing done? What type of cement is used? How accurately will the group be cemented?

There are going to be QA checks within the manufacturing and assembly process.  We also assume, at least for better lenses, the individual glass elements are tested on an interferometer prior to coating because that’s fairly standard. There is, probably, a centering check done for lens groups after they are cemented together, or bonded within a plastic sleeve, both because that is simple and inexpensive, and because it’s a place where an error is likely to occur. The particular optical design may need certain elements to be shimmed or adjusted during assembly and techniques for that have to be set up.

How Things Go Wrong

All of this feedback, in theory, should result in a good lens that can be accurately manufactured with good copy-to-copy consistency. However, there are some pretty obvious problems that not only can occur, they probably do occur to some degree in every lens.

Feedback between the designers may be limited or not exist at all.

While this probably isn’t a big problem in large, dedicated lens manufacturers, it’s actually fairly common when things are outsourced. Sometimes the lens is optically designed by one firm, mechanically designed by another, assembled by a third, all for a fourth company who brands and markets the lens. Even when it’s all done in-house, there are deadlines for both time and money to be met; the designers might want 6 more months to tweak things but prototyping is going to start Thursday. In any case, there are certainly times where the optical designer ‘assumed’ a lens or group would be placed to a certain specification, while the mechanical designer or assemblers ‘assumed’ the optical designer knew it couldn’t, etc.

Feedback between manufacturing and design isn’t accurate.

This is probably more common than any of you imagine. I’ve learned the hard way manufacturers don’t always meet their promises, and we always have to remember that outsourced parts are usually provided by the lowest bidder. To a greater or lesser degree, nearly every manufacturer uses some outsourced parts. A few companies make almost everything in-house; others make almost nothing in-house.

Even when the part is made in-house, there are cost and time constraints. Is that tool getting a little worn and the manufactured tolerance increasing? Is a subcontractor having financial problems and using equipment past the time it should have been replaced? Quite commonly someone expected someone else would deliver to a specification they promised, but didn’t meet. I have watched this happen myself on a small scale, and been told of first-hand experiences on a large scale.

Workers are Workers

Joe on the assembly line might not be quite as worried about eventual quality control as he is about meeting his quota for the day. Or he may be hung over, not feeling well, or really pissed at his boss. No matter how good the design and tolerancing are, some lenses are going to be assembled poorly. Consider that dozens of people contribute to manufacturing and assembling every lens. It only takes one goofing off to mess up the lens or camera you just bought.

One guy goofing off is all it takes to ruin things for everyone.

Testing is Inadequate

We only know of two SLR lens manufacturers (Sigma and Zeiss) that state they optically bench test every fully-assembled lens (for at least some lines) after manufacturing. Sigma has a proprietary optical testing machine and Zeiss uses their own MTF benches. But even in those cases, we don’t know if that means they test every single lens they make or only certain lenses. We assume other manufacturers test to some degree after complete assembly, because autofocus and image stabilization, at least, have to be calibrated into the lens chip.

The key point, though, is that what type of testing is generally quiet and what standards they test to are pretty vague. Many manufacturers use very simple testing;  basically, some slanted lines printed by a laser printer or pojected on a screen, photographed with the lens mounted to a computer-tethered camera. For many lenses, this is all the optical testing that is done.

There are reasons for this (note I didn’t say good reasons). Testing isn’t done by an optical engineer, it’s done by a low-wage employee. Simple test charts can be shot with a camera tethered to a computer that gives a green (pass) or red (fail) output.

That is sufficient for programming autofocus, but it’s not nearly sufficient to determine if a lens is optically acceptable. This type of testing arguably was adequate back in film days but is not even close to adequate for a lens being used on a high-resolution digital sensor.

I know there are many among you screaming you’re absolutely certain this isn’t true, but I can assure you it is. I’ve had more than a few manufacturers tell me their factory tests showed no variance at all in their lenses. When we found out what their factory testing consisted of, it wasn’t even as sensitive as what a lot of you do at home. As I told one factory representative, the reason they call it an AF1951 chart is because it was developed in 1951.

Don’t get me wrong. Doing a 4-quadrant, 21-point MTF on every lens after assembly isn’t necessary for quality control. It can see things that just aren’t going to be seen in the real world. But something more than deciding if the bars on an ink-jet print are sharp enough to make the computer beep green probably is necessary. I’m 100% for automating testing because automated things are less expensive and more accurate. But it needs to be a meaningful test.

Compromises on Quality MUST be Made

All of the feedback loops I talked about exist between the people with different agendas. Perhaps the optical design and mechanical design departments both agree the lens should be a little less sharp in order to be assembled consistently. The marketing people may scream bloody murder because they want the best possible MTF charts to show in their brochures. The financial people have put a ceiling on what the costs of lens parts are going to be. In every case, with every lens, the final manufactured product is going to be a compromise. Even if it’s a $5,000 lens. Even if it’s a $20,000 lens. Just because you think no compromises should be made, doesn’t mean it can be so.

In some cases, lowering QA standards by just a bit makes a lens that, at least on paper, meets everyone’s needs. Here’s a simple example. What if a fairly large number of lenses are just barely failing the company’s current QA standards? Some are awful, of course, and must never leave the factory. But what about the ones that are just barely not OK? If we sold those 1,000 barely not OK lenses, probably 950 of them would stay sold and then we just write off those 50 that come back. That might be the ‘best’ solution. Costs restraints are still met, the MTF curves are still great, the designers get their bonuses.

This last illustration probably raised your blood pressure a bit. But it’s a reality. No group of lenses is perfectly identical, so the decision isn’t black-and-white. It’s shades of gray. And someone, somewhere, in the company has to decide what shade of gray is OK and what isn’t OK. And for many companies, at this stage, it’s all theoretical. The lens and mechanical designers have plugged their variation numbers back in their software and told the company what to expect. Even if it’s worse than expected, the company may, or may not, be doing testing that would detect it.

Don’t think the company doesn’t realize that only a few of their customers are pixel-peepers – the majority won’t notice one corner is slightly soft or that their copy isn’t quite as sharp as Joe’s copy. Also, they realize that internet noise is real. There are going to be complaints about the best lenses, there are going to be people who love the worst lenses. If the lens in question is just a little worse, the internet conversation is not going to seem much different than any of the others. 600 people saying it’s great and 400 people saying it’s sucks isn’t all the different from 600 people saying it sucks and 400 people saying those 600 are just bad photographers.

After Release 

There are some after-the-fact things a manufacturer can do to improve quality in an existing product. The manufacturers service center notes if a lot of lenses are returning for a particular problem, the company figures out why that problem occurs, and substitutes an improved part or assembly technique the next time that lens has a manufacturing run. They can replace a weak electronic or mechanical part pretty easily. Changing the optics, though, is much more difficult.

Two copies of the same lens, before and after a manufacturing change. In later versions (left) the manufacturer simply incorporated the electronics you see on the right into other boards. If you own one of these, from the outside you have no idea which type you have. In this case, BTW, it makes no difference they perform identically. It was probably a cost-saving change.  

 

(For those who don’t know, manufacturers make a given lens in batches. An assembly line is tooled for a given lens and makes X number of them. Then the line is broken down and set up to make a different lens. So Canon 35mm f/1.4 lenses, for example, may get manufactured every X number of months. I’ve heard as often as every 6 months or as rarely as once every couple of years, depending on the lens, but that’s just what I’ve heard.)

One of the biggest surprises in my journey through the lens world is how often this ‘after-release’ feedback loop doesn’t exist at all. I’ve talked to optical and optomechanical designers and told them their lens has this or that problem and found they were shocked. In one case, the mechanical designer had designed several different lenses with the same weak part over many years and had never been told the part failed frequently.

In general, a large company that does everything, including repair, in-house will have a strong after-release feedback loop. Canon, Nikon, Zeiss, and Sigma, for example, all get this information and do make changes. Small companies, especially those that outsource repairs, are less likely to have strong after-release feedback loops

Company attitude also varies quite a bit. We’ve all seen companies that when confronted with a very obvious problem on a new item issue a recall and fix the very obvious problem. We’ve seen others that when confronted with a very obvious problem simply repeat over-and-over, there is no problem.

Most of the time things are done pretty quietly. If the company identifies a problem occurring in some of their lenses, they may quietly fix it at no charge for all that are sent in. This often occurs along with a change in future lenses. For example, years ago a certain lens very commonly had a failure of its stabilizing unit. A year or so after release, we noticed that some copies sent in for repair for other reasons also had the stabilizing unit replaced at no charge. Not long after that, we noticed that new copies no longer had stabilizer failure. It’s now one of the most reliable lenses in our fleet.

For things like optical copy-to-copy variation, though, after-release fixes are less likely. Changing the optical formula isn’t an option; that would require a complete lens redesign. Changing the mechanical design isn’t much more likely but could happen. It’s possible, I suppose, that a certain lens element might be made to tighter tolerances, or that a group might be cemented more accurately by a new assembly technique.

So is Everything Doom and Gloom?

No, not in the big picture it’s not. In fact, I’m really quite positive about improving quality control and reduced sample variation going forward.

I know at least one manufacturer that recently realized their optical testing standards just weren’t adequate and they are spending money improving them. Testing is a back-burner, non-profit area that they just haven’t thought about recently. Once the QA problem was identified and got their attention, they addressed it.

Other manufacturers are looking to reduce variability by installing better, during-assembly optical tests of each component as the lens passes down the assembly line. New glass types, improved grinding techniques, better mechanicals and more accurate robotic assembly is both cost-effective and reduces variation.

A couple of manufacturer’s have quite obviously started improving copy-to-copy variation already. At least one of them has started to use this very quietly in their marketing. I really hope that more of them are going to see the advantage in publishing something like our variation number, or even just a real-world resolution number that they guarantee their lenses will meet. Some manufacturer, someday, will realize the advantage of not having people question how much their lenses vary. Once one of them does this, maybe optical quality and consistency will become just as important a marketing tool as computer generated, little-to-do-with-the-copy-you-get MTF charts are now.

I expect we will see a lot more copy-to-copy consistency and fewer problems in lenses released in the near future. I do NOT expect, though, that we’ll suddenly see an existing lens that has been known to have high variance suddenly have less. It could happen, but the changes required would make that very difficult.

 

 

Roger Cicala, Brandon Dube, and Aaron Closz

Lensrentals.com

February, 2016

 

 

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Equipment
  • Good job!

    I was an electronics test engineer for many years, and rather than take time to explain concepts like tolerances and MTBF, I just roll my eyes when someone says, “Why didn’t they test this?”

    Now, I can point people to your page, and then roll my eyes. 🙂

  • Michael Clark

    A link to this blog post needs to be added to the “My Lens Is Soft…” series!

  • ipdouglas

    As a (former) QA IT Quality engineer as well as a keen photographer I applaud your detailed discussion and description. Indeed were I still in that role myself I might use one or two of your descriptions to teach the value of Quality Assurance to the less enlightened.
    The only improvement I could ask for is that as one of the (very) few to know (backed by facts and measurements) that you at least publish which lenses have issues and what they are as you find them? It is notable that several ‘Class Actions’ (paricularly in USA) have had manufacturers finally admit issues when poor consumers like me have battled fruitlessly for months beforehand.
    Please tell us and help the small guy.

  • tadamis

    That’s as detailed a description of the ins and outs of manufacturing lenses and quality control as I’ve ever read… or could handle. Convincing. Thank you.

  • Dan Harper

    Excellent article and information. Lots to know and keep in mind. Thanks!

  • appliance5000

    I guess i would say this:
    Sounds like lens making is complex – that’s the business they’re in. Most have been in it for a long time.
    With advances in cad, with institutional knowledge of the base level performance a particular lens configuration (how many variations are there – not many), advances in materials tech,prototyping, and savings in manufacturing due to digital processes, the manufacturers are saving money all the way down the line.
    I think, on most lenses, the profit margins are more in the 300% than 100% level. Just a guess, but Canon lowered all their prices fairly significantly and I’m sure they’re still keeping the investors happy.
    I think that it is not overly demanding to ask that the lens you buy is centered and operates at a consistent level of accuracy and precision. One may be slightly sharper than another in a batch, but they should meet the specs you paid for. That $20 – $50 per lens cost for testing is easily absorbed in $800 – $3000.00 +lenses.
    If discarding a poor lens is upsetting, that could inspire more QC. Certainly it would be less stressful to sony corp, than the poor guy who just lost half his vacation pics.

  • TCav

    There’s a misunderstanding here.

    Quality Assurance is testing the final product before it ships, to assure that it meets specifications.

    Quality Control is controlling the process, from design and raw materials, through manufacturer and assembly, to make sure that a quality product is probable.
    They’re both important, but while quality control is a lot more work, it pays off better in the long run.

    As is often the case, however, people often use the two terms interchangeably, which is incorrect.

  • Kevin Purcell

    This is the critical point: “QA isn’t just there to make sure individual items are produced to within designed tolerances, they’re there to make sure the procedure produces items produced to within designed tolerances”

    The QA org is there to improve processes that generate product of a known quality. This applies to the low wage guy doing verification testing, through the production QA engineers (who design the testing procedures and figure out what’s going wrong with the production process) to design QA engineers (who work with the optical and mechanical designers to determine where the issues will be) to the managers (who decide what process are to be applied across the whole org). All have a role in fixing the process (and so fixing the product).

    The saying is “You can’t test quality into a product” but a lot of the Internet forum posters seem to think that’s exactly what you can do.

    Large chunks of the British and American motor industry used to think this too.

  • Mel Gross

    One of the things not mentioned in this article is the large degree the camera body itself plays in this QC problem. So if the back focus of the lens is perfect, but that of the body is not, it will still be off. If another lens has a back focus problem that cancels out that of the body, it will seem perfect.

    So the good lens may get sent back, and the “bad” lens kept.

    We see the same thing with side to side focus problems. Is is all the fault of the lens, all the fault of the body, or some of both. Is a body that’s out of parallel, out in the opposite direction, or the same direction? I’ve seen it in the up and down direction as well as the right to left.

    These problems make evaluating lenses we buy more difficult. We may not see a problem on a slower lens, and think it’s fine. It may have all of these problems, but depth of field wide open covers it. A better lens may have more of a problem because it’s faster. So the slower lens is better, and the faster lens is worse. Problems, problems.

  • Mel Gross

    Some companies are, and some aren’t.

  • Max Rottersman

    Most people understand that when one measures the width of one’s foot in inches, it’s easy enough to come up with an acceptable answer. What they don’t understand is the more precision you go for, the more you’re likely to come up with a measurement that can’t be repeated. I can measure my foot and say 12 inches and come up with that measurement all day long. If I measure it to 6 decimals, at 12.369102 inches, mostly likely I will NOT get that exact measurement the next time. It might be 12.369201. You point this out, of course, but where people might be confused is in thinking better equipment will get a perfect 12.369201 type number. At those levels, there is never perfect, there is only TIME, as you point out, which is used to re-sample the measurement to arrive at a mean that is hopefully closer to the “truth.” Time is money. It doesn’t matter whether you have an engineer or high-school drop-out running the machine, it will cost you more money (time) to re-sample every measurement to get at high tolerances. (measurements you need for both building and testing) And, as you point out, you need to do those in the range of situations the equipment will find itself in, from cold to warm, etc. Even if the machine were free, that time would add to the cost of the lens.

    I understand your frustration, “If you ever say, “for that kind of money I expect perfect” please don’t read further” asks the question, is the worst modern lens today better than the best manual lens made 50 years ago? If it is, then a person shouldn’t compare lenses that today can be built and analyzed using high-precision equipment, against their idea of “quality” that is sentimental and nostalgic. Today’s lenses compete against themselves (the limits of current measurement in manufacturing and testing), and the value they give, compared to yesterday, to me, is something to celebrate!

    One last thing, many people who return “bad copies” are probably wrong, IMHO. They didn’t do enough testing to confirm that they aren’t trading in one set of optical problems for another. They create doubt among photographers that is impossible for the manufacturers to combat.

    Thanks for a great article!!!!!

  • Douglas Watt

    A truly superb review by Roger (not surprising as he generally does everything competently). Underscoring just how complicated it is to design and then manufacture a really good lens.

  • David Bo Hansen Cartagena

    7 years ago I recieved a brand new Sony Zeiss 24-70 f/2.8 for a-mount. AF didn’t work and after 3 weeks repair I got it back. But it was never really sharp from 50-70 at one side of the frame even at f/8 or f/11. So must have been assembled on a monday 😉 Anyway I sold it to another and told him about my issues. My Sony 70-200G was decentered and repaired so bad that they had to repair it again. They simply destroyed some of the outer screws on the lens.

    Second time it came back working perfect and centered. I also had an old Minolta 24-105 where the AF locked within a year. So out of 5 lenses I own or owned. 3 have been faulty. One after a year and two already at arrival.

    Not a good commercial for SONY or in case of one lens Minolta.

  • Richard Clapp

    Thanks for your info. I was commenting on the info in the article that some lens QC uses analysis of ink jet printed charts and the added variability this will introduce in a final “answer”.

  • Jeff Forbes

    For what it’s worth, it is still in the manufacturer’s best interest that you’re satisfied with your purchase. The other thing I think that would disagree with that notion is that sorting in that manner would add even more overhead. I would be more inclined to buy from a bigger company with better support, though!

  • Jonathan Isaac

    Is that lens that had stabilizer issues the Canon 100-400? I sent mine in for a repair (broke the tightener ring) and it came back with the AF and IS module replaced for no apparent reason. I always wondered why it got extra service procedures.

  • mijami

    Want extremely high precision optical flats? then Australia’s CSIRO can make them for you. Clients include Canon, Nikon and NASA. http://www.acpo.csiro.au/

  • jay thomas

    Actually this is applicable to any mass manufactured product in general. Electronic, automobile. Since variations are much obvious in the case of lenses it became a hot topic for discussion. How about image sensor?. And other critical camera modules.?..
    One other thing is until recently no photographer was looking for surgical precision for their lenses..as long as the images are okay for human eyes…now with pixel peeping everyone preoccupied with how the light ray is bend in their specific copy or how the pixels are processing the photons…. reality is that too much QA is redundant or over engineering my not be worth as we anticipate, for a everyday photographer… also an image is combination of many variables including lense precision….a lenses showing varying degrees of sharpness at different settings says,, optical precision at all times is not the only factor for a better picture or not a realistic expectation

  • KSharp

    I think the only thing I’d add would be something like QA isn’t just there to make sure individual items are produced to within designed tolerances, they’re there to make sure the procedure produces items produced to within designed tolerances. My best friend from the uni works at a company that makes semiconductor wafers, and he has this discussion every day both up and down his chain.

  • Wild Biker Bill

    All ‘grey market’ means is that the product was meant for a different marketing/sales zone from the one it was actually sold in: Japan vs. USA vs. Europe for example.

  • iamipa, I’ve never heard of that at all, from anyone.

  • Johnny Alamo

    Years ago, my original American-bought Pentax AF EDIF 600/f4 gave me great, sharp images from f4 through f11 at any focal distance. My friend’s similar Pentax 600f4 lens, procured through Europe, was not sharp until f8-f11, and he was not a happy camper. He eventually switched to Canon, who could blame him… I had lenses from 20mm thru 600mm with variants in what this article would call quality control, and I always understood the differences called sample variation. I did not understand the discrepancies well until reading this article, so thanks to the author. It just means the buyer has to be the final quality control agent, and use the lens/camera/P&S a lot during the return period, so as to get rid of a lemon or even a near-lemon and reach back into the bag to obtain and test another sample while the cost of doing so is only shipping and insurance back to the seller.

  • iamipa

    At the risk of opening a can or worms, do manufacturers of lenses (and cameras!) release product that doesn’t meet their usual standards onto the grey market?

  • Brandon Dube

    The Trioptics ImageMaster does not use any printed media to delivery test results.

  • mauriceg

    One wonders whether some of the borderline acceptable lenses would get sent to locations in the world where there is no hope of money back or replacement. i live in Bali, Indonesia, and would never purchase a lens or camera (or again, just about anything portable), because the idea of consumer rights doesn’t exist.

  • Richard Clapp

    I have worked as an manager (QC, prod dev, manf) in the paper industry and have developed ink jet papers. There is much variation in ink jet paper as well as the inks and printers that are used. QC results from analyzing ink jet prints are also a significant source of variation.

  • Donald Van Patten

    I must admit buying a Sony Zeiss lens lately is a real crap shoot. De-centered lenses seem to be their forte.

    But I guess we are reaching the limit MP’s wise of what 35mm lenses can be fairly good, let alone excellent.

  • Wilson Laidlaw

    The quality control at one of the world’s most expensive lens makers, seems non-existent in recent years. Three out of my five most recent new Leica lenses had to be sent straight back to the factory. It took me only a few seconds with the crude device of a rangefinder camera with an EVF to detect a problem. At one point, I questioned whether Mr. Magoo was alive and well and working as a lens checker in Wetzlar. I dread to think what differences a more detailed MTF assessment on an optical bench would have showed up. It can rarely work in the other direction, I have a 35mm/f1,4 ASPH Summilux, a lens notorious for aperture shift but I am lucky enough to have one that hardly aperture shifts at all.

  • Brandon Dube
  • FelipeGR

    That’s some pretty serious stuff.

    I agree on serviced items skewing the data, but, you could compare serviced vs unserviced, serviced vs out-of-the-box-new, etc. Could yield some interesting results.

    Definitely a tremendous endeavor never the less. Best of luck!

Follow on Feedly