Equipment

Optical Quality Assurance

Published February 5, 2016

 

Pinhole images from new, but optically decentered lens. Olaf Optical Testing, 2016

 

After we published a number of posts about copy-to-copy variation, people were quick to say that this company or that needs to ‘just’ improve quality control. I totally agree but realize most people don’t have a clue what ‘improving quality control’ would really look like. I think they have some vague idea of hiring a guy named Joe to sit at the end of the assembly line, check all the lenses, and reject all the bad ones. Optical quality control for lenses is way, way more complicated than that.

Before we start, let’s get a couple of things out of the way.

  1.  If you expect that the lens you buy has the exact same MTF that the manufacturer published, then don’t read this. You’ve got too much reeducation to do.
  2.  If you ever say, “for that kind of money I expect perfect” please don’t read further. Perfect doesn’t exist. For $20,000 and up you can get pretty close. For $2,000 you should be able to get reasonably close.

On the other hand, a reasonably expensive lens should have reasonable quality control and minimal copy-to-copy variation.  We know it can be done; some manufacturers are already doing it. Others totally suck. But when they suck, sticking Joe at the end of the assembly line won’t really fix things. If you want to read this pretty long article, I’ll explain why, and also explain some of what can be done.

A Few Basics First

To Some Degree, Consistency Costs Money.

Here’s a quick, relatively cheap example. I need to buy a fairly large, optically flat piece of glass to calibrate our MTF bench. But how accurate of an optical flat do I need? A flat accurate to 1/4 wavelength of visible light should be flat enough, right? That costs $450. But 1/20 wavelength is more accurate. That one costs $825. Is it worth the difference? Well, it depends on what you’re doing with it. For my purposes, 1/4 wavelength is plenty good.

 

Please don’t respond with, ‘well there will be discounts for quantity’. Sure there are, but better still costs more. Want a million 1mm screws toleranced to be between 0.96 and 1.04mm? They’re really cheap. Want them to be between 0.99 and 1.01mm? The price is going up. It’s cheaper per screw to buy a million, but the tighter tolerance is still more expensive.

Optical Testing May Be Cheap. Or Not.

People love to say testing won’t add much cost to a lens.  ‘Not much’ is a pretty variable term, though. A state-of-the-art MTF bench costs between $250,000 and $500,000. And it takes the time to test a lens. A single test run across the lens takes a minute or two. We need at least two angles (4 would be better) for each lens; plus, mounting, unmounting, and other things take time. So we can test a lens in 5 minutes; 12 lenses an hour. Let’s work 12 hour days and we can test 144 lenses a day. Unless it’s a zoom lens. Then you must test each lens at multiple focal lengths (3 or 4) so things down to maybe 3 or 4 lenses an hour.

 

If your assembly line is churning out 5,000 lenses a week you will need a lot of benches. Any way you slice it, the equipment and the operators to run it are going to add something to the price of the lens. I think a company being really efficient could do it for an additional $20 per prime lens, $60 per zoom. (I couldn’t break even at that cost, but I’m assuming they have economies of scale that I don’t have.)

 

Of course, if you want the best, state of the art testing, like interferometry, things could get really pricey. Interferometry requires different reference elements for each type of lens, and reference elements alone cost $25,000 to $50,000 each.

You can set up an Imatest-type program or even high-quality optical targets on a rail system to keep things properly square and get good results, especially if the images are computer analyzed.  That’s still very good and certainly less expensive. So there are lots of good ways to check optical performance to a high level, but they are going to cost a bit of money.

 

 

Or, you can make some ink-jet printed, thick-bar test charts, hook a camera to a computer, and let the computer say ‘pass/fail’ and poorly test a lot of lenses for almost no cost at all. This is actually what many manufacturers do. It’s quick, cheap, and has the added advantage of setting the failure bar really low so there aren’t many rejections. They have to do something like this anyway, in order to set autofocus and back focus for most lenses.

Some manufacturers do have high-level test equipment and test every lens. More often, they just do this type of testing on a percentage of lenses. (BTW – this sure doesn’t mean that type of equipment is available in your country’s repair center; it may only exist at the corporate manufacturing facility.) Many manufacturers do have some type of testing other than the ‘computer takes images of a few thick, ink-jet bars’. Then again, many do not.

But the Most Expensive Thing is Failing

But if you’re thinking, “OK, so they all need to have better testing and that doesn’t cost too much,” well, no there’s a lot more to it that that. The worst thing that can happen from the manufacturer’s standpoint is for a lens to be completely assembled and then fail quality control at the end of the assembly line. Let’s say that lens you pay $1,000 for costs $500 to manufacture and assemble and then fails a quality check.

What do you do? Throw it in the trash and take a loss? That’s actually fairly likely if it’s a consumer-grade lens. Disassembling and testing each component to salvage the good parts may be more expensive than throwing it away. You could just disassemble it, reuse all the parts without testing them, and hope it was just an assembly error and not a bad part. (This is pretty common among small manufacturers, and actually works pretty well.) But it still means the assembly cost of the lens is more than doubled. Now we assembled, disassembled, and reassembled and still have just one lens to sell.

Whatever you do, you will have to account for the cost of failed lenses. If you have to manufacture 1,100 lenses to sell 1,000, then each of those 1,000 lenses has to include the cost of those 100 failures. If you had to disassemble the 100 but were able to reuse the parts that still costs money and the cost has to be accounted for. One way or another, failure is always expensive. So good quality control needs to be done before and during lens assembly, not just afterwards.

And, just in case you haven’t thought about it everyone, from the owner of the company to the manager of the assembly line, at some point has to think, “well, just how bad did those 100 lenses fail? Are we maybe being too picky?”

Optical Quality Assurance is a Series of Feedback Loops

Design Feedback Loops

You may not have thought of it, but the biggest part of quality assurance probably occurs during design and prototyping.

Optical Design

The lens designer is given a set of goals and constraints; price, size, necessary features, release date, etc. The designer usually tries to avoid really expensive glass types and extreme lens shapes; these can cost 10X to 30X more than common glass.

Prototype elements for a future lens. Image courtesy C-4 Optics.

 

After the initial design is completed there are a series of redesigns to refine the lens. A lot of this refinement has to do with ‘tolerancing’. Each element in the lens is evaluated given the known accuracy to which it can be made, to see what effect that variation will have on actual performance of the assembled lens.  The lens may have to be redesigned several times so that inevitable variation in glass element production doesn’t degrade performance too severely.

There are tradeoffs for every change. We may get better consistency by changing this, but it might cause a slight aberration or increase the cost significantly. Other people are sitting in these meetings along with the optical designers. The marketing guys may want the best MTF curves possible, even if that may mean the lens is going to have more variance. The financial people may veto a more expensive glass. Different companies may give different emphasis to different things. The same company may give different emphasis with different lenses.

Mechanical Design

The mechanical design of the lens is usually far more complex than the optical design, and the same steps have to be made during this design. How accurately can each element be placed in the lens barrel? How much variation will occur during focusing or zooming? The optical designer will have identified if there are critical elements that may need to be adjusted in the lens so that the mechanical designer can add shims, eccentric collars, centering rings, or other forms of adjustment to his design.

The complexity here has to be seen to be believed. I’m working with another company designing some simple, manual-focus prime lenses. The designs for the mechanical parts run well over a hundred printed pages. That’s just mechanical parts; no autofocus, no zoom mechanisms, no image stabilization, and no optics. Every single part has to be sourced to certain specifications.

Image Courtesy C-4 Optics

 

Just like the optics, the mechanical design has to be toleranced. This piece may need to be accurate within a few microns, this other piece may be ok if it’s within 0.1mm. Even things like thermal expansion of parts need to be considered. The same type of compromises has to be made here. This part may be more durable made out of this substance, but that will have worse thermal characteristics and cost more. It’s endless.

Not to mention, the mechanical designer has to let the optical designer know these things. For example, if the mechanical design can only place an element within X microns, but the optical formula will be altered if the element is moved by 1/2X microns, there’s a problem and we have to go back to square one. The optical designer might have to change his formula again, perhaps giving up a little sharpness or adding a slight aberration to obtain more consistency.

Assembly Design

The designers and the assemblers also need to give feedback to each other. Can the manufacturing department put all the pieces together accurately and quickly? What type of automated machinery does the assembly line have and how accurate is it? Using existing machinery is always going to be preferred over buying new types of equipment just to assemble one specific lens out of the dozens of types that must be assembled each year.

It isn’t surprising that when you view our teardowns that you see different brands of lenses tend to have a similar look inside. The mechanical designer would much rather reuse or simply resize parts from drawings and mechanisms that already exist instead of redesigning hundreds of pieces from scratch. The assembly line will be more efficient if the process is similar for many lenses, not different for each specific type of lens.

Assembly variations must be considered in the optical design, too. A lot of lenses contain groups of cemented elements, for example. How is the cementing done? What type of cement is used? How accurately will the group be cemented?

There are going to be QA checks within the manufacturing and assembly process.  We also assume, at least for better lenses, the individual glass elements are tested on an interferometer prior to coating because that’s fairly standard. There is, probably, a centering check done for lens groups after they are cemented together, or bonded within a plastic sleeve, both because that is simple and inexpensive, and because it’s a place where an error is likely to occur. The particular optical design may need certain elements to be shimmed or adjusted during assembly and techniques for that have to be set up.

How Things Go Wrong

All of this feedback, in theory, should result in a good lens that can be accurately manufactured with good copy-to-copy consistency. However, there are some pretty obvious problems that not only can occur, they probably do occur to some degree in every lens.

Feedback between the designers may be limited or not exist at all.

While this probably isn’t a big problem in large, dedicated lens manufacturers, it’s actually fairly common when things are outsourced. Sometimes the lens is optically designed by one firm, mechanically designed by another, assembled by a third, all for a fourth company who brands and markets the lens. Even when it’s all done in-house, there are deadlines for both time and money to be met; the designers might want 6 more months to tweak things but prototyping is going to start Thursday. In any case, there are certainly times where the optical designer ‘assumed’ a lens or group would be placed to a certain specification, while the mechanical designer or assemblers ‘assumed’ the optical designer knew it couldn’t, etc.

Feedback between manufacturing and design isn’t accurate.

This is probably more common than any of you imagine. I’ve learned the hard way manufacturers don’t always meet their promises, and we always have to remember that outsourced parts are usually provided by the lowest bidder. To a greater or lesser degree, nearly every manufacturer uses some outsourced parts. A few companies make almost everything in-house; others make almost nothing in-house.

Even when the part is made in-house, there are cost and time constraints. Is that tool getting a little worn and the manufactured tolerance increasing? Is a subcontractor having financial problems and using equipment past the time it should have been replaced? Quite commonly someone expected someone else would deliver to a specification they promised, but didn’t meet. I have watched this happen myself on a small scale, and been told of first-hand experiences on a large scale.

Workers are Workers

Joe on the assembly line might not be quite as worried about eventual quality control as he is about meeting his quota for the day. Or he may be hung over, not feeling well, or really pissed at his boss. No matter how good the design and tolerancing are, some lenses are going to be assembled poorly. Consider that dozens of people contribute to manufacturing and assembling every lens. It only takes one goofing off to mess up the lens or camera you just bought.

One guy goofing off is all it takes to ruin things for everyone.

Testing is Inadequate

We only know of two SLR lens manufacturers (Sigma and Zeiss) that state they optically bench test every fully-assembled lens (for at least some lines) after manufacturing. Sigma has a proprietary optical testing machine and Zeiss uses their own MTF benches. But even in those cases, we don’t know if that means they test every single lens they make or only certain lenses. We assume other manufacturers test to some degree after complete assembly, because autofocus and image stabilization, at least, have to be calibrated into the lens chip.

The key point, though, is that what type of testing is generally quiet and what standards they test to are pretty vague. Many manufacturers use very simple testing;  basically, some slanted lines printed by a laser printer or pojected on a screen, photographed with the lens mounted to a computer-tethered camera. For many lenses, this is all the optical testing that is done.

There are reasons for this (note I didn’t say good reasons). Testing isn’t done by an optical engineer, it’s done by a low-wage employee. Simple test charts can be shot with a camera tethered to a computer that gives a green (pass) or red (fail) output.

That is sufficient for programming autofocus, but it’s not nearly sufficient to determine if a lens is optically acceptable. This type of testing arguably was adequate back in film days but is not even close to adequate for a lens being used on a high-resolution digital sensor.

I know there are many among you screaming you’re absolutely certain this isn’t true, but I can assure you it is. I’ve had more than a few manufacturers tell me their factory tests showed no variance at all in their lenses. When we found out what their factory testing consisted of, it wasn’t even as sensitive as what a lot of you do at home. As I told one factory representative, the reason they call it an AF1951 chart is because it was developed in 1951.

Don’t get me wrong. Doing a 4-quadrant, 21-point MTF on every lens after assembly isn’t necessary for quality control. It can see things that just aren’t going to be seen in the real world. But something more than deciding if the bars on an ink-jet print are sharp enough to make the computer beep green probably is necessary. I’m 100% for automating testing because automated things are less expensive and more accurate. But it needs to be a meaningful test.

Compromises on Quality MUST be Made

All of the feedback loops I talked about exist between the people with different agendas. Perhaps the optical design and mechanical design departments both agree the lens should be a little less sharp in order to be assembled consistently. The marketing people may scream bloody murder because they want the best possible MTF charts to show in their brochures. The financial people have put a ceiling on what the costs of lens parts are going to be. In every case, with every lens, the final manufactured product is going to be a compromise. Even if it’s a $5,000 lens. Even if it’s a $20,000 lens. Just because you think no compromises should be made, doesn’t mean it can be so.

In some cases, lowering QA standards by just a bit makes a lens that, at least on paper, meets everyone’s needs. Here’s a simple example. What if a fairly large number of lenses are just barely failing the company’s current QA standards? Some are awful, of course, and must never leave the factory. But what about the ones that are just barely not OK? If we sold those 1,000 barely not OK lenses, probably 950 of them would stay sold and then we just write off those 50 that come back. That might be the ‘best’ solution. Costs restraints are still met, the MTF curves are still great, the designers get their bonuses.

This last illustration probably raised your blood pressure a bit. But it’s a reality. No group of lenses is perfectly identical, so the decision isn’t black-and-white. It’s shades of gray. And someone, somewhere, in the company has to decide what shade of gray is OK and what isn’t OK. And for many companies, at this stage, it’s all theoretical. The lens and mechanical designers have plugged their variation numbers back in their software and told the company what to expect. Even if it’s worse than expected, the company may, or may not, be doing testing that would detect it.

Don’t think the company doesn’t realize that only a few of their customers are pixel-peepers – the majority won’t notice one corner is slightly soft or that their copy isn’t quite as sharp as Joe’s copy. Also, they realize that internet noise is real. There are going to be complaints about the best lenses, there are going to be people who love the worst lenses. If the lens in question is just a little worse, the internet conversation is not going to seem much different than any of the others. 600 people saying it’s great and 400 people saying it’s sucks isn’t all the different from 600 people saying it sucks and 400 people saying those 600 are just bad photographers.

After Release 

There are some after-the-fact things a manufacturer can do to improve quality in an existing product. The manufacturers service center notes if a lot of lenses are returning for a particular problem, the company figures out why that problem occurs, and substitutes an improved part or assembly technique the next time that lens has a manufacturing run. They can replace a weak electronic or mechanical part pretty easily. Changing the optics, though, is much more difficult.

Two copies of the same lens, before and after a manufacturing change. In later versions (left) the manufacturer simply incorporated the electronics you see on the right into other boards. If you own one of these, from the outside you have no idea which type you have. In this case, BTW, it makes no difference they perform identically. It was probably a cost-saving change.  

 

(For those who don’t know, manufacturers make a given lens in batches. An assembly line is tooled for a given lens and makes X number of them. Then the line is broken down and set up to make a different lens. So Canon 35mm f/1.4 lenses, for example, may get manufactured every X number of months. I’ve heard as often as every 6 months or as rarely as once every couple of years, depending on the lens, but that’s just what I’ve heard.)

One of the biggest surprises in my journey through the lens world is how often this ‘after-release’ feedback loop doesn’t exist at all. I’ve talked to optical and optomechanical designers and told them their lens has this or that problem and found they were shocked. In one case, the mechanical designer had designed several different lenses with the same weak part over many years and had never been told the part failed frequently.

In general, a large company that does everything, including repair, in-house will have a strong after-release feedback loop. Canon, Nikon, Zeiss, and Sigma, for example, all get this information and do make changes. Small companies, especially those that outsource repairs, are less likely to have strong after-release feedback loops

Company attitude also varies quite a bit. We’ve all seen companies that when confronted with a very obvious problem on a new item issue a recall and fix the very obvious problem. We’ve seen others that when confronted with a very obvious problem simply repeat over-and-over, there is no problem.

Most of the time things are done pretty quietly. If the company identifies a problem occurring in some of their lenses, they may quietly fix it at no charge for all that are sent in. This often occurs along with a change in future lenses. For example, years ago a certain lens very commonly had a failure of its stabilizing unit. A year or so after release, we noticed that some copies sent in for repair for other reasons also had the stabilizing unit replaced at no charge. Not long after that, we noticed that new copies no longer had stabilizer failure. It’s now one of the most reliable lenses in our fleet.

For things like optical copy-to-copy variation, though, after-release fixes are less likely. Changing the optical formula isn’t an option; that would require a complete lens redesign. Changing the mechanical design isn’t much more likely but could happen. It’s possible, I suppose, that a certain lens element might be made to tighter tolerances, or that a group might be cemented more accurately by a new assembly technique.

So is Everything Doom and Gloom?

No, not in the big picture it’s not. In fact, I’m really quite positive about improving quality control and reduced sample variation going forward.

I know at least one manufacturer that recently realized their optical testing standards just weren’t adequate and they are spending money improving them. Testing is a back-burner, non-profit area that they just haven’t thought about recently. Once the QA problem was identified and got their attention, they addressed it.

Other manufacturers are looking to reduce variability by installing better, during-assembly optical tests of each component as the lens passes down the assembly line. New glass types, improved grinding techniques, better mechanicals and more accurate robotic assembly is both cost-effective and reduces variation.

A couple of manufacturer’s have quite obviously started improving copy-to-copy variation already. At least one of them has started to use this very quietly in their marketing. I really hope that more of them are going to see the advantage in publishing something like our variation number, or even just a real-world resolution number that they guarantee their lenses will meet. Some manufacturer, someday, will realize the advantage of not having people question how much their lenses vary. Once one of them does this, maybe optical quality and consistency will become just as important a marketing tool as computer generated, little-to-do-with-the-copy-you-get MTF charts are now.

I expect we will see a lot more copy-to-copy consistency and fewer problems in lenses released in the near future. I do NOT expect, though, that we’ll suddenly see an existing lens that has been known to have high variance suddenly have less. It could happen, but the changes required would make that very difficult.

 

 

Roger Cicala, Brandon Dube, and Aaron Closz

Lensrentals.com

February, 2016

 

 

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Equipment
  • RLThomas

    First, thanks again Roger, Brandon and Aaron for the wonderful article on Optical QA!
    Over the years, you’ve made a huge contribution to my understanding of how things really work and as a result have helped me get far more enjoyment from my photography. For that small niche of photographers like me who want to get the most from their gear, your knowledge and perspective have been invaluable.

    For me, the Holy Grail of perfection is not finding “The” perfect lens but rather finding a retailer/manufacturer that is willing to provide me with sufficient information so that I can make an informed decision on the trade-offs I’m willing to accept on a particular piece of equipment – especially when I’m spending thousands of dollars to buy it. Providing theoretical MTF curves and marketing hype do little to satisfy that desire. I would gladly pay an extra $100-200 for the confidence of knowing that the particular lens I’m buying isn’t de-centered and what its ACTUAL strengths and weaknesses are based on meaningful optical testing performed on that lens. Copies will vary. I get that. There will always be trade-offs, I just want to be in the position of choosing the trade-offs I’m accepting.

    It strikes me that LensAuthority is in a great position to provide the kind of purchasing information I would like to have. Given that LR often has dozens of copies of the most popular lenses, it would be great to know when I’m buying one where it stacks up relative to its peers and how that particular lens copy performs optically. I would be happy to pay extra for that kind of information. How about it Roger? Could LensAuthority offer a run through OLAF with printed MTF curves on lenses I purchase from them?

    Randy

  • Zuikocron

    Great article!

    Always love your blog entries – too bad Fuji is such a small manufacturer, I’d love to hear/see what you think about their lenses (optically and copy-to-copy consistency)

  • Jon

    You’re right 🙁 It sure doesn’t feel like it when you pony up for that red ring on the front end 🙂 but i see your point. We have optics for our laser interferometer at work that cost more than L series glass for a single element. I guess it’s true: everything is relative.

    One thing I do wish though is that you could pay a fee to have the entire lens built with extra attention to overall performance (mechanical and optical). I know our products will fall within a certain statistical performance spread if the normal assembly process is followed. But if extra measurements and alignments are done mid-assembly, the same parts can be assembled to perform to their absolute best which is in that top 3-5% range. I would pay an extra 5% for a test report showing the lens has been optically calibrated to the best possible.

  • Brandon Dube

    To add, here is the same Zeiss 15mm f/2.8 tested in July and tested again in early December:
    http://puu.sh/mZsDI/d7e6cd33e3.png
    http://puu.sh/mZsGw/ab68867ae0.png
    There is essentially 0 change.

  • Brandon Dube

    Consumer lenses aren’t precision optics, they are merely ordinary optics 🙂 Precision commands 10x the price!

  • I rented a high-end 50/1.4 from LR a few weeks ago, which turned out to be a good copy. That convinced me I wanted this lens, as it was clearly superior optically from my current 50/1.4. After 1 new and three used copies, the 5th copy of this lens, a used one, was as good as the LR copy I first used, both mechanically and optically.

    Thanks for this article. On the razor’s edge of a high-end lens it illuminates why it took me 5 attempts.

  • Albert

    Great article, and very enlightening. I cannot say whether or not companies would actually use consistency numbers in their marketing, since doing so would admit there is variability in the sample quality. You might think this would be a gimme, but most I know don’t really look that closely unless it is blatantly bad, and with good reason: how do you test?

    I bought a Tamron 35 f/1.8 and actually used your numbers as part of my decision process. I live abroad and the good average numbers, which were good to me, combined with the low variability, meant I had a better chance of not being stuck with a dud, where exchanges would be a serious hassle. I have now received it (and love it as I cannot believe how sharp it is wide open almost edge-to-edge), but could not tell you ever whether it is better than average or worse than average. How could I? It would mean having someone else with the same camera, same lens, and same conditions to shoot in to properly compare. Not to mention the disposition to go through all this.

    That said, as I myself used your variability numbers, you might consider publishing those of more lenses, not just the highest end ones. I know that next on my list is most likely a Tamron 70-200 f/2.8 VC, which is a lens that is well viewed overall, but you forewent on testing, only posting Canikon numbers.

  • We do this to a degree, but all we’ve learned is that sometimes lenses get dropped or jarred (either by shipping or cusgomers) and knocked out of alignment. There’s not a gradual ‘as it ages’ change. One day some lens comes back markedly different. Or not.

  • I hadn’t considered robot adjustment, but the eccentric screw adjustments are much easier to do in ‘real time’.

  • Jon G

    Came here to say I am a ME and work in product design and can confirm this is true. There are many more than 14bit shades of grey when it comes to decisions. I think the best you can hope for from a manufacturer is transparency and honesty. To me, quality control is paramount. It shows competency and capability and commands respect. When I spend my hard earned dollars in precision optics, I want to know with certainty I am getting at least what I’m paying for. Satisfaction is expectation minus reality.

  • Ken Elliott

    I’d be interested in seeing variability over the life of a lens. LR is in a unique position to see what effect shipping and usage has on lenses. I propose you take a number of lenses and test them every X rent/ship cycles.

    As part of an R&D group, we toured one of the major hubs for one of the worlds largest shipping companies. I saw a belt line moving packages at 45 MPH, and a “kicker” knocking the box off the belt, onto a slide that dropped about 15 feet to the back of the delivery truck. The “kicker” had to have enough force to push large packages onto the slide within reason, but small packages (typical lens package) were kicked into the air, and fell onto the slide. I realized this could account for some sample variation of lenses.

    If you tracked some of the lenses you ship, I believe you would find some lenses are more prone to such damage than others. Is this something of interest to you?

  • I’m completely convinced, tnx for the reply. 🙂

  • Brandon Dube

    The tooling and manufacturing procedures for lenses does not change across their lifetime unless there is a massive issue, the cost of making the tools is simply too high. Batch-to-batch variance, optically, does not exist.

    (If you need convincing, Roger has published a handful of blog posts in the past with very large sample sizes – 100+ – of a couple lenses).

  • Maya

    Another interesting article ! Thanks a lot.
    Canon’s been talking about fully automated lens production since a while ago (I remember reading articles about more comprehensive automated lens production back in the late 2000s if I recall correctly), but it seems that it’s only since the last few years that they might have achieved their goal. Since you disassemble a lot of lenses, have you noticed in their internal construction differences that would be indicative of a fully robotic production ? One thing I believe you frequently mention is that Canon tends to use screws for adjusting elements, and less so spacers or shims – would that be, at least to some extent, because it’s easier for a robot to turn a screw instead of removing an element, changing the spacer / shim and repositioning the element ? This is just a wild guess on my part, so if I’m being stupid, don’t hesitate to correct my assumptions.

  • Zaphod

    When you pull a lens apart, can you usually tell what it is that is causing lens variation? If yes, any chance you could do a teardown of a Zony 16-70 f/4? 🙂

  • Great article as always, but it’s made me wonder again about something that crossed my mind when you introduced the consistency number.
    When you get a new lens (e.g. 50mm 1.8 STM) all 10 lenses that you test certainly came from the same batch, and maybe were even produced by the same people on the same day at the factory.
    On the other hand, when you test an older lens, could it be that some samples were produced years before or after others? Maybe even in a different factory with different assembly process?
    Do you think that your system could be biased towards new(er) lenses?

  • Lee

    My blood pressure isn’t raised by the example of selling the lenses that are alllllmost okay knowing most people won’t notice. Because, indeed, most people won’t notice. “Good enough” is different for different people. And yeah, it’s gonna be expensive to have a bench at the end of every assembly line.

    BUT, you need to take care of the people who DO notice. Don’t have a bench at the end of every assembly line, fine. But for damn sure have one at the repair facility. If I catch you pulling the “eh most people won’t notice” on me, you better know how to adjust (or be willing to replace) mine. Don’t just tell me I dropped it or lie to me about its grey market status. THAT’S what gets my blood pressure up.

  • DerekL

    ” In every case, with every lens, the final manufactured product is going to be a compromise. Even if it’s a $5,000 lens. Even if it’s a $20,000 lens. Just because you think no compromises should be made, doesn’t mean it can be so.”

    I served on a submarine costing the better part of a billion dollar – and there were design compromises (big and small) throughout the boat. Nothing designed by the human brain lacks compromises in it’s design.

  • Yeah yeah, I know you’re busy. 🙂

    Thanks for the explanations you were able to give on reddit. Really interesting information there.

    Roger, holding out on software updates on us? :-p

  • et tu, Brandon 🙂

  • Brandon Dube

    Kinda busy getting a degree over here. And writing the software for OLAF. And trying to get around to writing full length articles for photozone 🙂

    Nag Roger – he’s had v2.1.1 of the software with fancy dandy plotting techniques for a while now and hasn’t shared yet, hehe

  • John Dillworth

    Well said sir. I buy in a different line, but as a paying consumer I too, look forward to their arrival. At that price point it seems a gaunlet has been thrown. I look forward to the duel and wish the new challenger the best. Now drop the glove and call the victor!

  • I see you over there on reddit answering complicated questions. Why aren’t you translating even more of your extensive knowledge into blog posts. Your explanation on CaF2 coatings would be a really interesting and indepth read.

    Signed,
    The Editor that always wants more nerdy articles

  • CarVac

    And the nighttime stalker’s dream lens too…

  • Mark

    Great read. Are these micro differences in the same lens reflected in your Keepers Program prices?

  • Brandon Dube

    Aaron commented on the blog? Now I’ve seen everything.

  • AaronClosz

    John:

    It’s a range that we’re very familiar with, optically and mechanically. There are certain expectations that come along with these focal lengths, and well, you add “Master” to the description one wants to be impressed. We will certainly be looking forward to their arrival.

  • John Dillworth

    It’s articles like this that make me love you guys. Really helpful and educational. I like trying to read between the lines and figure out what manufactures you are referring to in the last paragraph but I can make some pretty good guesses based on your recent teardowns and copy to copy variation tests. to some extent you get what you pay for. BTW, how excited are you guys to get your hands on those new Sony G lenses?

  • Brent Simmons

    So, I guess that terrifying Light Bender fisheye is going to be a real thing, huh?

  • RomainCR

    Amazing article, extremely interesting. Thank you for all the work you do, for allowing us to realize the constraints of manufacturers and appreciate the inner beauty of lenses (literally).

    I’ve become more sensitive to QC these past years, especially after switching to Sony and noticing that all (!) the lenses I bought had to be exchanged at least once or twice before being acceptable (to my own standards obviously). I obviously am not statistically representative, but I found that extremely annoying, and that made me loose trust in that brand. Thus, It’s a great thing that you raise awareness about this kind of issue, as it probably has more concrete consequences to photographers than new optical designs that bring supposedly higher resolution that are never achieved in practice. Hopefully lenses manufacturers will pay more attention to this issue, and at least those who do are justly rewarded.
    We are bombarded by marketing nonsense (and meaningless, ethically dubious, lenses reviews) and your insightful analysis bring some truth to this complexe world. And satisfy my inner geek. Thank again for your work.

Follow on Feedly