LensRentals.com

Introducing the Optical Bench

Posted by

Note: This is a Geek article. If you aren't into geeky stuff, you won't be into this.

"We take a step back so that we may leap further." African proverb

I wrote a post a couple of weeks ago about our first copies of the Sigma 50mm Art lens, and promised to follow up when we got more samples. Unfortunately, for the first time in history, Tyler just can't get any copies, so I've been unable to look at sample variation any further.

It's always been my practice to not write anything when I don't have anything to write about -- somewhat unusual for a blogger, I know. But it's been so long since I posted that several people have asked if I was OK. So I thought I'd show you why we've been so busy, and why we should start posting some very cool things quite soon.

More Equipment

Right after we moved into new space, we took delivery of OLAF and started improving our lens adjustment and assessment techniques. We spent most of April learning what OLAF could show us and how we could use it.

Our Imagemaster optical bench was delivered from Trioptics the first week in May, arriving in a crate that was larger than some hotel rooms I've stayed in. Unpacking it was fun - although we had to wait for Byron, the Trioptics engineer, to arrive before they'd let us open it.

Byron wouldn't let us touch much stuff.

Even warning labels seem much more serious when they're in German. I thought this meant 'have a beer before setting up this equipment', but apparently it's just 'Handle with Care'.

Set next to OLAF, I thought our new toy testing equipment looked very nice and I was ready to play with it.

The equipment in our optical testing room now is worth slightly more than my house. I'd appreciate it if nobody tells my wife, OK?

But assembly took a lot longer than I expected. Judging from his expression, I think it took longer than Byron expected, too.

Once it was finally together, I thought we'd get to play with it. But there was lots of calibration and recalibration left to do.

Followed by two full days of rather intense training complete with a 200-page manual to study. I was afraid there'd be a test and we wouldn't get to keep the machine if we failed.

Aaron (on the left) takes notes during training. I rely on sheer memory power -- and access to Aaron's notes.

And I've always wanted to have a machine with an emergency stop button. I can't imagine what we would need to emergently stop testing a lens for, but we're absolutely ready if it happens.

Of course after taking 3 days for that stuff, we were so backed up in repair that we had to spend a couple of more days catching up. So it was last week before we really got to start playing with using our new toy equipment.

So What Does This One Do?

First and foremost, of course, it measures the MTF of the lens. As opposed to Imatest, it measures with the lens focused to infinity (depending on focal length, Imatest measures at distances from 5 to 30 feet). Having both capabilities is important because many lenses perform differently at infinity than they do close up.

It also measures just the lens, where Imatest and DxO measure a lens-camera combination. That's important because it lets us separate out many variables, especially when so many cameras are now processing even raw files in-camera. And it does it quickly, as you can see in the video below.

MTF testing with the TRIOPTICS ImageMaster from LensRentals.com on Vimeo.

It does a lot of other very nice things, too, like measure field-curvature, distortion, actual focal length, lateral and spherical chromatic aberrations, and shoe size. The bottom line is we now have access to more optical testing equipment than - well, anyone in the U. S. except a few government installations and some Universities.

We'll be using it for lens tests and reviews going forward, but before we start doing that I have to assemble a comparative database: right now I have data for thousands of lens copies using Imatest, so I know things like expected sample variation, acceptable ranges, etc. Until we get some similar data using the Imagemaster I won't be comfortable making generalizations. But in the meantime I thought I'd show you some examples.

Some Optical Bench Examples

MTF Across the Field

Remember that in some ways optical bench results are very different from Imatest results, as I blogged about earlier and more like the MTF maps that manufacturers give for their lenses. With the major difference that these are measurements of real lenses, not computer-generated ideal-o-graphs.

Below is an example for Imagemaster results for a very good lens, the Zeiss 21mm f/2.8. Notice the graph shows 10, 20, 30, and 40 line pairs / mm. (Since we've previously reported MTF50 in line pairs / image height, we can multiply by 24mm to get the numbers per image height, so if you'd prefer this is the graph for 240, 480, 720, and 960 lp/ih.) The solid lines are sagittal resolution (lines going from the center of the image toward the edge); the dotted lines are tangential (lines at right angles to sagittal lines). The difference between the two is astigmatism.

Probably the first thing you notice is this lens has a bit more astigmatism on the far right side. This is just minor sample variation, every lens is slightly different. For comparison the image below shows another copy of the Zeiss 21mm that was decentered enough to be noticeable when carefully shooting a test chart. Not only is the astigmatism much more severe, but the absolute MTF numbers are lower (notice the 20 lp/mm lines are around 0.9, or 90% in the good lens, while they're largely below 0.8 in the bad lens).

Frequency Graphs

Frequency graphs show us how a single point on the lens (usually the center) resolves as the lines get smaller and smaller. Here is a frequency graph comparing the centers of the two lenses in the graphs above. The better lens is the red curve and again, solid lines are sagittal and dotted lines tangential. You don't have to understand what this is to realize it's a tool that does a nice job of comparing lenses.

Distortion

It doesn't take an optical bench to show distortion, but it's nice to be able to check it at infinity. The Canon 50mm f/1.2, for example, shows the same amount of distortion (1.45%) at infinity as it does close up (the dotted line here shows a normalized curve, the solid line actual plot points).

 

It's also nice to be able to test multiple copies, because there is a bit of copy-to-copy variation with distortion, too (note the scale of the graph has changed here). The graph below shows 3 copies of the Sigma 50mm f/1.4 lens.

Field Curvatures

One of the coolest, and perhaps almost overwhelming, features is the ability to plot out field curvatures. It's the only really time consuming test, taking 5 to 6 minutes while the others are done in a minute or less, but it's well worth the wait.

Obviously, knowing the field curvature of a lens is helpful when framing a shot, but it's helpful when we're adjusting a lens optically, too, since a field tilt often requires different adjustments than other causes of 'soft on one side'.

Astigmatism

Another test result that's interesting in comparing different lenses, but also very useful when we're optically adjusting lenses is an astigmatism plot.  The first two examples below just show different lenses, one in which astigmatism occurs even slightly off-center, the second in which it occurs only near the edges, and a third in which it is abnormal because of decentering.

Actual Focal Lengths

The Imagemaster contains a grid that lets us measure the exact focal length without a lot of calculations. It isn't particularly important to note that the Sigma 50mm f/1.4 Art lens has an actual focal length of 49.5mm, while the Zeiss 50mm f/1.4 is 51.5mm and the Canon 50mm f/1.2 is 51.8mm. But I found it amusing when I saw the Zeiss 55mm Otus is actually 54mm, because I'd just read a comment that someone found the Canon 50mm focal length much more usable than the 55mm of the Otus. Really, 52mm is much more usable than 54mm?

It does become slightly interesting with zooms. For example, the 24-70 f/2.8 zooms are between 66 and 67.5mm at the long end, while all 70-200 f/2.8s we've tested start at 74 to 76mm.

So Why Aren't You Posting Some Tests, Roger?

Time, of course, is the major factor. We've spent most of the last two weeks just learning the equipment; testing lenses we already know well to make sure our results are accurate and reproducible. Our second priority (which justifies spending all of this money) was getting the optical bench inserted in our testing protocols and learning how to use it to help us with optical adjustments.

We will be starting to do some testing and comparisons, soon, but before that I've got to figure out the best ways to present the data from multiple copies -- the major contribution I make when evaluating lenses is showing sample variation and the range of expected results. The sheer mass of data generated by the Imagemaster makes that challenging. An Excel spreadsheet with the results of one MTF run has about 500 cells of data. Multiply that by two to three runs per copy of 25 copies and pretty soon it's a lot of numbers.

For example, below are composites made from MTF runs of 5 copies of the Zeiss Otus 55mm and 5 of the Sigma 50mm f/1.4 Art, each run at two angles.

Showing sample variation in so crowded a graph is going to be difficult. Standard deviation bars are probably the best solution, although they'll get crowded and aren't very elegant. But I'm open to other suggestions as I try to figure this out, so if you've got some great ideas please let me know.

I know many of you have 632 awesome ideas for things we should test tomorrow, but please be patient a bit longer. We still don't have all of the various lens mounts machined and calibrated yet, so it will be a while before we can test those Sony and micro 4/3 lenses without a camera modifying the images. Canon and Leica comparisons should be coming out fairly soon, and hopefully we'll use those first posts to work out the best ways to present all this data to you.

Roger Cicala and Aaron Closz

Lensrentals.com

May, 2014

59 Responses to “Introducing the Optical Bench”

Someone said:

Thanks for the short video. Man that thing sounds like a dinosaur, like a raptor from Jurassic Park. Nice to meet your new machine, Mr. JP Raptor. j/k

Things are getting exciting over there I'm sure. Can't wait to see how your review evolve with the new test equipment, err, I mean toys. :)

Btw, it's always a please to read your posts and your humor always improves my day!

gandalfii said:

The label reads:

Danger of Breaking
Do Not Throw
Protect from Wetness

Sharky2014 said:

Roger, how much, if any, of a role have you seen interior lens dust, smoke, grime, haze, etc. play in your testing experience? Have you seen it affect results at all?
How concerned (or not) should we be about this stuff? It's easy to keep the outside clean, but any recommendations or guidelines for interior lens cleaning?

LensRentals Employee

Roger Cicala said:

Sharky, I've never seen dust have an effect unless it's a total coating of dust or a huge piece (think insect). Smears like from lubricants, though, have more of an effect, probably because they actually refract light rather than just stopping it, or interfere with coatings.

Roger

Daryl Ovadia said:

Hi Roger,

Where can I get one of these? :-)
Please blog all you can, for us lens geeks this is heaven. The graphs you have posted are spectacular, many I have never seen but the information is so useful. The field curvature images are difficult to understand, can you give some explanation, if only briefly, as to their meaning?
Can't wait to see further information. Thank you.

LensRentals Employee

Roger Cicala said:

Daryl,

I'll try to explain the field curvature thing. When we test the MTF, the machine focuses best for the center, then keeping that focus tests all of the other points on the graph. When we do the field curvature test, the machine goes to each testing point, goes through all of the possible focuses, and notes which one is best. The plot then show where the lens is sharpest with focusing point being one of the axes. So if on the edges, the machine focuses at +37 microns, while in the center it was +26 microns and half-way +30 microns, it shows that as curvature. If the field was perfectly flat, the best MTF reading would be at +26 microns at every tested point.

I think I said that right, but it's Sunday and I'm not all that mentally engaged.

genotypewriter said:

Very nice article and gear, Roger.

A quick question... when this measures the MTF, does it measure it on a flat field or based on individual lens' measured curved field (as in the point of best focus)?

Thanks!

LensRentals Employee

Roger Cicala said:

geno - it can do several ways, but what we're generally measuring is MTF at best center-focus. We can individually focus at each point but I feel performance at a single focus is more real world.

genotypewriter said:

Thanks, Roger. So if resolution at best focus is taken off center, which do you think wins... the Apo Summicron 50 or the Otus 55? :)

Leave a Reply