Autofocus Reality Part 4: Nikon Full Frame

I started this series to simply demonstrate a well-known fact; that contrast detection autofocus in Live View was more accurate than standard Phase Detection autofocus for still subjects. It ended up being a bit more than I’d bargained for just with the Canon cameras. After that series was done, I promised several people that I would take a similar look at Nikon cameras. I also got several pertinent suggestions and criticisms in the first series about gathering more data and presenting more examples. I promised to do that, too.

You ever notice how easy it is to promise to do some work later on? Only then do you realize that what you’ve promised is going to take about 30 hours to get done. That causes an immediate disturbance in the force, and your day job workload triples because there’s a holiday weekend coming up. Plus, some cool stuff that you really want to play with need to carefully evaluate comes in. So for the 3 or 4 people who were waiting for this, I apologize for taking so long.

The Conclusion

Several people wanted to see more data presented, so I’ll try to show more this time. Most people, though, simply want to know what I found. A few, of course, want to know how they can work in what I found into some raving Fanboy post or other. Sorry, Fanboys, there really isn’t any Canon – Nikon forum fodder here. These tests are fairly blunt tools that are only looking at one tiny aspect of autofocus.

Anyway, if you just want the results, they are pretty straightforward:

1. The Nikon full-frame cameras all do well with center-point, single-shot AF.
2. The newer bodies seem a bit better than the older ones.
3. There don’t seem to be dramatic changes with a few camera-lens combinations like the newer Canon bodies and lenses showed.

There are a few rather interesting things we found this time around. Additionally, my “write as you go, not when you know” technique in this series has gotten several knowledgable people to email more thoughts and information. That helped me accomplish my main purpose: find out more about how phase detection AF works. Which lets me speculate a bit more at the end of the article.

Data

We tested a larger number of lens-camera combinations this time around. As always, we used cameras and lenses that had recently been tested and found acceptable. One series of shots, discussed here, was removed and not included in these results since the lens used was found to be decentered.

I’ll show some example graphs of various lens-camera runs to give an idea of what the actual data points look like. This space is too small to try to give a graph of every data point, so there will still be some summary numbers. The graphs show resolution (MTF50 measure by Imatest) for different lens-camera combinations. One lens tested on one camera is a run – then we changed both lens and camera for the next run, etc. For each run I show the results for 10 shots taken with Liveview (contrast AF) as diamonds, 10 shots with standard phase AF as crosses. The vertical axis is average resolution at 13 points on the sensor (corners, edges, 4 midpoints, center) while the horizontal axis is resolution at the center point (all in line pairs / image height from unsharpened raw files).

Each run (represented by one color) is one copy of the camera with one copy of the lens; no lens or camera repeats. For example, when we tested Nikon D800s with Nikon 28mm f/1.8 lenses, we tested 4 different lenses on 4 different D800s. I’m only showing two runs in the graphs because they get too crowded and run over each other, but all 4 are used in determining the average numbers. I repeat this because tomorrow someone will be saying “they only tested two cameras” because, well, some people apparently find reading too hard and they just look at the pictures.

One other thing is worth mentioning. We look at the standard deviation as a convenient measure of variation between the sample sizes. That’s a fairly valid and convenient measurement when we’re comparing D700 contrast detection to D700 phase detection. When we start comparing D700 numbers to D800 numbers, though, the resolution numbers are quite different. The absolute numbers of the D800 are much higher, so the same perecentage of variation is going to make a slightly larger standard deviation.

Some Visual Examples

The range of the axis on the graphs below change depending on which camera is used. In the Nikon D700 graph, for example, the peak resolution range is 400 to 900 LP/IH. With the D800 the range shown is 700 to 100 LP / IH.

For the D700 and D8000 I only graphed two of the 4 runs to clearly show how contrast detection and phase detection varies. For the Nikon D4 and D800 I graphed 3 runs. It makes the graphs a little more crowded, but in both of those graphs the runs colored red make a point I want to use. They also show why I didn’t try to put 4 runs on one graph.

 

The D700 shows the difference betweeen phase and contrast detection similar to what we’ve seen in the better Canon bodies. Contrast detection gives a very tight pattern, about as good as I could do using careful live view manual focusing with magnification. Phase detection gives some shots every bit as good as contrast, some not quite as good numerically but not different enough that you could easily tell in a photograph, and an occasional shot that has just missed focus.

 

 

The Nikon D3x gave us similar results. I should mention that out of the 4 runs with these two cameras we had 3 completely miss focus with the D700 (one shows in the graph above) and 4 with the D3x out of 40 shots with each. That’s about what we’ve seen with most Canon cameras and similar to the missed focus rate (around 10%) others have reported with phase detection.

 

 

The D800 example above shows something worth demonstrating again, because so many people still don’t get it. For the green and blue runs phase detection is similar to the previous cameras: a bit wider spread (larger standard deviation) but generally close to the contrast detection results.

The red run (run 2), though, shows the phase detection average is clearly a bit lower than the contrast detection. In other words, the pattern is similar to the phase pattern of the other runs, but the average is further away from the best shots obtained with contrast detection. That’s because this lens require microfocus adjustment of -5 on this particular camera. When we made that adjustment and repeated the run, the red pattern looked similar to the others.

The point: microfocus adjustment brings the average up of phase detection AF up closer to the contrast detection range, but it doesn’t reduce the variation seen with phase detection shots. The D4 graph below shows a similar example. For purposes of the averaged numbers I’ll talk about below, we repeated those runs after making appropriate microfocus adjustments and used the adjusted runs for our actual calculations.

 

You’ve noticed in the graphs, I’m sure, that both the D4 and D800 seemed to have fewer ‘bad misses’ than the D3x and D700 did. The D800 had 2 of 40 missed shots, the D4 1 of 40. That’s not clearly significant, but certainly an interesting trend.

Data Summary

The summary of multiple runs using different camera and lens each time for 28mm f/1.8 lens shot at f/2.8 on the various cameras is shown in the table below.

Camera	Cont	Phase	Avg Res
D4	14	22	689
D800	15	24	834
D3x	16	36	741
D700	12	30	621

It doesn’t look like much, I know, but that table represents 320 Imatest shots. Before we start discussing it, let me be clear: the numbers we’re looking at aren’t a statistical analysis and there’s anyone who tries to say “Camera X was 30 and camera Y was 17, so camera Y has a better autofocus system” is missing the point entirely. We’re looking at one tiny part of the AF system: how well does the phase contrast system compare to contrast detection looking only at the center point, with a still target.

Before we started we all knew contrast was more accurate. When we tested Canon cameras, most had a standard deviation about twice as large for phase detection as for contrast. The Canon 1D Mk IV was more accurate than that. When mounted with a select set of the newest lenses, the Canon 5D Mk III and Canon 1Dx were just about as accurate in phase detection as with contrast detection. With older lenses, though, they were about as accurate as the 1D Mk IV.

With the Nikon full-frame cameras we see that the D700 and D3x seem similar to the Canon cameras: phase detection has a standard deviation about twice as large as contrast detection. The newer cameras, the D800 and D4, seem clearly better, very similar to the newer Canon bodies on most lenses.

Of course, we did check other Nikon lenses to see if there was a subset, like the newest Canon lenses, that seemed as accurate with phase detection as with contrast on the D4 and D800. We did smaller runs (2 lenses on 2 cameras) with the  70-200 f/2.8 VR, 35mm f/1.4 G and 85mm f/1.4 G. Results were similar to the 28mm f/1.8. We’ll check back with the next new Nikon lens release to see if it has improved accuracy on these cameras, whenever that is.

 Discussion

There’s not really any difference between Canon and Nikon cameras for the lenses we all use every day. Both do pretty well, and the newer cameras are a bit better than older models. Of course, that doesn’t mean there’s a huge difference in real life.

A better real life comparison would be “how many shots were badly missed” like the ones you see in the D700 graphs. That degree of miss would be obvious in the photograph. The variation in standard deviation probably would only be detected by pixel-peeping. Unfortunately, it would take several hundred more shots with each camera to get large enough numbers to determine statistical significance for number of missed shots. At roughly an hour per Imatest run, there’s no way I’ll have enough time to ever do that.

These results don’t translate to outer focus points, Servo focus, etc. It simply hints that the manufacturers are doing something that makes phase detection more accurate on still shots using the center points.

There does seem to be a small group of Canon lenses (24mm and 28mm f/2.8 IS, 40mm f/2.8, 300mm f/2.8 IS II) that are more accurate on just the newest bodies. We’ll have to wait for the next new Nikon lens release to see if there is a similar improvement with newer Nikon lenses.

On the other hand, there’s one thing worth considering about the “more accurate” group of new Canon lenses. All are f/2.8 prime lenses. It will be very interesting to see if the the Canon 24-70 f/2.8 II lens, and any future fast prime lenses, are also more accurate. And yes, I was able to get the Imatest shots to check that, it will  take me a while to get the numbers run, though.

I know some of you want to know how the crop sensor Nikons did. I’ll be honest; this is incredibly time consuming work and I just can’t get to more cameras right now. I’m comfortable than none of the crop sensors will be better, and some will probably be worse than the full-frames. That’s what we saw on Canon cameras and I don’t think it will be hugely different for Nikon. But it’s certainly possible that Nikon crop cameras are just as good as Nikon full-frames.

I’ve also had several people ask me to compare contrast detection accuracy between SLRs and mirrorless cameras. That’s kind of a fool’s errand. Contrast detection on SLRs is as accurate as I can measure. We can’t detect more accurate than we can measure.

I do think it would be worthwhile for someone to compare the speed of contrast detection in different systems. While I expect all contrast detection to be as accurate as we could measure, doing this little experiment made me very aware that the speed to lock on using contrast detection is very different with different cameras. The most practical way would probably be to record the AF motor sound and then use that to measure time.

Notice I’m giving this suggestion out for someone to do — because I’m not going to be able to get to it any time soon. If one of you wants to do it, we’d be happy to publish the results. Heck, if one of you wants to come here and do it with our gear, we’d be happy to set you up.

 

Roger Cicala

Lensrentals.com

September 2012