LensRentals.com

Wavelength-Detecting Sensor Eliminates Bayer Filter, Triples Resolution

Posted by

“Current camera sensor technology is completely backwards.” Dr. Eno Lirpa

Everyone knows that in order to generate color, a digital camera’s sensor is overlaid with a Bayer filter. The filter makes each pixel sensitive to either red, blue, or green light.

 

Standard Bayer filter, courtesy Wikipedia Commons

 

Software than interpolates this red, green and blue image into the final color image we see.

 

Representation of an scene (above) and the raw Bayer data captured by a digital camera (below). courtesy Wikipedia Commons

 

In effect, our 24-megapixel color camera doesn’t resolve any better than a 14 or 15 megapixel black-and-white camera would.

There have been several attempts to improve on the Bayer-array method of detecting color. The Foveon sensor, which stacks red, green and blue pixels at different depths at each pixel (sensor site), certainly provides higher resolution than a standard Bayer sensor, although the Foveon sensor has it’s own limitations.

 

Foveon X3 sensor stack, courtesy Wikipedia Commons

 

Fuji has altered the array in their sensors, creating a more random pattern. This gives (arguably) some improvement over the standard Bayer array but still uses the same basic principle with inevitable loss of resolution.

 

Fuji X-trans array, courtesy fujifilmusa

 

At WPPI, I had the chance to spend time with the team from Baceolus Imaging, a small Italian imaging technology company with a growing patent portfolio and plans to make a big splash.

Using Energy to Detect Color

I was able to talk with Dr. Eno Lirpa, one of the optical physicists on the Baceolus team, about their new sensors.

“When looked at from a physics point of view, current sensor technology is just a foolish design”, Lirpa says. “You give away so much resolution just to detect color. It’s just not necessary. Every photon already carries a color message.”

Every first-year physics student learns the simple formula E = hc/ λ where E equals photon energy and λ it’s wavelength. If you know a photon’s energy it’s simple to calculate wavelength and therefore determine the photon’s color.

For example, blue light, with a wavelength of 400nm has an energy of 3 electron volts per photon, while 700nm wavelength red light has an energy level of 1.77 electron volts, and green light 2.43.

“It’s a fairly simple engineering matter to measure a photon’s energy as it strikes the camera sensor, but everyone has been so focused on cramming more megapixels onto the chip,” Lirpa continues. “There hasn’t been much interest in adding additional technologies to the chip.”

There is no Bayer array over a Baceolus sensor. Other manufacturers have used new back-illuminated sensor technology to move the wiring behind the actual photosensor, rather than in front of the sensor.

 

Difference between standard and back-illuminated CMOS sensors. Sony: http://www.sony.net/SonyInfo/News/Press/200806/08-069E/

 

Baceolus goes one step further. They’ve taken advantage of a back illuminated sensor to place energy sensing circuitry in front of each photo well. A simple calculation converts energy level to wavelength, determining that photon’s color.

 

Schematic of Baceolus sensor, courtesy Baceolus Imaging

 

“It is a simple thing, in addition to detecting that a photon has struck the sensor, to also determine the energy level of that photon,” Lirpa explains. “Our sensors record that energy and pass it along to an in-camera chip that uses this information to not only detect the intensity of light striking each pixel, but also its energy level. The chip converts that energy level to wavelength, which shows the color of the photon.”

“Right here we can tell,” Lirpa continues, showing a diagram, “that a photon with energy level of 2.14 eV struck this pixel. That’s a lovely yellow photon, probably from a sodium vapor streetlight. A blue photon struck this pixel and an orange-red photon over here.”

Numerous Advantages

According to Dr. Latot Parc, the researcher designing the computer chips to process the images in-camera, a 20 megapixel Baceolus sensor not only provides the same resolution as a 40 megapixel Bayer array camera but an amazing 32 bits of color depth. I was only allowed to keep one image taken with the camera, but below are 100% crops taken with a Canon 5D III and a Baceolus preproduction camera using a Canon 135mm f/2 lens. (Yes, the Baceolus camera will mount any Canon EF lens, so a huge lens selection is already available. The Baceolus team would not comment, but I have the impression a Nikon-mount version may be in development.)

 

Overall Image

 

100% crops comparing Baceolus sensor (left) compared with standard sensor using same lens (right).

 

There are other advantages to an energy-detecting sensor. High-energy ultraviolet and low-energy infrared light can be screened out in firmware, as the camera processes the image, so an infrared filter is not required. A simple flip of a switch tells the camera to change from visible light to infrared or even ultraviolet.

 

 

A simple switch on the camera changes from normal mode . . .

 

 

To combined ultraviolet, visible, and infrared mode, or any combination of these. 

 

Additionally, the sensor can act as a ‘photon trap’, detecting only photons of a given energy range. This could allow an astrophotographer, for example, to set the camera to only accept light with wavelengths of 410 nm, 434 nm, 486 nm, and 656 nm  (the spectrum of hydrogen light) or to capture simultaneous UV and IR images of a nebula.

 

Image of 30 Doradus in full spectrum of UV through IR (above) as compared to normal Bayer sensor image (below). Credit F. Paresce and R. O’Connell

 

I did ask Dr. Parc about rumors that some beta testers had set their Baceolus sensor to detect only flesh tones, allowing a photographer to basically take an image of a person’s body right through their clothes.

“It’s certainly possible but would require long exposure times.” Dr. Parc said. “Doing so would completely overexpose the subjects face, hands, and other areas not covered by clothing, so we doubt anyone would be interested in doing that. Besides, our extensive research shows that very few photographers are interested in shooting nudes. At any rate, a person concerned about being photographed through their clothing can simply wear thick woolen underwear, which is an effective photon blocker.”

“The advantages our sensor brings to resolution and color, especially in the studio, are amazing,” Dr. Parc adds. “For example, everyone shooting a digital SLR today is frustrated by their camera’s inability to properly reproduce the color Periwinkle. We hear it every day. Whether it’s a model’s periwinkle blouse, or a beautiful periwinkle flower, the photo is drab and lifeless because the Bayer array’s red, green, and blue filters just can’t reproduce periwinkle accurately. With our sensor, periwinkle is simply 2.513 electron volts, rendered every bit as accurately as any other color.”

 

True Periwinkle (left) is often rendered inaccurately by Bayer-sensor cameras, appearing too purple (center) or aquamarine.

 

Even more important, the Baceolus team believes, will be the ability to bring cell-phone staples like face detection and smile detection to full SLR cameras. They envision a setting in which the camera identifies areas of 2.03 to 2.13 Ev color (flesh tones), locking that area in as face detection. It then automatically takes a picture when a minimum of 5% of that area changes to 3.1 Ev – the near-ultraviolet color of bleached teeth — so you get the perfect smiley face every time.

And One Disadvantage

There is only one thing not improved on a Baceolus sensor. The circuitry used to measure photonic energy does detract from dynamic range slightly. Pre-production Baceolus sensors have a DR of about 12 electron volts, less than most current SLR cameras.

“This is no problem for good photographer,” states Yug Diputs, who recently joined Baceolus as director of marketing, in slightly broken English. “Combining two images at different exposure is best way to take picture anyway. Only bad photographers are limited by dynamic range. You see this repeated on every forum on internet — problem is always bad photographer, not bad equipment.”

When Can You Get One?

Very soon, according to what I’m hearing. Release in Europe and Japan is expected by early summer.

“We expect to have our camera on the market well before the Canon 200-400 f/4 IS lens is released,” Parc told me. “We got very lucky because Nikon has, for some reason, a large supply of autofocus sensors they can’t use which we were able to buy very cheaply. That moved our release date up several months.”

Release in the United States, unfortunately, will be delayed for quite a while. “In the entire world, only the United States, Burma and Liberia do not use the metric system. Because our system converts the electron volts into nanometers, it is calibrated for metric light and we can’t guarantee image quality in areas using nonmetric light. We hope to have a firmware upgrade that will convert electron volt measurements to inches by September or October.”

“Dual-system countries, like Great Britain and Canada, appear to have sufficient metric-wavelength light to allow our cameras to work properly,” Diputs was quick to add.

Price has not yet been determined.

Roger Cicala

Lensrentals.com

April 1, 2013

65 Responses to “Wavelength-Detecting Sensor Eliminates Bayer Filter, Triples Resolution”

mentatmark said:

I have a metric to english filter. Is it possible to reverse the optic so I can convert english to metric? Is there degradation of the resultant image?

eddy said:

I believed everything until I read the comments! You got me Roger!

By the time I read this, it’s already 2 April in my country.. not fair, not fair.. hehehe.. Eno Lirpa.. damn.. right in front of my nose!

Nikhil Ramkarran said:

Clever :) Had me going till “Yug Diputs”.

Milton Simoes said:

Roger, Jesus, I mean Roger,
I didn’t deserve the Honour (seriously)

Now back to business:

How could I miss it, the Double Barrel Shotgun Old Trick……
But this is the first time I ear of Possum Ducks (by the way I used to be a Biologist, but my speciality was Tropical Grasslands Ecology, which could explain my lapsus mentis )

I have to confess that I am a member of the Slow Street Photographers Assoc., well, virtually the president by merits. And I think this could be a wonderful People Freezing Devise (one way or another, Capicci?).
Do you mind to work with me in a little experiment? Let’s go to some nice place for street photos, like NYC, and there you could teach me the Shotgun Hypnosis Technique and both try it with the people in the streets!!! It sure will give us some nice and original pictures, and this would make the Found Portraits concept of M. Reichmann pale by comparison. (well, the people would be a bit Pale too, but we can “fix it in Post”)

Technically, we should use high ISOs, to AVOID blurring. It have been reported that this method of hypnosis induces some trembling on the subject, so we have to compensate with a speed of 1/2000 at least, to get sharp hands and knees. Other thing that has to be expected is that if we use this for more than 15 minutes, the models will develop some thick drops of sweating. But we can “fix it in Post” too.

Well, as some one said before:

“Photographers are violent people.
First, they Frame you,
Then, they Shoot you,
And, finally they Hang you on the wall.
And wile doing it, they insist that you Smile”

Best wishes.
And happy shooting (well, from now on, a will omit this last one……)

PicNic said:

Very nice… Quite a novel concept, nonetheless.

“Besides, our extensive research shows that very few photographers are interested in shooting nudes.” – April Fools!!!

crusaderky said:

You had me until I read about the nonmetric light wavelenghts…. too bad, those duck crops had my eyes fall out of my sockets :|

TomH said:

Nice slow introduction of the absurd components, well done.

There are some devices which can measure the energy, and thus wavelength, of individual photons: Calorimeters and Multi-Anode Microchannel Arrays (MAMAs) if I recall correctly. Both are unfortunately useless under normal illumination where they are overwhelmed by photons and produce useless data, but for light-starved applications like telescopes can work quite well.

Garret van der Veen said:

You will have better resolution with a 3 CCD digital camera.
And more lightgrasp if you use 3 digital cameras and 3 lenses; one for each color….

Oscar Medina said:

This is a really well written April Fools joke. Well done! I used it to fool most of my photographer friends.

Dr. Eno Lirpa and Dr. Latot Parc ….. Classic misdirection! Words spelled backwards. Fabulous!

Who would have known there were such things as metric vs. american standard photons. LOL

When I read that, I thought of an practical joke we would use on beginner lighting assistants. We’d send them off to find a metric crescent wrench.

Jason said:

Hey, is that Friendship Bay on the Island of Bequia? It looks like it, I use that hill as a line-up to find a shipwreck of Isle Le Quatre.

LensRentals Employee

Roger Cicala said:

Jason,

It’s St. Thomas, looking across the bay from the mountaintop.

MI said:

I must confess, I laughed so hard a little bit of wee came when I read “True Periwinkle (left) is often rendered inaccurately by Bayer-sensor cameras, appearing too purple (center) or aquamarine”.

LensRentals Employee

Roger Cicala said:

Wow! I’m smarter than I thoughth :-)

Leave a Reply