Humor and Sarcasm

Wavelength-Detecting Sensor Eliminates Bayer Filter, Triples Resolution

Published April 1, 2013

“Current camera sensor technology is completely backwards.” Dr. Eno Lirpa

Everyone knows that in order to generate color, a digital camera’s sensor is overlaid with a Bayer filter. The filter makes each pixel sensitive to either red, blue, or green light.

 

Standard Bayer filter, courtesy Wikipedia Commons

 

Software than interpolates this red, green and blue image into the final color image we see.

 

Representation of an scene (above) and the raw Bayer data captured by a digital camera (below). courtesy Wikipedia Commons

 

In effect, our 24-megapixel color camera doesn’t resolve any better than a 14 or 15 megapixel black-and-white camera would.

There have been several attempts to improve on the Bayer-array method of detecting color. The Foveon sensor, which stacks red, green and blue pixels at different depths at each pixel (sensor site), certainly provides higher resolution than a standard Bayer sensor, although the Foveon sensor has it’s own limitations.

 

Foveon X3 sensor stack, courtesy Wikipedia Commons

 

Fuji has altered the array in their sensors, creating a more random pattern. This gives (arguably) some improvement over the standard Bayer array but still uses the same basic principle with inevitable loss of resolution.

 

Fuji X-trans array, courtesy fujifilmusa

 

At WPPI, I had the chance to spend time with the team from Baceolus Imaging, a small Italian imaging technology company with a growing patent portfolio and plans to make a big splash.

Using Energy to Detect Color

I was able to talk with Dr. Eno Lirpa, one of the optical physicists on the Baceolus team, about their new sensors.

“When looked at from a physics point of view, current sensor technology is just a foolish design”, Lirpa says. “You give away so much resolution just to detect color. It’s just not necessary. Every photon already carries a color message.”

Every first-year physics student learns the simple formula E = hc/ ? where E equals photon energy and ? it’s wavelength. If you know a photon’s energy it’s simple to calculate wavelength and therefore determine the photon’s color.

For example, blue light, with a wavelength of 400nm has an energy of 3 electron volts per photon, while 700nm wavelength red light has an energy level of 1.77 electron volts, and green light 2.43.

“It’s a fairly simple engineering matter to measure a photon’s energy as it strikes the camera sensor, but everyone has been so focused on cramming more megapixels onto the chip,” Lirpa continues. “There hasn’t been much interest in adding additional technologies to the chip.”

There is no Bayer array over a Baceolus sensor. Other manufacturers have used new back-illuminated sensor technology to move the wiring behind the actual photosensor, rather than in front of the sensor.

 

Difference between standard and back-illuminated CMOS sensors. Sony: http://www.sony.net/SonyInfo/News/Press/200806/08-069E/

 

Baceolus goes one step further. They’ve taken advantage of a back illuminated sensor to place energy sensing circuitry in front of each photo well. A simple calculation converts energy level to wavelength, determining that photon’s color.

 

Schematic of Baceolus sensor, courtesy Baceolus Imaging

 

“It is a simple thing, in addition to detecting that a photon has struck the sensor, to also determine the energy level of that photon,” Lirpa explains. “Our sensors record that energy and pass it along to an in-camera chip that uses this information to not only detect the intensity of light striking each pixel, but also its energy level. The chip converts that energy level to wavelength, which shows the color of the photon.”

“Right here we can tell,” Lirpa continues, showing a diagram, “that a photon with energy level of 2.14 eV struck this pixel. That’s a lovely yellow photon, probably from a sodium vapor streetlight. A blue photon struck this pixel and an orange-red photon over here.”

Numerous Advantages

According to Dr. Latot Parc, the researcher designing the computer chips to process the images in-camera, a 20 megapixel Baceolus sensor not only provides the same resolution as a 40 megapixel Bayer array camera but an amazing 32 bits of color depth. I was only allowed to keep one image taken with the camera, but below are 100% crops taken with a Canon 5D III and a Baceolus preproduction camera using a Canon 135mm f/2 lens. (Yes, the Baceolus camera will mount any Canon EF lens, so a huge lens selection is already available. The Baceolus team would not comment, but I have the impression a Nikon-mount version may be in development.)

 

Overall Image

 

100% crops comparing Baceolus sensor (left) compared with standard sensor using same lens (right).

 

There are other advantages to an energy-detecting sensor. High-energy ultraviolet and low-energy infrared light can be screened out in firmware, as the camera processes the image, so an infrared filter is not required. A simple flip of a switch tells the camera to change from visible light to infrared or even ultraviolet.

 

 

A simple switch on the camera changes from normal mode . . .

 

 

To combined ultraviolet, visible, and infrared mode, or any combination of these. 

 

Additionally, the sensor can act as a ‘photon trap’, detecting only photons of a given energy range. This could allow an astrophotographer, for example, to set the camera to only accept light with wavelengths of 410 nm, 434 nm, 486 nm, and 656 nm  (the spectrum of hydrogen light) or to capture simultaneous UV and IR images of a nebula.

 

Image of 30 Doradus in full spectrum of UV through IR (above) as compared to normal Bayer sensor image (below). Credit F. Paresce and R. O’Connell

 

I did ask Dr. Parc about rumors that some beta testers had set their Baceolus sensor to detect only flesh tones, allowing a photographer to basically take an image of a person’s body right through their clothes.

“It’s certainly possible but would require long exposure times.” Dr. Parc said. “Doing so would completely overexpose the subjects face, hands, and other areas not covered by clothing, so we doubt anyone would be interested in doing that. Besides, our extensive research shows that very few photographers are interested in shooting nudes. At any rate, a person concerned about being photographed through their clothing can simply wear thick woolen underwear, which is an effective photon blocker.”

“The advantages our sensor brings to resolution and color, especially in the studio, are amazing,” Dr. Parc adds. “For example, everyone shooting a digital SLR today is frustrated by their camera’s inability to properly reproduce the color Periwinkle. We hear it every day. Whether it’s a model’s periwinkle blouse, or a beautiful periwinkle flower, the photo is drab and lifeless because the Bayer array’s red, green, and blue filters just can’t reproduce periwinkle accurately. With our sensor, periwinkle is simply 2.513 electron volts, rendered every bit as accurately as any other color.”

 

True Periwinkle (left) is often rendered inaccurately by Bayer-sensor cameras, appearing too purple (center) or aquamarine.

 

Even more important, the Baceolus team believes, will be the ability to bring cell-phone staples like face detection and smile detection to full SLR cameras. They envision a setting in which the camera identifies areas of 2.03 to 2.13 Ev color (flesh tones), locking that area in as face detection. It then automatically takes a picture when a minimum of 5% of that area changes to 3.1 Ev – the near-ultraviolet color of bleached teeth — so you get the perfect smiley face every time.

And One Disadvantage

There is only one thing not improved on a Baceolus sensor. The circuitry used to measure photonic energy does detract from dynamic range slightly. Pre-production Baceolus sensors have a DR of about 12 electron volts, less than most current SLR cameras.

“This is no problem for good photographer,” states Yug Diputs, who recently joined Baceolus as director of marketing, in slightly broken English. “Combining two images at different exposure is best way to take picture anyway. Only bad photographers are limited by dynamic range. You see this repeated on every forum on internet — problem is always bad photographer, not bad equipment.”

When Can You Get One?

Very soon, according to what I’m hearing. Release in Europe and Japan is expected by early summer.

“We expect to have our camera on the market well before the Canon 200-400 f/4 IS lens is released,” Parc told me. “We got very lucky because Nikon has, for some reason, a large supply of autofocus sensors they can’t use which we were able to buy very cheaply. That moved our release date up several months.”

Release in the United States, unfortunately, will be delayed for quite a while. “In the entire world, only the United States, Burma and Liberia do not use the metric system. Because our system converts the electron volts into nanometers, it is calibrated for metric light and we can’t guarantee image quality in areas using nonmetric light. We hope to have a firmware upgrade that will convert electron volt measurements to inches by September or October.”

“Dual-system countries, like Great Britain and Canada, appear to have sufficient metric-wavelength light to allow our cameras to work properly,” Diputs was quick to add.

Price has not yet been determined.

Roger Cicala

Lensrentals.com

April 1, 2013

Author: Roger Cicala

I’m Roger and I am the founder of Lensrentals.com. Hailed as one of the optic nerds here, I enjoy shooting collimated light through 30X microscope objectives in my spare time. When I do take real pictures I like using something different: a Medium format, or Pentax K1, or a Sony RX1R.

Posted in Humor and Sarcasm
  • This is a really well written April Fools joke. Well done! I used it to fool most of my photographer friends.

    Dr. Eno Lirpa and Dr. Latot Parc ….. Classic misdirection! Words spelled backwards. Fabulous!

    Who would have known there were such things as metric vs. american standard photons. LOL

    When I read that, I thought of an practical joke we would use on beginner lighting assistants. We’d send them off to find a metric crescent wrench.

  • Garret van der Veen

    You will have better resolution with a 3 CCD digital camera.
    And more lightgrasp if you use 3 digital cameras and 3 lenses; one for each color….

  • TomH

    Nice slow introduction of the absurd components, well done.

    There are some devices which can measure the energy, and thus wavelength, of individual photons: Calorimeters and Multi-Anode Microchannel Arrays (MAMAs) if I recall correctly. Both are unfortunately useless under normal illumination where they are overwhelmed by photons and produce useless data, but for light-starved applications like telescopes can work quite well.

  • crusaderky

    You had me until I read about the nonmetric light wavelenghts…. too bad, those duck crops had my eyes fall out of my sockets 😐

  • Mike R

    That “single photon detector” mentioned in one comment? It’s here:
    http://www.nbcnews.com/technology/futureoftech/infrared-camera-takes-3-d-images-miles-away-1C9255119
    Only infrared – but if it can be done for IR, maybe just a matter of time until it’s no longer April Foolish?

  • PicNic

    Very nice… Quite a novel concept, nonetheless.

    “Besides, our extensive research shows that very few photographers are interested in shooting nudes.” – April Fools!!!

  • Milton Simoes

    Roger, Jesus, I mean Roger,
    I didn’t deserve the Honour (seriously)

    Now back to business:

    How could I miss it, the Double Barrel Shotgun Old Trick……
    But this is the first time I ear of Possum Ducks (by the way I used to be a Biologist, but my speciality was Tropical Grasslands Ecology, which could explain my lapsus mentis )

    I have to confess that I am a member of the Slow Street Photographers Assoc., well, virtually the president by merits. And I think this could be a wonderful People Freezing Devise (one way or another, Capicci?).
    Do you mind to work with me in a little experiment? Let’s go to some nice place for street photos, like NYC, and there you could teach me the Shotgun Hypnosis Technique and both try it with the people in the streets!!! It sure will give us some nice and original pictures, and this would make the Found Portraits concept of M. Reichmann pale by comparison. (well, the people would be a bit Pale too, but we can “fix it in Post”)

    Technically, we should use high ISOs, to AVOID blurring. It have been reported that this method of hypnosis induces some trembling on the subject, so we have to compensate with a speed of 1/2000 at least, to get sharp hands and knees. Other thing that has to be expected is that if we use this for more than 15 minutes, the models will develop some thick drops of sweating. But we can “fix it in Post” too.

    Well, as some one said before:

    “Photographers are violent people.
    First, they Frame you,
    Then, they Shoot you,
    And, finally they Hang you on the wall.
    And wile doing it, they insist that you Smile”

    Best wishes.
    And happy shooting (well, from now on, a will omit this last one……)

  • Clever 🙂 Had me going till “Yug Diputs”.

  • eddy

    I believed everything until I read the comments! You got me Roger!

    By the time I read this, it’s already 2 April in my country.. not fair, not fair.. hehehe.. Eno Lirpa.. damn.. right in front of my nose!

  • I have a metric to english filter. Is it possible to reverse the optic so I can convert english to metric? Is there degradation of the resultant image?

  • jb

    NICE !!
    I was completely fooled until I read 3 times the end about “metric-light”…
    You’re funny when serious, and the other way too…
    Keep going roger

  • Roger Cicala

    Eno Lirpa backwards is April One

  • Gavin Melville

    Ok — I got it.

  • Gavin Melville

    I still don’t get the Dr Eno Lirpa reference — is this a USA only in joke ?

  • As someone who does sceintific spectroscopy professionally and photography as a hobby, I have to say that I want one of these sensors for both work and play. You’ve clearly outlined the advantages in imaging, but you have no idea how happy I’ll be to get rid of the expensive, large, and now outmoded junk in my lab, like monochromators, Fourier transform spectrometers, bandpass filters, and the like. One prosumer-grade camera body, and my lab is all set! I’ll go start smashing my monochromators and using their 10 cm, 2400 l/mm gratings as drink coasters! Thanks!

  • NancyP

    Best photo April Fool’s Day joke of 2013. Hands down.

  • Wayne Morellini

    Good laugh. I was looking at a similar idea in the 90’s, but the problem anybody thinking about this as they read it, is how do you seperate out the colors of millions of different colored photons hitting the sensor at once, you might as well say it is a small cube powered by a dwarf holding it jumping on one foot. The foveon actually tries to achieve this. However, after all these years, and a lot more experience, I looked at it again, and dud come up with a solution (this is April the second).

    After my quick glance through some joke busting moments:

    – Scientist think how sensors work is foolish. I doubt their us a sane scientist that does. Existing sensors are an efficient compromise.

    – Existing sensor makers are interested in only more pixels rather than adding to the pixel itself. Much research goes into improving the pixels, including adding things to them to improve performance, which enables smaller pixels etc.

    – foveon actually tries to achieve this.

    – As above, separating out colors, you can get a average value maybe. (hmm, just thought of another way to do it).

    – The duck. Bayer is not going to produce such a large size of detail loss on the hires Canon, but comparing still mode to the lacklustre compression of lower res video mode would, or just shifting focus. It is obvious that it is the sane frame and neither the duck or the camera has changed position.

    – If the face and exposed skins burns out in cloths piercing nudie shots, people would not be interested in doing it, right, or increase sakes by ten. Anybody heard of hdr techniques, or adding hdr technologies used in other sensors, or getting the sensor to detect other ranges if too bright for the pixel. Actually the idea itself has merit, without long exposure. I thought of the possible mechanics of doing this a few tears back when solving another problem.

    – Expecting everybody else to wear thick woolen underwear, that’s funny.

    – The color periwinkle, they hear it every day, what’s periwinkle. Try blue or red clothing or lighting on Bayer, then displayed on 4:2:0 video signal.

    – dynamic range, you would add hdr sensor technologies.

    Just to make the less observant wake up that there is something fishy going on, you add all the metric etc light stuff keeping it out of the US, brilliant. I think I’m going to post a thread about this over on a certain camera forum and see who picks up on it by the time they get to the comments. 🙂

    I don’t like lying, but this was a pretty brilliant effort Roger.

    Thanks.

  • Steven

    Loved the non-metric light line. When I was an undergrad in Mech. Eng. I wished every day that the US would move to the metric system.

    I’d like to point out this is a real product though, used in Sigma cameras (they make cameras too).

    http://en.wikipedia.org/wiki/Foveon_X3_sensor

    To summarize: “…Bayer images [are] ahead on fine monochrome detail, such as the lines between bricks on a distant building, but the Foveon images are ahead on color resolution.”

  • Rhombus

    The time-energy uncertainty principle means that the more accurately you learn the photon’s energy, the less you can discern its arrival time. So Dr. Lirpa’s sensor requires long exposures, hence small aperture, and then all those nasty Nikon oil spots will show up!

  • Ben

    Damn it you got me good. It’s April 2nd here down under so I was off my guard. I even told a few people about this awesome tech coming. Wasn’t til I just now read the comments. Nicely played.

  • simon

    nice. 🙂

  • Richard

    Doctor Parc and Mr Yug were my favourites as well 🙂

  • Nqina Dlamini

    Had me going there for a moment. The single photon issue was niggling me but decided to believe you. Dam You got me good. The last part was hillarious.

  • I totally believed the article until I read the “When Can You Get One” and saw how it is difficult to use in non-metric countries due to measurement of light.

  • Mike

    I bought special bulbs from London, so I have metric light in my home studio. I am so excited!

  • Dan

    You had me until the very end, I was gaffing at the AF sensor bit, and the broken english conversations but it wasn’t till I saw the date when I got it. Best prank of the day so far

  • You left off my favorite feature–the accelerometer at the bottom of every pixel well so you can discard photons that are moving fast enough to be motion blurred at your current shutter speed!

  • I didn’t know that the light that hits the US parts of Canada and the British Isles was non metric! 🙂

  • If the unions manage to raise my income every April 1st, I hope I will eventually be able to afford to buy one. Otherwise I’ll just rent a camera with this technique, once they become available and autofocus- and oilspots have been sortet out. Don’t think, software will be able to autoretouch those supersharp dustbunnies or worse. Handlingtimes in postprocessing will increase dramatically, rendering this technology useless for the average pro. Maybe CIA will be interested, stands for Cameras 1 April anyway, doesn’t it?

    Greets
    Ralf C.

  • Daniel

    Well, I didn’t know that it is fools day today (I am somehow confused by the coincidence with the Easter Monday).
    Well, I woke up once I read about the non-metric light in USA 🙂
    Good one!

Follow on Feedly