LensRentals.com

What’s All This RAW Hack Business?

Posted by

You’ve more than likely heard about the Canon 5D Mk III RAW Hack. We too have been keeping a close eye on the Magic Lantern (ML) team as of late. In the last 4 months they’ve enjoyed their greatest triumphs since initially cracking the 5D Mark II’s code almost 4 years ago. It’s been a wild ride, and even though it’s far from over, we’re ready to join in. The LensRentals crew has been independently testing the various new firmware builds all summer long and we feel it’s now safe to recommend them to our customers.

Fig. 1: Artist rendering of what the Magic Lantern team might look like.

As of last Friday we are renting a pre-hacked version of the 5D mark III that includes one of the 1000x Lexar cards required to get the most out of it. We are not saying the Raw Hack is perfectly stable, actually it is wise to keep a backup camera on hand; however, at this point in time we don’t think the hacked cameras are any less stable than the RED Scarlet, and that’s one of the most popular cinema cameras we carry. To celebrate the recent achievements of the ML project we’ve made a video and this blog post to acquaint the uninitiated with some of the amazing features they’ve implemented this summer. This is not an in-depth technical examination, but rather an attempt at introducing ML to a broader audience. So if you’ve been curious about why those ML posts have recently dominated your favorite camera blog, please read on.

RAW Video

Since last May experimental alpha builds of the ML hacked firmware have been circulating on the ML forums. These new versions feature the ability to capture RAW video with the 5D Mark III (5D3), hence the name “The 5D RAW HACK”. This wizardry is accomplished by diverting raw image data away from the image processor while it’s held in the camera’s internal buffer; instead of getting compressed by the image processor, the visual information is stored directly onto the camera’s memory card in a .RAW file. Later, that file can be converted into Adobe Digital Negative (.DNG) image sequences by a free program called RAW2DNG. Each 1920×1080 14-bit uncompressed .DNG file is a single frame and when viewed in sequence, at the proper frame rate, you have a video. Think of it as shooting photos in continuous mode, silently, at 24fps.

Fig 2: Technical illustration of the Raw Hack in action.

At this point things will become very familiar to anyone who has used a Blackmagic Design Cinema Camera, but there will be a bit of a learning curve for everyone else. You will need to organize the tens or hundreds of thousands of .DNG files known as your project and get to editing. We cover this a bit more in the video. It’s true that a post production workflow using unwieldy CinemaDNG sequences can be a bit challenging, not to mention tedious; however, in this case we think it’s absolutely worth it. Also, try to keep things in perspective. This workflow is some sort of miracled, reverse engineered, workaround for getting the most out of a much beloved camera. The Blackmagic Design Cinema Camera shipped like that.

[for nerds] Many people have been inquiring about possible shooting resolutions. Interestingly enough, you can set your own. In the ML menu there is a resolution option, in some previous builds you could dial in a wide variety of values; although, as of mid-August, the options are locked to just the most popular ones possible on fast Compact Flash cards. In theory, if the cards were fast enough, we’d all be shooting the 5D MK III’s native sensor resolution of 5760 × 3840 pixels, but they are not, so we are not. Fortunately, no matter what the resolution is set to we still get the whole sensor image. That’s right, no sensor cropping, but then how is the image is getting scaled? Typically, interpreting the unprocessed image data, in this case resizing, would involve a process called debayering; however, if the files produced by the RAWHack are truly RAW, they couldn’t have been debayered. So, how does the camera downsample the images without debayering? Well, only Canon knows that… but on his splendid blog Prolost, Stu Maschwitz theorized that it’s a combination of vertical line skipping and horizontal row binning. [/for nerds]

*It should be noted that there is no audio recorded with the Raw Hack at this time, but they anticipate getting it working again in the coming months.*

Flicker-Free ETTR Time-Lapse

Around the same time the RAW Hack started making the rounds, the ML team also finalized the long in development Flicker-Free ETTR Time-Lapse (FFETTRTL) feature. Using the intervalometer, ETTR, Bulb Ramping, and deflicker options in unison this feature finally makes in-camera flicker-free timelapses possible… almost. There is a script that the files need to run through in post, and it takes a while, but other than that it’s mostly automated from capture to completion.

To understand the FFETTRTL feature, we should start by talking about the ETTR part of that initialism. “Exposing to the Right” (ETTR) is a photography technique pioneered by Michael Reichmann and Thomas Knoll more than a decade ago. The term “Expose to the Right” itself is a reference to image histograms. A histogram is a linear graphical representation of the tonal distribution of an image (huh? read this). In a histogram shadows are represented on the left and highlights are shown on the right; therefore, following a line from the left to the right of the graph will take you from dark to bright (Fig 3). So when someone says “expose to the right”, they mean “lean toward overexposing the image, rather than a balanced image exposure.” This is where we get to the actual principle behind ETTR. More usable visual information is contained in the tonal values of highlights than is available in shadows; therefore, it is sometimes prudent to overexpose (without clipping of course) an image so you can balance the exposure later while retaining more detail in the process. Basically, it’s collecting the maximum amount of light to get the best performance out of a camera’s sensor. That’s a gross oversimplification of the whole idea, but should be fine for our purposes. One last thought before moving on, even though ETTR is a proven technique, that doesn’t mean it is always the “best” or “right” way to shoot. When possible, think ahead to how you plan to post process an image and shoot accordingly.

Fig 3: Exposure on the x-axis of a histogram.

So how does ETTR fit into this new FFETTRTL feature? ML offers an Auto ETTR option that will let you shoot in manual mode, but bump up your exposure to the edge of clipping every time you take a picture. When Auto ETTR is used with ML’s built in intervalometer, the camera will learn from the pictures it is taking and adjust exposure (ISO & Shutter Speed) accordingly as it goes. This is accomplished by having the camera evaluate the RAW histogram of the previous image it took before taking the next picture. Each image created in this way will have a matching .UFR sidecar file containing extra information about exposure and the image sequence. Once acquisition of the time-lapse is finished, all the images along with their matching sidecar files are run through a free script available from ML. If done correctly, you will end up with a smooth transitioning image sequence free of flicker.

Dual ISO Dynamic Range Hack

The most recent breakthrough came in July with the surprise unveiling of a Dual ISO Hack. This newest feature uses multiple ISOs to capture a single image resulting in significantly expanded dynamic range. The 5D Mk III has a dynamic range of around 10 stops depending on ISO settings. So by exposing at two ISOs about 4 stops apart at the same time the image gains those 4 stops on top of what it already has. You’re left with almost 14 stops of dynamic range and it’s a pretty massive improvement. Also, did we mention it works with the RAW video shooting?

One of the really unique things about this workaround is that it’s using both ISOs in a single frame. Most high dynamic range (HDR) solutions involve multiple frames being combined. That is true of HDR photography, RED’s HDRx mode, and even ML’s own DSLR HDR Video Hack from last year. Ironically, this innovation was inspired by a perceived weakness in the camera’s design. The 5D Mk III’s sensor performs differently depending on what ISO it’s set at. Some of the higher ISOs handle shadow detail better, but low ISOs definitely handle highlights better. This is not typically the case with most DSLRs, so the 5D Mk III is unusual in that respect; nevertheless, the ML team uses this fact to their advantage in a really creative way. They have the camera capture images interlaced with the 2 different ISOs alternating every 2 lines (Fig 4). For example, let’s say you’re shooting with ISOs 100 and 1600. Pixel rows 1&2 would be ISO 100, then pixel rows 3&4 would be ISO 1600, rows 5&6 100, rows 7&8 1600 and so on.

Fig 4: Dual ISO photo prior to de-interlacing. 

Now here comes the truly brilliant part, the image can be de-interlaced and maintain its 14 stop dynamic range. After taking an image it is quite obviously interlaced and must be put through another script the ML team came up with called Cr2HDR. It takes a minute and then spits out the de-interlaced image. At first the image will appear very dark and underexposed; however, once loaded in an image editor like Adobe Camera Raw you can bring the exposure back up and be delighted by how little noise is present in those recovered shadows.

The bad news is, in all of the finagling to wring an extra 4 stops out of the shadows (literally) you will have sacrificed about a quarter of the image’s vertical resolution. The ML team stands by the opinion that it is nearly imperceptible, and in most situations I agree (Fig 5). Especially if your intended display is a computer monitor, you’ll get away with a bit of murder and it will require a fair amount of zooming and scrutiny to find the seams. On the other hand, if you intend these pictures for high resolution purposes, like large prints or significant post cropping, it will be a bit more difficult to get by unnoticed. Very clean solid colors work great as do exceptionally busy materials like grass or concrete, but simple patterns will give you trouble. Also, as is usually the case, the deinterlacing process introduces a lot of aliasing and moire making it difficult to use for video. As a last thought, I cannot stress enough how well this feature works and useful it is as long as you’re careful with it.

Fig 5: 400% Crop of a test chart after de-interlacing… yeah, that’ll do.

ML team member A1ex released a very detailed technical breakdown of the feature; it includes both an explanation of the basic science behind imaging sensors, as well as how they exploited these principles to pull off their hack. It’s not for the casual photographer, but I would consider it a must read for any camera nerd. Terrific stuff!

Wrap-up

All of this stuff might sound complicated; however, thanks to a thriving community, the learning curve isn’t so bad. Just be aware that there is a learning curve and becoming proficient with these new features will take some practice. A quick Google search can resolve most concerns; if that doesn’t work, one can simply go back to the source and search/post in the magic lantern forums. If you’re still left scratching your head, give us a call and we’ll take a crack at it. We’ve been enjoying these new features for a while now, but it’s exciting to finally be able to formally recommend them to our customers. Thanks for reading, here’s a goodie bag for your time; it has one short .RAW clip and one Dual ISO image file to play with.

If you would be interested in more articles like these or would like us to go into greater detail about any of the topics mentioned, please let us know. We look forward to answering your questions and continuing our discussion in the comments section. Happy shooting!

Kris Steward
Video Technician
LensRentals.com

 

Meanwhile at Canon…

 

34 Responses to “What’s All This RAW Hack Business?”

Brandt Steinhauser said:

As a Nikon shooter I have no use for this, but I do love the photos from HACKERS!

Sean said:

Brandt, as far as I can tell, this is just for the Canon 5D Mark III, so even though I shoot a 6D, the version of Magic Lantern being promoted in this article didn’t mention my camera 1 time.

LensRentals Employee

Kris said:

Thanks Brandt, there are definitely some Hackers fans around here.
@Sean We’re only renting the 5D3 version right now, because it’s the only one stable enough for us to confidently recommend; however, most current generation cameras from Canon work in one way or another. If you ever want to try it on your 6D, check here for details.

Jeff Gelzinis said:

The first pic was great, but the shot of Canon HQ killed me. Genius. Great article too, as always. Hackers of the world Unite!

Jao said:

Kris, just a quick correction suggestion on the ETTR part. It works not because of the tonal value issue but because of the stochastic (shot noise) nature of photons. Longer exposures lead to higher signal to noise ratios because of the higher number of photons hitting the sensor sites which inherently lowers the SNR because the shot noise is lowered and the readout noise will be relatively lower. The original ETTR article got this wrong and unfortunately their (unphysical!) explanation stuck. The correct explanation is given here: http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#ETTR

Tom said:

Can you provide a direct link to download the firmware version of ML, that would be great.

Wayne said:

Wow, that’s a lot of info. I don’t know how you folks find time to ship out rentals with all the investigations and studies you do.

LensRentals Employee

Kris said:

Due to the fact that recent builds have been changing quickly and most users will require a significant amount of information prior to installing it anyway, we have chosen not directly link to a download of the firmware; although, we do link to this page, which can get you downloading the firmware in just two more clicks.

L.P.O. said:

Jao wrote:
“Longer exposures lead to higher signal to noise ratios because of the higher number of photons hitting the sensor sites which inherently lowers the SNR because the shot noise is lowered and the readout noise will be relatively lower.”

I think you mean “heightens the SNR”, in other words, “make signal-to-noise ratios better”. Also, shot noise does not get lower with longer exposures: with thermal noise it gets higher. The trick is that shot noise gets higher _slower_ than the signal. Then there’s fixed-pattern noise that increses linearly with longer exposure times and prevents you from getting a better SNR after a certain point, but which fortunately can often be removed more or less perfectly with dark frame subtraction.

I’m sorry to be as smartass, but your correction of the “unphysical explanation” didn’t exactly make things better.

As for the RAW hack, it is very interesting. When they get it stable for the 5D Mark II, I surely will try it out!

Walter Schulz said:

Corrections:
ML is not firmware! It doesn’t replace Canon’s firmware!

Imagine Canon’s firmware is a computer’s operating system (Windows 7, Linux, Mac OS, etc.) and Magic Lantern is a program running at startup by the operating system and staying in memory while the operating system is up and running.

The only hack done to the camera is to make Magic Lantern loadable at startup.

For most users this doesn’t make much of a difference, of course.

Ciao
Walter

Samuel H said:

Great to hear. I’ve been a ML fan since I bought my T2i to shoot video 3 years ago.

One great thing you could do to help the ML community is to try to get a report from each rental, to see if they had or didn’t have any problems with their camera. (a comparison with, say, the Scarlet, would be great, but I’m not sure it’s safe to publish, plus yes, I read the comment on the lens repair data blog about having competitors and not being too eager to give them too much valuable information)

Samuel H said:

Also, I should point out: for the dual-ISO thing, using two values that are not that far apart leads to much cleaner images. For example, if you use 100-400 instead of 100-1600, final images will be basically clean of artifacts, and you still get 13.5 stops of DR (against 14.7 for the 100-1600 combination; always using dxomark numbers, which are pretty close to what a1ex has measured).

Daniel said:

I do find that Hackers image amusing as the black guy on the left is called Lord Nikon in the film.

Andy600 said:

I just wanted to clarify a couple of things regarding Magic Lantern. It’s a common misconception but Magic Lantern is not actually a hack. It’s software that is loaded into memory and the only alteration to the firmware is a flag that tells the camera to load software from the card.

Nearly all Canon DSLRs from the 50D to the 5D MkIII are capable of recording raw video however, the write speeds required to record Full HD is around 80MB/s and can only be achieved on cameras with larger internal buffers and a CF card. Cameras with an SD card are limited to approx. 20MB/s or in the case of the 6D 40MB/s which restrict the recordable frame size and/or length of record time. Currently the 5D MkIII and (pre-5D MkII) 50D can record continuous Full HD to 1000X CF cards although the 50D can only do this in crop mode which is 1:1 pixel sampling. This increases the crop fact by 3x on top of the APS-C 1.6x crop so everything bar the very widest lenses will become telephoto (although the original focal length will remain the same). The 5D Mk III can shoot full HD in normal mode and is capable (at low FPS) of upto 3.5k recording.

When in crop mode the cameras record with virtually no aliasing and moire.

The 7D is capable of raw video, however it is currently limited to under 600px vertical resolution but should be fixed soon.

There is a full list of cameras raw capabilities here: https://docs.google.com/spreadsheet/ccc?key=0AgQ2MOkAZTFHdFFIcFp1d0R5TzVPTVJXOEVyUndteGc#gid=5

Auto ETTR is a very useful new module, however it is better suited to stills shooting as it alters shutter speed. It can be linked to the Canon shutter speed setting but in bright conditions it will increase speed and thus reduce shutter angle. It can be used more effectively for video using a variable ND filter however it is best to use the either the histogram or zebras set to raw for setting exposure. Yes! The histogram and zebras now accurately measure raw exposure levels allowing you to ETTR accurately.

One final thing. Magic Lantern is soon to get it’s own raw video format (.mlv) which is currently in development. The new format includes full metadata with user text input in a similar fashion to the BMCC.

Follow the forum for more info :)

Nqina Dlamini said:

Meanwhile at Canon. Killer.

Chuck Jones said:

Great stuff Kris, thanks for letting us know. I think it is fantastic that your renting these now! I’ve got a couple of quick questions though. What is the typical learning curve to use ML? By this I mean should your renters (like me!) plan to add an additional day of rental to learn to use it, or is it strait forward enough to pickup in a few minutes?

I’ve yet to load ML onto my own 5D3, mostly due to the fact I also own an Atomos Ninja 2 field recorder that requires the latest Canon firmware build (1.21 as I recall) to provide a clean HDMI out. Any idea if ML supports a clean HDMI out yet?

Again, thanks for posting this and please thank Roger for letting you all turn into “ML Hackers!”

LensRentals Employee

Kris said:

@ChuckJones Thanks for the kind words. I would expect at least one full day to learn the workflow from end to end; lots of shooting, followed by lots of post. Just to be safe, I wouldn’t embark on any kind of serious work until successfully shooting, editing, and delivering a test video.

derek said:

hey, just for this ML3 thing , I got a Canon in addition to my Nikon kit.

now , I am addicted to it.
Thanks for your hard work and your support, I was kind of insecure using it but now feel more secure after reading this amazingly well written article.

David said:

Great article. I don’t shot Canon, but love articles like these. Please keep them up. Also I am a full tech geek and Scientist, so I love more details.

Now that you offer to rent this, I might consider it.

Frank Kolwicz said:

So, did I understand that right: the “hack” give you essentially a 24 fps in full res RAW mode, as if that was the camera’s burst frame rate, for stills? For a 5dII?

Samuel H said:

@Frank: no, you get a stream of 24fps 1920×1080 RAW (DNG) files. Full res could be possible if the CF controller and cards were fast enough, but the datarate would be insane (it’s 83 MB/s with 1920×1080, so 750MB/s at full res; and that’s megabytes, not megabits; a 128GB CF card would hold less than 3 minutes of footage).

Frank Kolwicz said:

Samuel H:

Well, I’m thinking of 1 second bursts or less, so total Meg wouldn’t be a problem, datarate would be something else, though.

Thanks

Samuel H said:

If you’re fine with a short burst, then it’s going to be fine: the camera takes the pictures and stores them in the buffer until it’s full, then offloads that to the card. I think right now it’s doing 24fps with a central crop of 3584×1320. There may be a sensor readout bottleneck here too, so I’m not sure they’ll eventually get it working in the full sensor.

Nic H said:

Thank you very much for helping spread the ML word. I’m a computer guy just as much as a camera guy and this type of stuff lets me indulge on the best of both worlds. I’ve been toying with ML (as well as CHDK & others) for a few years and I’m awed at the amount of time and complete genius that these teams put into their software. I’m looking forward to “dual ISO” on my 7D.

LensRentals Employee

Kris said:

@Brian B – It certainly does! You will need to follow the steps in this guide to set it up.

David said:

Kris, there is a very interesting paper about ETTR that explains how it works, and why Michael Reichman got it right, but for the wrong reason.

http://theory.uchicago.edu/~ejm/pix/20d/tests/noise/noise-p3.html#ETTR

It’s a long paper, and highly technical, but it’s really good information for anyone that wants to dig in deeper.

Lynn Allan said:

As of late August, 2013, ML has gotten their DUAL_ISO module to work on the 5d2, and perhaps other DSLR’s. It turned out that their early evaluation that only the 7D and 5d3 could interlace iso’s was incorrect.

And … that’s a good thing.

fletcher murray said:

Hi,
I’d like to recommend my filmmaker/students rent the hacked 5D from you, if they don’t want to risk their camera.

I’m having trouble with the workflow. I watched you great video. everything working fine through RAWmagic. I can see the DNG files on my MacBook Pro in their folder window. But when I drag into Davinci Resolve I just see a red noise movie. Did I miss a step? (When I launch resolve it warns “Your GPU memory is full. Try reducing timeline resolution or number of correctors”. I’ve never used it before. What’s the deal?”

David Arango said:

Hey guys! Great post! I’m about to give ML a try on my 7D and I’m thinking on purchasing new CF cards, I’m torn appart between the Komputerbay and the new Sandisk Extreme Pro 1066x, have you tested the later ones? If you could do some benchmarking it would be SO HELPFUL.
Thanks in advance!

LensRentals Employee

Kris said:

Thanks David! We haven’t received our Sandisk cards yet so the jury is still out, but the Komputerbay cards are pretty unreliable. Only every third or fourth card hits advertised speeds, and those that actually perform up to spec fail after a few months anyway. Right now the only cards we recommend are the Lexar 1000x. We hadn’t ever given much attention to Lexar prior to renting these, but they’ve proven very trustworthy after all that we’ve put them through.

Ron Miller said:

Great movie! Any update of this hack?

LensRentals Employee

Kris said:

We have been renting cameras loaded with Magic Lantern for two months now and it’s been very successful. Some have struggled a bit with the learning curve, but most customers have risen to the challenge and really enjoyed the camera. We haven’t had a single problem out of the fifteen hacked cameras we rent and the hacked 5D3 is still my goto camera, even with all of the other gear available to me. If you were asking about Magic Lantern itself, the answer is… not really. There have been some new builds in the past two months, but no major changes yet.

Leave a Reply