locked
Image stacking for noise reduction RRS feed

  • General discussion

  • Hello everyone! I’d like to suggest an addition to future ICE updates: image stacking for noise reduction & image quality purposes. 
     
    Image stacking, for long well known to astrophotographers and then pro photographers of a more generic sort but still technologically savvy, is becoming an increasingly popular concept to the point where even smartphone users without any overt photographic inclination have begun using it without actually realizing what it is, much less understanding it: they just know that their phone’s app tells them to frame, press the button, try keeping steady while the phone takes several pics, and voilà, a minute or two later the app comes with a higher quality picture than the stock or previous camera app, which didn’t include this “multi-shot thingie”, could. 
     
    ICE already does what is arguably the hardest part about it: successfully aligning many hand-held shots. I know from experience that many programs and methods to perform image stacking either use so unreliable aligning algorithms that the result is actually worse than a single picture, or else are so complex as to discourage the more casual user who can’t figure out why (or, for that matter, how) to baby-sit the entire process through unintuitive user interfaces. 
     
    An example of the former would be PhotoAcute, otherwise an amazing program that actually goes beyond mere pixel averaging and promises (and indeed often delivers) increased resolution, but relies on such hit-or-miss aligning algorithms, and anyway constrains itself so much to camera profiles, that it often turns out all but useless to everything not done with one of the few supported camera models AND a tripod (which kind of misses much of the point of image stacking -if you had one, you’d simply set a longer exposure time in exchange for keeping the ISO low); as for examples of the latter, I’d go with Hugin, AutoPano and a whole lot of other panorama-makers out there, which are amazing if you have eight hours to read the entire manual and then a couple more for each photographic set to stare at countless anchor points, RMS numbers and whatnot, before you come up with something that often still needs exporting and being processed with other programs to perform the actual averaging. So far my best option has been Photoshop CC, surprisingly good in its auto-alignment and not all that difficult to use... but of course this is still an all-purpose and, for some, too expensive program. 
     
    As I said, ICE already seems to have the hardest part done: a highly reliable realignment algorithm even for hand-held sets without any kind of tripod; all it needs now is an option to, instead of blending the aligned shots at just the overlapping areas, actually fully average them pixel by pixel (perhaps offering an option to discard areas not covered by all the shots or, for that matter, averaging these only with the shots that do cover it). I don’t believe it’d need to go as far as offering actual resolution enhancement, as PhotoAcute does (though I’m aware that MS research has also tackled the concept) or, for that matter, any of the fancy stuff of similar programs, such as exposure bracketing-based HDR shots or ghosting removal (moving subjects that appear in only a few shots)… at least not yet; noise averaging in and on itself would already be a huge and very welcome improvement.
    • Changed type Landaree Levee Tuesday, February 10, 2015 6:16 PM Make it clear it’s a feature suggestion
    Tuesday, February 10, 2015 6:15 PM

All replies

  • I stumbled upon this recent work of MSR a few days ago which is exactly what you are proposing to add to ICE.

    Fast burst images denoising

    YouTube video


    • Edited by Chrysler Tuesday, February 10, 2015 8:58 PM
    Tuesday, February 10, 2015 8:58 PM
  • Another post suggested a complimentary feature to this (Super Resolution).

    I'd like to cheer on these two requests, and give an example of an application.

    One of the things I dabble in is underwater photography... not scuba, but cameras that are planted on the seafloor at great depths, and take pictures in a fixed direction at regular intervals. This is not new technology, but I've been involved in a couple twists.

    Many years ago I developed a rotary underwater camera, taking 'formatted' panoramic stills in bursts, in time-lapse operation. More recently I've developed a time-lapse video camera... takes video clips (fixed direction) at regular intervals.

    What I'm starting to work on is a panoramic video time-lapse camera... Take a relatively inexpensive 'action' camera, and periodically capture video clips while rotating the camera (and lighting). Using ICE, it's easy enough to generate a panorama image from each clip, and then combine using FFMPEG into a time-lapse movie.

    Given the over-abundance of data from a slowly changing video, I'd LOVE to see ICE convert the movie into a higher resolution and lower noise panoramic image. Spin faster to save power and memory, or spin slowly to over-sample the environment and yield a higher quality result.

    Is there anyone working in this direction who I would talk with?

    Sunday, October 4, 2015 11:07 PM