Proof of Concept: Boost.GIL for HDR Processin

Davide
2012-04-19
2013-04-29
  • Davide

    Davide - 2012-04-19

    Hi all,
    I'm currently the head of the project Luminance HDR and I am going through an analysis of several image processing library in order to rewrite the underlying engine.
    At the beginning I though to write my image processing engine, and I have set up a project for it: http://code.google.com/p/libhdr/, however I am focusing more on data structure then algorithms (where I would love to work more). Currently I thinking to change the core of LibHDR to use GIL in order to speed-up my time-to-market.

    However, I still have some questions I could find an answer for:
    1. is there any way in GIL to get a progress report of the current algorithm?
    2. Does GIL support XYZ colorspace?
    3. How complicated is to write a new IO module? I would surely need EXR and RadianceHDR.
    4. Is it possible to add EXIF functionalities in GIL? Many HDR merge algorithms use this informations for the Radiance Map reconstruction.
    5. Is there any way in GIL to access the underlying vector of data, in order to ensure compatibility with C API?

    I would like to hear your comments on this idea, to understand whether it is a doable idea and whether GIL could work well for me and my project.
    Luminance HDR currently works in sRGB 32 float as default colorspace and most of the HDR tonemapping algorithms use XYZ as internal colorspace.

    Thanks,
    Davide
    http://qtpfsgui.sourceforge.net/

     
  • Anonymous - 2012-04-19

    Hi Davide,

    thanks for your interest in gil. It surely is a great library.

    Let me shortly introduce myself. I'm the developer of the upcoming io_new extention for gil which will bring great improvements for reading and writing images. The lib has been reviewed and approved inside the boost community where gil lives now. I'm in the final steps of adding required features so I can finally include the code into boost. Apart from io_new is a small helper extension called toolbox which has been reviewed, as well. It contains some new color spaces and other stuff.

    Here are the answers for your questions.

    1. I'm not sure what you mean.

    2. The before mentioned toolbox has xyz color space. Have a look here:
    http://code.google.com/p/gil-contributions/source/browse/trunk/gil_2/boost/gil/extension/toolbox/xyz.hpp

    3. It's not very hard to write a io module. I'll assist you. But I would suggest you have a look at io_new, since it will replace the old io extension.
    Here is a link to the reviewed code:
    http://gil-contributions.googlecode.com/files/boost_review.zip

    4. I'm not an expert in your field. What do you exactly need?

    5. Yes, it's fairly simple.

    for instance,

    typedef rgb8_image_t Image_t;
    Image_t img(640,480);
    Image_t::view_t::x_iterator data_it = view(img).row_begin(0);
    unsigned int* data = (unsigned int*) &gil::at_c<0>(*data_it);

    Beware this doesn't work for planar images.

    Your project sounds interesting and I would like to help out. Let me know when you have any questions.

    Regards,
    Christian

     
  • Mateusz Loskot

    Mateusz Loskot - 2012-04-19

    Hi Davide,

    I'm not yet an expert of GIL but I have used it and I have written a few lines of new code for GIL, so I will try to chime in with some ideas.

    First, I really like the idea and as keen amateur photographer, Luminance HDR is great!
    I believe it would be a great application and proof of concept for GIL.

    Now, moving to your questions.

    1. Most algorithms execute user-supplied functor, so it should be possible.

    2. Check the Toolbox extension, there is CIE XYZ support http://gil-contributions.googlecode.com/svn/trunk/gil_2/boost/gil/extension/toolbox/xyz.hpp

    3. I'd say, it feels complicated at the beginning and involves reading code of current GIL formats. But the basic ideas are very simple and you will grasp it quickly. Use this one or boost-users list and you will be helped.

    My suggestion is to start using the new Boost.GIL IOv2 and don't waste time with the old version.
    The new version has been reviewed by Boost and it should be released with Boost soon, within 2-3 months I guess.

    Here is document which will give you good overview http://gil-contributions.googlecode.com/svn/trunk/gil_2/libs/gil/io_new/doc/io.qbk

    I have stared implementing new format and here you can see what it takes to have basic scanline-wise reading
    https://github.com/mloskot/boost-gil-workshop

    4. I'm not sure what is the status of this.
    There has been some brainstorming on Adobe Wiki about exposing dictionary of metadata
    http://stlab.adobe.com/wiki/index.php/GIL_File_Format_Import/Export_Factory
    Internally, in format implementation, you can access and use metadata (e.g. PNG and TIFF formats use such properties)
    However, I don't think there is a unified API pass metadata across GIL utilities and algorithms.

    5. The fundamental concept in GIL is view and iterators. I haven't seen any means to access GIL image in terms of

    std:vector<byte> v;
    c_call_fills_image(&v);

    Looking forward to seeing your project happen!

    Cheers,
    Mat

     
  • Davide

    Davide - 2012-04-19

    I can't believe I've got a reply from Christian and Mat in such a small amount of time! You give me the push to try this adventure!

    I'll try to make more clear what I meant here and there.

    1. EXIF Data: It's important for me to retrieve Shutter Speed, Aperture, and ISO Speed for each picture I read from disk during the merge algorithm. Infact, these numbers are necessary to calculate what is called Average Scene Luminance, like:
    http://code.google.com/p/libhdr/source/browse/src/libhdr/exif/exifdata.cpp
    Currently I am using libexiv2 to retrieve the exif data, that I keep inside my LibHDR::Image (which I would replace with GIL image/view).

    2. For "progress report" I mean something that in the C world is implemented with callbacks. Many algorithms for tonemapping are actually quite slow and I need a way to notify the user that something is actually "happening". This applies to I/O as well.

    3. Adobe Factory: I've seen that portion of the documentation this morning and I found it quite interesting. I've trying to code something similar using the Factory class described by Alexandrescu in "Modern C++ Design": http://code.google.com/p/libhdr/source/browse/src/libhdr/template/objectfactory.h
    I am not sure whether a similar approach could work in the GIL world.

    Other random questions:
    a) I've seen here and there that Intel TBB is use together with GIL. I've done some work with TBB and I love it. It would be great if I could use it for such a project, but I am not sure on licenses issues. I know this is not the best place to ask, but if you can point me to some links, I'll read them. Currently I've planned to release LibHDR as LGPL, but I am open to suggestion.
    b) is this a better place than boost-users to talk specifically about GIL?

    I think a library like GIL + TBB can definitely bring Luminance to the next level :)
    Thanks,
    Davide

     
  • Anonymous - 2012-04-19

    Hi Davide,

    you are welcome and I hope you're up to some adventure. As I said I would like to help wherever I can.

    1. I think it's pretty straightforward to add EXIF to io_new. I could provide the skeleton if you want me to. Have you had a chance to take a look at io_new? Also I couldn't see where in your code you do the actual reading?

    What's the difference between EXIF and RAW? I think I have reading RAW somewhere.

    2. I think your request bears a lot bearings. Same goes for TBB facilities. But I'm not sure how a generic design would look like. One try was given here:

    http://dancinghacker.com/code/dataflow/dataflow/introduction/dataflow.html

    3. Do you mean something like dynamic_image? Look here:
    http://stlab.adobe.com/gil/html/giltutorial.html#DynamicImageSec

    As for TBB. I'm not an expert. But I usually think it's best to get a single processor version running and than tackle MP.

    Regards,
    Christian

     
  • Mateusz Loskot

    Mateusz Loskot - 2012-04-20

    Hi Davide,

    Regarding TBB, you may check interesting example of threading with GIL

    https://github.com/bogado/gil_threaded

    Regarding progress report, if you use functor-based algorithms from GIL, you can implement such feature within functors
    (e.g. transform_pixels ). Some algorithms can operate on chunks of image (see gil_threaded for example), then it is possible to report progress for each chunk.

    Cheers,
    Mat

     
  • Davide

    Davide - 2012-04-20

    Hi all,
    thanks for your help!

    @Mat: using functor would make the final design really similar to the one I am currently using for LibHDR, that just uses a simple implementation of the observer pattern: http://code.google.com/p/libhdr/source/browse/src/libhdr/coreobject.h
    I know, it's not the best solution ever, but I thought it was a good start and never bother to improve it :)
    Could it make sense to implement this thing with a policy class?

    @Mat: I had a look at the project yesterday, but it uses Boost.Thread. However, TBB works with iterator and it would fit nicely with gil::view, I guess.
    @Christian: I agree with you that a fully-functional single-core version must come first. However, as said before, if the multi-thread library works nicely with iterators, I think it should be pretty easy to obtain MT using TBB.

    @Christian: I think there is a subtle misunderstanding on what EXIF means. EXIF carries metadata (shutter speed, aperture and so on), it is not a image format. I'm currently retrieving metadata in LibHDR by passing the filename to ExifData's fromFile() function, during the reading process (http://code.google.com/p/libhdr/source/browse/src/libhdr/io/jpegreader.cpp#372 for jpegreader).

    @Christian: I forgot I would love to implement a RAW reader and possibly a DNG writer.
    @Christian: I am looking right now at your implementation of io_new: a simple skeleton would definitely help me to push my learning curve, but I don't want to still any of your precious time, because I'll probably take advantage of it when I will seriously need it :)
    @Christian: I am looking inside io_new and I've found any evidence of color correction. Do you think is doable in any way? Maybe using a different ConversionPolicy class in the class "reader"?
    In Luminance HDR, one of my collaborator is currently doing a lot of work to integrate LCMS into each reader, in order to properly color-correct every input image and I would love to keep this feature into the new GIL-world as well.

    Final general question: do you think is better if I work in the "boost::gil" namespace, or I work in the "libhdr" namespace? What's the best practice in this case?

    Best,
    Davide

     
  • Anonymous - 2012-04-20

    Hi Davide,

    As far as I know Olivier once created a raw reader for io_new. It's based on libraw, or something. I have sent him an email asking him to share the code.

    I think it might be a bit overkill to read metadata like we do for image data. I think a simple wrapper to EXIF data should do.

    Regarding, the ConversionPolicy, yes, it's suppose to be a functor detailing the conversion from one color space to another. But the basic image types for each image format is defined inside supported_types.hpp.

    I suggest you use boost::gil::hdr. What do you think? I mean the end result should be an extension to for gil. Correct?

    Enjoy your weekend,
    Christian

     
  • Davide

    Davide - 2012-04-21

    Hi Christian,
    I would be great if I could look at that code. We use libraw as well in Luminance HDR and I would be happy to use the same library into this new project.

    I'm trying to figure out how I can properly  set up a class that allows me to read the EXIF data, however I am having trouble to compile a simple example with GIL (you have a defect report in Google Code). I've also set up a new branch inside my Git repo for LibHDR and I've started following your guide on the new io system . Any comment is welcome: include/libhdr/io is probably the most interesting folder right now.
    The repo is currently a mess, because the old stuff is still there, but it will be gradually remove if this project takes off.

    ConversionPolicy is surely something I will work on a lot, I guess.

    Do you accept patches to gil-contributions?

    https://github.com/davideanastasia/LibHDR/tree/gil-experimental (which is a mirror of http://code.google.com/p/libhdr/source/browse/?name=gil-experimental, which unfortunately doesn't seem to be working properly right now!).

     
  • Anonymous - 2012-04-21

    Hallo Davide,

    I have fixed the malloc.h issue. Please send me your email so I can add you my project. You can then apply patches.

    I'll let you know once I have my hands on the raw code.

    Christian

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks