Feedback for the 2.0 direction

Jesse
2009-03-14
2013-05-02
  • Jesse

    Jesse - 2009-03-14

    I was wondering if there would be any open discussion on the 2.0 internals. Or at least a way for other developers to check out what's being done so that their work will be forward compatible. I had an idea about how to abstract the process of colour conversions while I was writing a patch, but as I was in the middle of moving at the time, I hadn't the opportunity to experiment with it in code, nor to finish the work I had promised to start earlier in the  month.

     
    • Denton Woods

      Denton Woods - 2009-03-14

      I had not really thought about it, but having a discussion on the 2.0 internals is a great idea.  I guess we could just discuss in this thread right now, and I will link it from the homepage.

      You can see my current changes in the branches/DevIL2 branch in Subversion.  The biggest change that I am making at the moment is dealing with ILimage's instead of integer names.  The whole idea of binding images will be gone in the next version.  This way it will be possible to load an image while performing some kind of processing on another (or loading two at a time, though that will be harddrive-limited).  So DevIL will hopefully be threadsafe by the time version 2 is released.

      I am still trying to decide what to do about being able to load from file streams and memory simultaneously.  I am thinking that ilLoadF should have an additional parameter that is a pointer to a structure that has the function callbacks that you want to use, similar to the callbacks passed to ilSetRead.  If this parameter is NULL, it uses stdio.h functions.  ilSaveF would have similar functionality.

      The next issue is everything in il_states.cpp.  I want to get rid of the state stack and just keep one copy of the states.  I am trying to determine the best way to make this thread-safe.  There should not be any reason that a program will change and/or read the states lots of times, but maybe I am wrong on this.  If programs were changing the states many times, and different threads needed different states, then it would make sense to have the user pass a structure (or a pointer to a DevIL-internal structure) with the states.  However, this makes the library much harder to use, and it would definitely be a pain in other languages.  I would rather just have the one set of states.  Do we really need to use mutexes for these?  I am thinking that to be safe, we probably should.

      One minor thing that I just did was remove the Bpp parameter from ilTexImage, since it was totally superfluous.

      All source files now have .cpp extensions as well.

      I will try to post some updates in this thread, and hopefully some people will give their comments and suggestions.

       
      • Denton Woods

        Denton Woods - 2009-03-14

        So I just ran across another design issue.  In the SVN repository, I have redefined ilLoadImage as:

        ILimage* ilLoadImage(ILconst_string FileName);

        So when you want to load an image, you can just do:

        ILimage *Image = ilLoadImage("test.tga");

        The alternative would be to define ilLoadImage as:

        ILboolean ilLoadImage(ILimage *Image, ILconst_string FileName);

        Then you would do something like:

        ILimage *Image = ilGenImage();
        ilLoadImage(Image, "test.tga");

        I am liking the first method better, but the second method allows you to reuse the same Image structure.  The difference internally in ilLoadImage is that the first method creates a new ILimage with ilNewImage and fills it, while the second method uses ilTexImage on the Image parameter and fills it.

         
        • Denton Woods

          Denton Woods - 2009-03-15

          For the moment, I am going to stick with the second method.  It is similar to the current way of using images with DevIL, so it will require less rewriting by developers wanting to switch from DevIL1 to DevIL2.

           
    • andreaplanet

      andreaplanet - 2009-03-20

      I was just looking for Image processing library and found devil. So I'm newbie but reading about your question I like to write some words.

      Multithreading:

      To allow full multithreading (working on multiple images at the same time on different threads) you must provide a 'state' parameter for each function. It's up to you if a pointer a number or whatever. Else how can I have use internal states at the same time?
      The question is how you pass the 'state' parameter to the functions? For C style functions there is no other way than passing the state parameter / reference to each function. But with a C++ wrapper you can create a simple class encapsulating the state parameter (or a reference to it). The wrapper will call the internal C style function with the state parameter. With the wrapper the syntax of the existing functions remain the same, you only have to prefix it with the state class.

      Example existing function:
      ILboolean ilLoadImage(ILimage *Image, ILconst_string FileName);

      New functions:
      // private internal function
      ILboolean ilLoadImage_internal(ILimage *Image, ILconst_string FileName, ILstateStruct *state);

      // classic call without threading
      ILboolean ilLoadImage(ILimage *Image, ILconst_string FileName)
      { return ilLoadImage_internal(image,FileName,state_single_threaded); }

      // c++ wrapper
      class ILState
      {
        ILboolean ilLoadImage(ILimage *Image, ILconst_string FileName)
        { return ilLoadImage_internal(image,FileName,this->state); }
      }

      Who develop a multithreaded application already handle the synchronization between his own data. So you don't have to do that in your library. Just suppose that for each 'state' the calls are like single threaded. In other words there are never two calls at the same time on the same 'state' parameter. This will maintain the design of your code and library simple and without an overhead. If I have 4 threads working with images I will create 4 state parameters, one for each thread. Maye even more if I want to maintain different state parameters without setting them individually.

      Accessing data from different threads must always be synchronized, if at least one of those threads updates the data.

      The last question is if you want to allow that a 'state' parameter can be used on different threads (but never you will get two calls at the same time). Let's say I create a image on one thread, process it and when the job is down I want to show it on a window using another thread, doing some resizing for example.  Allowing this is a plus of course.

      just my 2 cents

      Andrea

       
    • Jesse

      Jesse - 2009-03-24

      As Andrea said "...working on multiple images at the same time on different threads... you must provide a 'state' parameter for each function. It's up to you if a pointer a number or whatever."

      I was thinking about thread communication using domain sockets. If you think of the thread function as (I'm inclined to pthreads here)

      void *image_processing_thread( int sock_fd )
      {
          //.. initial code of some kind

         union {
           char buf[1];
           int msg_type;
           struct {
             // msg specific data
           };
         };

         do {
           recv( sock_fd,buf,sizeof(int),0 );

           switch ( msg_type ) {
              case SOME_REQUEST:
                  size_t sz = recv( sock_fd,buf,SIZEOF_REQUEST,0 );
                  // process this request
                  break;

               // whatever requests this object handles
            }
         } while ( msg_type != KILL_THREAD );
      }

      The pseudo-code is somewhat naive but the approach has several advantages.

      1) Using threads this way makes the code thread safe.
      2) Global "context" data should not be necessary. Image data would be thread local and accessed through message requests.
      3) Sockets (like pipes) can be chained together, so that the output of one image process is the input of another.
      Complex image filters can be created this way, and user-made filters can be hooked in.
      4) It wouldn't take much extra to become network transparent (for what that's worth)

      However, there are some big drawbacks to this method.

      1) It's a huge design change. Those are always painful.
      2) It requires a lot of cross-platform wrapping (Windows threads are different than pthreads though it's socket's API is quite similar)
      There are probably more points to add to that.

      From what I understand this method is not uncommon. I tried something similar with audio once. Anyway, posted this more to spark thought than as an actual suggestion.

       
    • Denton Woods

      Denton Woods - 2009-04-06

      Thanks for the suggestions, Andrea and Jesse.  I will probably end up doing something similar to what Andrea proposes...

       
      • Anton Sukhinov

        Anton Sukhinov - 2009-04-13

        Why do you need to pass a state in the all functions?

        Just do as all multithreaded libraries do: query the current thread id inside the functions, and select state according to this id.

         
        • andreaplanet

          andreaplanet - 2009-04-13

          you are passing a state parameter to each internal function, even if you use the current thread id as an implicit parameter (no need to specify it in the parameter list).

          The drawback with this solution is that you can't make call's between threads. For example load an image in one thread and process it in another thread. Everything must be handled in the same thread.

           
    • Denton Woods

      Denton Woods - 2009-04-27

      I just thought I would point out that this weekend, I started adding an ILstate structure to DevIL, similar to what Andrea suggested.  It's going to take awhile to get everything to use it, and the SVN branch of DevIL 2 actually will not compile fully at the moment.  You can look at the partial update of IL.h (and more) in SVN:

      http://openil.svn.sourceforge.net/viewvc/openil/branches/DevIL2/include/IL/il.h?revision=1655&view=markup

       
    • Mihail Naydenov

      Mihail Naydenov - 2009-06-21

      First, congratulations for the decision to start Devil 2. It was about time!

      And indeed multithread-safety was the single reason I switched to using FreeImage years ago.
      But this is the place to note, that there is enormous difference between being thread-safe and being multithreaded...the conversation so far blurred the two a bit I think.
      IMHO DevIL MUST be thread-safe (like FI is), in ordeder to be used it in modern multithreaded programs, but should it be multithreaded also?
      I doubt it!
      What will be the benefits, considering the (considerable) complication of the code, multithreading brings, that are not possible from within user code with a multithread-safe routines?
      Async loading? - achievable in user code + thread safety
      Better performance of filters/adjustments? - maybe, but CPU image manipulation in general is a bit "old school" right now...

      Don't get me wrong, it will be lovely to see multithread functions. But I would (many times) prefer thread-safety + a nice, clean (c++) API, and MUCH sooner release instead!
      For me multithreading will be a gimmick.

      that`s it:)
      MihailNydenov

       
    • Javier Taibo

      Javier Taibo - 2009-08-12

      I just found this thread because I was using DevIL in a project in which I now need thread-safety. So I think it is a very important improvement to the library, undoubtedly a great idea.

      Just one more little suggestion. I don't know if someone else has missed this, but I would like to be able to load an image in a user-provided buffer.

      I mean, I already have a buffer where I need the uncompressed image to use in OpenGL, and instead of loading it in the DevIL buffer with ilLoad*(), then ask for the pointer with ilGetData() and memcpy'ing it, ask DevIL to directly load it in the already existing buffer.

      Just an idea, it would be very useful to improve efficiency in some situations. If it fits well in the design of your new release, then excelent! :), if it does not, thanks anyway for the great work and for sharing it with the community.

      Cheers!
      Javier Taibo.

       
    • Jochen

      Jochen - 2009-09-10

      Hi!

      I'm also using DevIL for a private experimental 3D-engine project. For this I developed a flexible description of the image formats which could be interesting for DevIL to adopt.
      The format consists of 3 components: layout, type and swizzle.

      Tayout describes how up to four components (X,Y,Z,W = R,G,B,A) reside in memory. there are standard formats where each component uses one machine type and combined formats where all channels use some bits of one machine type. for example X8 is one byte, XYZW32 are four 32-bit integers (each integer in platform byte order) and X10Y10Z10W2 is one 32-bit integer where the first 10 bits are X and so on. The difference between XYZW8 and X8Y8Z8W8 is that XYZW8 uses four bytes and is therefore independent form the byte order while X8Y8Z8W8 uses one 32-bit integer.

      Type describes the type of the components, e.g. unsigned normalized, signed integer or float

      The swizzle describes the permutation e.g. to map the Z component to R. Whis this BGRA formats can be accomplished. It acts like the swizzle operator (e.g. value.zyxw) in shading languages.

      My format descriptor look like this:

      struct DataFormat
      {
          enum Layout
          {
              // standard formats
              X8,
              XY8,
              XYZ8,
              XYZW8,
              X16,
              XY16,
              XYZ16,
              XYZW16,
              X32,
              XY32,
              XYZ32,
              XYZW32,
         
              // combined formats
              X4Y4Z4W4,
              X5Y5Z5W1,
              X5Y6Z5,
              X9Y9Z9W5, // shared exponent
              X10Y10Z10W2,
              X11Y11Z10,
             
              // compressed formats
              DXT1,
              DXT3,
              DXT5,
              ATI1, // DXT5 alpha block
              ATI2, // two DXT5 alpha blocks
          };
         
          enum Type
          {
              UNORM,
              SNORM,
              UINT,
              SINT,
              FLOAT, // 16 bit: s1 e5 m10, 32 bit: s1 e8 m23
              SHARED_EXPONENT,
          };
         

          Layout layout;
          Type type;
          int swizzle;
      };

      Format conversion is done using LLVM. This is very simple and very fast ;-) (only DXT has to be handled separately) I could supply the code for this.

       
  • Denton Woods

    Denton Woods - 2009-09-29

    This looks like a good idea.  However, I'm really not sure when I'm going to have time again to work on DevIL, since I'm working on my PhD now.

     
  • Alistair Lowe

    Alistair Lowe - 2009-10-13

    Just a quickie on loading multiple files at a time. We could run into issues here when using third party libraries. We need to confirm that they're all thread-safe or ensure that whilst file load requests can be issued from multiple threads, they're ultimately only handled by one.

     
  • Denton Woods

    Denton Woods - 2009-11-06

    digisnap, how much of a speed hit do you get doing this?  I am not familiar with LLVM at all.

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks