Re: [Libwpg-devel] Proposal: input stream simplification

[Ariya H. <ari...@gm...>]

Hi Fridrich,

> First of all, thanks a lot for the nice work on perfectspot. It was  a
> great experience to see commits raining into the CVS.

It's long due. Beside, I would soon need proper complex gradient
support in Qt, which is available only in Qt 4.3 (already in Beta
now), to reproduce many WPG graphics faithfully in Perfectspot.

> Now, concerning the stream implementations, I really hate to look likeld n
> an enemy of progress, but I have to expose my take on it:

On the contrary, your input is very appreciated. I doubt that someone
else has better knowledge and insights to our WPX code :-)

> 1. The reason why I made libwpg to use the WPXInputStream interface was
> that sometimes in a next feature, I would like to branch libwpd and add
> some possibility of parsing embedded pictures. This would just allow me
> to extract the subdocument as a WPXMemoryInput stream and pass the
> pointer to libwpg and grab whatever it outputs and process internally.

> 2. The other reason is that three libraries actually use something that
> could be really really the same thing: libwps uses WPSInputStream that
> is derived from WPXInputStream. I made the libwpg use the WPXInputStream
> also for maintenance reasons. In the future, I would like to have only
> one WPXInputStream implementation library for free use, so that I do not
> have to commit the same change into 2 different cvs repositories and one
> svn too :-). To have the stream the same in all three made it very very
> simply for me to integrate the wpg importer into ooo-build some two
> weeks ago and allowed me to reuse in a considerable way the
> writerperfect infrastructure.

The difference in our opinion perhaps stems from the fact that we do
not agree whether libwpg needs an input stream at all. IMHO libwpg
does _not_ necessarily need any input stream. Arguably, clip arts in
WPG format are typically quite small in size [1] and hence it is quite
sensible to load the content directly in memory and work from there.
This is different than libwpd/libwps, because a text document can be
very large and it is quite not efficient to first load everything to
memory.

As another example, in case of libwpd with embedded picture support
[2], the relevant code needs to read the block corresponds to WPG data
into memory and pass it to libwpg. IMHO it's simpler than constructing
yet another memory stream.

But even if we want to be flexible and allow input stream for libwpg,
the stream implementation itself does not need to exist in libwpg.
This is similar e.g. to zlib. It does offer API for
compression/decompression in memory and through in-out stream. But the
actual stream code is left to the application that wants to use it.

> 3. To keep the stream abstract showed itself a very good solution with
> libwpd and allowed us to have a stable libwpd API and ABI for more then
> 2 years already. I really would like to have the library itself free of
> the stream implementation if it was possible.

I guess the best choice is then to create a separate stream library
for that [3], something called FGIOSL (Fridrich's Generalized Input
Ouput Stream Library), which is quite a cool name :-)

That way, libwpg/libwpd/libwps share most of the stream code and it is
easier to handle. And while we're at that, we might want to put our
output stream related-code there as well, e.g. something for
outputting the OpenDocument container, or even just to stdout and
plain file [4].

> 4. The dependency of libwpd is true, but it is actually a dependency on
> one only header file <libwpd/WPXStream.h>. I have to fiddle more with
> the build system, because libwpd should not be necessary for the binary
> libwpd and libwpg-stream, neither should it be necessary for linking the
> libwpg and libwpg-stream. For applications that use libwpg and do not
> use libwpd (very few of them actually), putting the WPXStream.h header
> somewhere should be enough.

That is a bit hackish, though. Since libwpd (at the moment) is needed
anyway when building libwpg, application which wants to use libwpg
(e.g. Perfectspot) is likely going to just use libwpd's header.

At least, from a packager's point-of-view, it will be:
  Perfectspot depends on libwpg + libwpd
  libwpg depends on libwpd

> 5. The convenience functions can be achieved this way:
>
> parseFile(filename, &exporter) -> parse(&WPGFileStream(filename),
> &exporter);
>
> parseBuffer(buffer, length, &exporter) -> parse(&WPGMemoryStream(buffer,
> length), &exporter);
>
> Since the temporary as a parameter of the function should survive during
> the life of the function, right?

True, but that's just semantics. See my take on (1) and (2) above.

> This is my take, and as I already said, I hate to be an enemy of
> progress :-( Nevertheless, the stream internalization does not look like
> a good idea for me at this stage.

I agree with you. In case I didn't express it clearly, my proposal was
more to entirely decouple the stream implementation from libwpg, which
is along the line with what you want to achieve if we have the
specialized separate stream library.

>From here, I see the following path:

(A) Make libwpg mainly works with memory buffer. Thus, the main API will be:
        libwpg::WPGraphics::parseBuffer(const char* buffer, int
length, WPGListener*)


(B) Make it possible to use input stream, i.e. libwpg depends on
FGIOSL and offer the following.
      libwpg::WPGraphics::parseInput(const WPXInputStream* input, WPGListener*)

Point (A) covers 90% of the use of libwpg (I hope). Point (B) is for
those who want the full flexibility.

Since you might want to start branching libwpd (for libwpd 0.9?),
libwpg has no release yet, and so does libwps, wouldn't it be a good
time now to start implementing FGIOSL?


Regards,


Ariya


[1] Unless you place 5 megapixels vacation pictures inside the file,
but that'll probably only 0.01% of the case.

[2] Feel free to start the discussion in libwpd-devel about this. I'll
be happy to help. Also, perhaps we can start wpd2odt subproject as
well there?

[3] Suppose we start sharing more code between KWord/Abiword/OO, say I
split the KWord's Ami Pro filter code into libami, or we collaborate
on Ultimate RTF library, then such stream abstraction library will be
also very useful.

[4] Somehow I feel we duplicate iostream and/or libgsf here, although
it's true that both do not really fulfill such stream's library
requirements (no hard STL dependency, easy to build on Win32, ...)