DjVuLibre

Hello.

I am happy to inform you that Gnome DE under Linux now can read djvu files as pdf or ps with Evince document reader. Thanks a lot for your work.

Our website is http://gnome.org/projects/evince

And a few questions:

1. We already have request from users to add "text find" features and so on. Is there are plans to extend public ddjvu API with such functions?

2. We are trying to use continuous view for djvu documents. To do that, we should firstly calculate
sizes of all pages on application start. Currently we use the code like this:


d_page = ddjvu_page_create_by_pageno (djvu_document->d_document, page);

while (!ddjvu_page_decoding_done (d_page))
ddjvu_message_wait (djvu_document->d_context);

if (width)
*width = ddjvu_page_get_width (d_page) * SCALE_FACTOR;
if (height)
*height = ddjvu_page_get_height (d_page) * SCALE_FACTOR;


But this works incredibly slow. It takes about minute or so to open document with 300 pages. Do you have any suggestion on how we can optimize this?

> Our website is http://gnome.org/projects/evince
I have seen your work indeed.
Pretty nifty document reader...

> We already have request from users to add
> "text find" features and so on. Is there are plans
> to extend public ddjvu API with such functions?
Absolutely. Lack of time has been the main problem.
But now I have a good motivation to do it.
I am planning to provide the text information
as s-expressions similar to the ones generated by djvused.


> But this works incredibly slow. It takes about minute or so to open document with 300 pages. Do you have any suggestion on how we can optimize this?

Basically it amounts to decoding all pages in order
to get their sizes. Does evince work on files only, or
do you expect it to work on a network connection.
In that case we want to be able to display the
first pages before receiving all the files. The only
way to do this is to assume that pages have the same sizes, and correct the layout when new information arrives.

I can add a function to pick the page sizes without decoding everything. But this will only work efficiently
on disk files, not network connections....

- L.

Thanks for answer leonb

>Absolutely. Lack of time has been the main problem.
>But now I have a good motivation to do it.
>I am planning to provide the text information
>as s-expressions similar to the ones generated by djvused.

That should be amazing. Probably if you can give breif description of what should be done we can try to implement such thing. The main part of djvulibre is very good, the problem is only to write high-quality public interface. Something should move after 9 month CVS delay :)

Here is the example (just to look at) of our find interface

struct _EvDocumentFindIface
{
GTypeInterface base_iface;

/* Methods */

void (* begin) (EvDocumentFind *document_find,
int page,
const char *search_string,
gboolean case_sensitive);
void (* cancel) (EvDocumentFind *document_find);
int (* page_has_results) (EvDocumentFind *document_find,
int page);
int (* get_n_results) (EvDocumentFind *document_find,
int page);
gboolean (* get_result) (EvDocumentFind *document_find,
int page,
int n_result,
EvRectangle *rectangle);
double (* get_progress) (EvDocumentFind *document_find);

/* Signals */

void (* find_changed) (EvDocumentFind *document_find,
int page);
};

>I can add a function to pick the page sizes without decoding >everything. But this will only work efficiently
>on disk files, not network connections....

I really want to see this feature since it's very user-visible bug. Is it very hard to impelement?

We don't use connection, we start to work with document as a whole. As a long-standing optimization it's possible to use single page view and switch to continuous when document will be transferred. But it's long-standing goal.

> > I can add a function to pick the page sizes
> > without decoding everything. But this will
> > only work efficiently on disk files,
> > not network connections....
> I really want to see this feature since it's very
> user-visible bug. Is it very hard to impelement?

Done (in CVS)
You might have to wait a few hours for
the anonymous cvs mirror.

The function is called
ddjvu_document_get_pageinfo(...)
in the "workaround" section.

I looked a little bit at your evince code.
It seems that you do not call ddjvu_message_pop
(at least I did not find it). This means that all
the messages remain allocated on the queue.
This could make a lot of them.

I think you should write a single function
to handle pending messages (see the "handle"
function in tools/ddjvu.cpp for instance).

Then you have nice constructs like:
....
while (! ddjvu_document_decoding_done(doc))
handle(TRUE);
...
while (! ddjvu_page_decoding_done(page))
handle(TRUE);
...
that make your life easier if you
want to decode djvu documents
synchronously.

Later you could give a try at
decoding pages asynchronously
(do not wait for the page decoding.
call render immediately and you might get
some pixels already. Watch for <m_redisplay>
messages indicating that fresher pixels
are available...)

Also when displaying a page, you can
call ddjvu_page_create() on the next page.
This starts a decoding thread for the next
page while the user is reading the current one.

- L.

Thanks, I've updated message handling code, now it pops messages from a queue. I'll try to use new part of API, really we should add dependancy on anoncvs of djvulibre :)

The problem is that our application already use async threads, it's useful since all backends share the same code, different threads can render multiple pages in one time. From this point of view, we need just sync interface to any backend. In djvu plugin we are starting thread for every page and this thread djvu-owned thread to render page, this looks a bit strange.

Async API is definitely very useful for applications, but syncronous is also improtant. For example, most libraries provide two kind of API. Like it's possible to read files with syncronously and with POSIX async API.

Really, the C++ djvulibre code should be mostly privite, it was very nice decision to create public API to library, but what do you think about extending it with sync functions?

About text extraction and metadata. Do I understand correctly that there is tools like djvused, and you just want to create API like available with such tools and then port them to this new functions?

> new part of API, really we should add
> dependancy on anoncvs of djvulibre :)

You can test it with DDJVUAPI_VERSION>14

> The problem is that our application already
> use async threads, it's useful since all
> backends share the same code,
> different threads can render multiple pages
> in one time.

So your thread mostly waits for my thread.
This is no worse than the shell...

DjVu is a lot more asynchronous than what is
currently in evince, because we assume that
the data is usually streamed from a network.
From experience people have trouble mixing
interthread communication (IPC) and GUI event models.
This is why ddjvuapi converts the IPC into events.

There is one feature you could take advantage of.
Currently you decode full pages into a big pixmap. If you get a 1200dpi djvu file at full zoom, this will either fail or be very slow. This is why we can render partial segments of the page from partially decoded data.
This fast enough to decode uncovered areas while scrolling the page. This is why djview feels fast.

> but what do you think about extending
> it with sync functions?

From experience, people then mix both
sync and async functions. Results are very bad.

Using the ddjvuapi functions in synchronous mode
is not very difficult. I added documentation about
the event pump plus some examples. Explaining is
bettter than adding more functions.

> About text extraction and metadata. [...]
> you just want to create API like available
> with such tools and then port them to
> this new functions?

Correct. This summer.

Thanks again.

- L.