Re: [jabber.el] [Patch] Support for server-side message archive management (XEP-0313)

[Thibault M. <thi...@gm...>]

Hi, thanks for trying this out.

I only tested with my prosody server, so I expect more work will be
needed to get this to work reliably.  Thanks for offering to help debug
this.

First a quick word on the module.  It spawns an XMPP request when (1)
opening a new chat buffer (in order to get the recently archived
messages) and when (2) calling `jabber-chat-display-more-backlog'.  The
XMPP request is asynchronous and the main function `jabber-mam-query'
spins until the incoming message handler releases the lock.  Hanging at
(while (not jabber-mam-lock) (sit-for 1)) means that the lock does not
get released by the message handler function.

The first thing to do to debug this is to capture the XMPP request and
the corresponding response message.  The attached patch should print
both in the *Messages* buffer.  If you could apply this patch, and try
again, we should be able to see the request and response.  That would be
the first step.

The *Messages* buffer should contain something like this:

* The prepared MAM request

,----
| MAM request: [<query xmlns='urn:xmpp:mam:0'><x xmlns='jabber:x:data' type='submit'><field var='FORM_TYPE' type='hidden'><value>urn:xmpp:mam:0</value></field><field var='id'><value>thi...@ex...</value></field><field var='with'><value>us...@ex...</value></field><field var='start'><value>2017-02-08T20:44:53Z</value></field></x><set xmlns='http://jabber.org/protocol/rsm'><max>10</max><before/></set></query>]
`----

I often try to check the query by using the XMPP console in pidgin to see the
server response.  To do so, I wrap the content of the query within <iq> tags:

<iq type='set' id='test'>
<query xmlns= .... REMAINDER OF THE REQUEST TEXT ...</query>
</iq>

* The initial response to the request

,----
| MAM request succeeded
`----

This simply means that the request succeeded, but this instantaneous server
response does not return any message.  If you get a failure message here, the
request was not accepted (e.g. because the server does not support MAM or the
namespace is wrong).  But if you had a failure, then the lock should be
immediately released, so it shouldn't hang.

* Incoming messages

Then the `jabber-handle-incoming-mam-message' function should process (and print
for debugging) the incoming messages.  Here is an example of message:

,----
| MAM incoming: (message ((to . thi...@ex.../20e82cb5-2d01-4a92-90ef-ffed886aee36)) (result ((id . 8b18e151-3199-4327-8a75-96e3bfd22c72) (xmlns . urn:xmpp:mam:0)) (forwarded ((xmlns . urn:xmpp:forward:0)) (delay ((stamp . 2017-01-27T05:04:06Z) (xmlns . urn:xmpp:delay))) (message ((type . chat) (to . thi...@ex...) (from . us...@ex.../015a473e-ed3a-4063-9d94-d42238631761) (id . purplea9f9333e) (xmlns . jabber:client)) (active ((xmlns . http://jabber.org/protocol/chatstates))) (body nil Message content open))))) |
`----

* A final message

The server should use result set management
(https://xmpp.org/extensions/xep-0313.html#query-paging) to page through
results.  After N messages, it should return a message containing the last
returned index so the caller can send a subsequent request and get the remaining
messages.  I suspect the issue is around here.

The final message I get may look like this.

,----
| MAM incoming: (message ((to . thi...@ex.../20e82cb5-2d01-4a92-90ef-ffed886aee36)) (fin ((complete . true) (xmlns . urn:xmpp:mam:0)) (set ((xmlns . http://jabber.org/protocol/rsm)) (first nil 64e7f38c-5c47-484b-8cae-897f0691009e) (count nil 75) (last nil 03d1feb1-bfc1-4e00-bc24-f88aef05d1e1))))
`----


My code tries to read this final message and determine whether to send a
following request or to stop and release the lock.  Among other things, it uses
the (complete . true) part of the response to make that decision.  I suspect the
code makes wrong assumptions about the structure of this message.

If you could report with the *Messages* output, that would greatly help.  I will
also try to setup an ejabberd server, to see if I can reproduce the
issues on my end.

Thanks for your help,

thibault