[Phpwiki-talk] RFC: EmailNotification Coding Strategy

[Joe E. <jo...@or...>]

RFC: EmailNotification Coding Strategy

0   QUESTIONS & ISSUES

I have some questions about how to implement the functionality described in
the other RFC-- "EmailNotification Syntax and Features".  Notably:

(1) I would like to implement all of this using plugins which are only
loaded by those who need the functionality.  This will require an extension
of the plugin concept to allow for "event-based" in addition to
"content-based" plugins, and the placement of hooks inside WikiDB.php to
trigger events.

(2) I would also like to modify the existing code to keep track of the time
that links were created, and I have some questions about how best to do
that.

(3) I would like to ask the various experts here on the list how they think
my plugin can deal with the database in the most portable, modular, and
flexible way.

(4) I will explain a little how the plugins will work internally.


1.1  EVENT-BASED PLUGINS

The event-based plugin functionality is orthogonal to the existing content
plugin functionality, and it could be a completely separate thing.  It seems
like it makes sense to allow plugins to be both at once, though.   Imagine a
plugin for a database-backed tree-hierarchy organization of pages based on
multidimensional scaling or clustering on page text:  the event-side could
do the necessary reindexing every time a page is modified, and the
content-side would allow people to view the index using the "<?plugin?>"
syntax.

The principal thing that an event-based plugin needs to do is register one
or more functions or methods to be triggered at certain times by the
WikiEventManager using a line like:

    new WikiEventHandler ('onMajorRevision',
'function_that_checks_for_notifications');

It can do this registration when it is include()ed, I guess.  Since these
functions won't be loaded by the content-plugin engine unless they're used,
they'll need to be included manually from index.php if they're going to be
turned on.

    include('lib/plugin/NotifyAboutLinks.php');
    include('lib/plugin/NotifyAboutChanges.php');

Much functionality which transcends mere content could be refactored into
event plugins.  Logging, for instance, is event-driven, as is maintanence of
the "recent" table (which could sit together with the existing RecentChanges
plugin code).  All of the wacky indexing schemes that have been proposed
could be done with plugins possessing and event/content dual nature.


1.2  HOOKS INSIDE WikiDB::createRevision

WikiDB::createRevision seems to be the most sensible place to check for
needed notifications, because it stands above the database backends and is
not tied to a particular user action like savepage.php is.  I intend  to add
four lines to this function like:

    if (!$data['is_minor_edit']) {
        global $WikiEventManager;
        $WikiEventManager->triggerEvent('onMajorRevision', $this);
    }

and, depending on how I solve the link.ctime problem below, perhaps also two
lines like:

    foreach ($backend->get_new_links() as $new_link)
        $WikiEventManager->triggerEvent('onNewLink', array( $this,
$new_link ));

Unless handlers for these events have been installed, these calls will be
no-ops.  This will be made possible be two little classes called
WikiEventManager and WikiEventHandler which I'll place in
lib/WikiEventManager.php.


2. TRACKING THE LINK.CTIME

PhpWiki already keeps information about which pages link to which other
pages in the "link" table for use by BackLinks and certain other features.
Currently, when a page is revised, all the old information about what the
page linked to is deleted from the database, the new page data is scanned
for links, and all of these links are added to the database fresh, even
though most of them may have been in the previous revision.  For my
application, I need to know which links are new, and I need to store
information about when each link on a page was added in the database.  I
could do this with a separate table, but that would involve an inelegant &
abnormal replication of the link table and would obscure what should
probably be a central service.

I need to know which links are new so that I can fire off "onNewLink" events
from createRevision (described above).  I need to store the age of links in
the database so that, when notifications are being sent, the script can
notify about just those links that the users haven't been notified about
yet.

The simplest thing to do would be to hack the PearDB::set_links function so
that it registers a ctime for new links, doesn't clobber old links, and
stores a list of the new links in some hidden instance variable; then I
could write a getter function (backend::get_new_links()) for that instance
variable which would only work when called immediately after a set_links
call.  Then WikiDB::createRevision could call that getter and use it to
trigger "onNewLink" events.

The most sophisticated thing to do would be to create a WikiDB_Link object
similar to WikiDB_Page and WikiDB_PageRevision and have that object support
arbitrary data get() and set() like the others, and do the same kind of
translation between a $data[] array and the database fields.  This involves
serious changes to the API for both the various backends and WikiDB,
complicates the code, and is something to be very careful about because
certain functions would be changing their return values from arrays to
iterators.  If the consensus is that this is the only way to do it, however,
I will try and do it myself in Jeff's style.  With this approach it is still
unclear where the best place to trigger "onNewLink" events would be.

I should also note that, for efficiency, link.ctime needs to be indexible in
the database, so there would be a link.ctime added to the database table
(rather than just a serialized "linkdata") anyway.

There are various solutions in between the simple one and the sophisticated
one.  For instance, rather than hack set_links as per the simple approach, I
could expand the backend API to include "add_links" and "remove_links" and
raise the detection of new links up into WikiDB's sphere of influence.

Personally, I don't think it makes sense to do the sophisticated thing now.
If we find out that we need a layer of abstraction around links in the
future, we can always do it then.

But that's just me, what do you think I should do?


3. WHERE SHOULD I PUT MY DATA?

 Aside from link.ctime, which is covered above, the NotifyAboutLinks plugin
needs to track two additional peices of data.

(1)  For each page which has a NotifyAboutLinks (older than X) block on it,
we need to cache that page's "expiration date", which is the earliest date
on which the last notification for the most recent event could be sent out.
This will save us from parsing through the full text of every page in the
database every time we check for notifications.  Since most pages won't have
expiration dates (they will tend to be on Category pages), I think it makes
sense to use a separate table for this.

(2)  For the entire wiki, I would like to store the date on which the last
check for notifications was run, so that we can check for notifications that
have become necessary in between then and now.

Existing plugins use code in WikiDB and in the backends to talk with the
database.  This is very portable but not so modular-- even folks who never
use the plugin have to wait while PHP parses the DB code for it on page
loads.  It seems to make sense to leave WikiDB as is and put mysql-only code
directly into the plugin file.  Is that right?

Also, is there any preference as to how I store wiki-wide dynamic data?
Should I make a "wiki" table with one row and (for now) one column?  Is this
something specific to my application or something that people would a
general (WikiDB) interface to?


4  ABOUT THE PLUGIN

4.1  SQL STATEMENTS

The most important SQL statement, run on every invocation of
admin/notify.php, finds the pages that need to be examined for outgoing
notifications, and looks like this:

    select page_id from page_notify
    where expire_date > [DATE OF LAST NOTIFICATION]

The "expire_date" is a calculated cached (ab-normalized) value that takes
the last time anything was linked to the page ("max(ctime) from link where
to=page_id") and adds the maximum time interval mentioned on the page in an
"(older than X UNITS)" clause.  If the last time we notified people all the
links on the page were too old to get notifications, and there have been no
new links, we don't need to check that page.

The other important SQL statement, run per-page on every page returned by
the first query, identifies, for each "(older than X UNITS)" block on each
page, which new links notifications should be sent out to all the emails
listed in that block.  Any link that has become relevant in between now and
the last notify is included.  It looks like this:

    select "from" from link
    where to=[PAGE_ID]
    and ctime > [DATE OF LAST NOTIFICATION] - ["OLDER THAN" DELAY]
    and ctime < now() - ["OLDER THAN" DELAY"]


4.2  DATA MAINTANENCE

Whenever there is a new link to a page with a NotifyAboutLinks keyword, or
whenever a NotifyAboutLinks time duration is added, removed, or changed, we
need to update that page's expiration_date.


4.3  NEW FILES

admin/notify.php
// functions to do actual notification

lib/WikiEventManager.php
// A simple class for abstracting the hooking mechanism used below

lib/plugin/WikiEmails.php
// Classes for the new syntax & for sending emails

lib/plugin/NotifyAboutLinks.php
// handlers for the feature

lib/plugin/NotifyAboutChanges.php
// handlers for the feature


4.4   CLASSES

A CLASS FOR PARSING THE SYNTAX

class WikiEmailList
Constructor:  new WikiEmailList( $wikiPage, $keywordString );
Methods:
    $w->findOnPage();  // returns 1 if another keyword-headed block is found
    $w->emails();  // returns a list of email address strings from the
current block
    $w->args();  // returns "older than" thingie if it's there in the
current block

class NotifyAboutChangesList extends WikiEmailList
Constructor:  new NotifyAboutChangesList( $wikiPage );

class NotifyAboutLinksList extends WikiEmailList
Constructor:  new NotifyAboutLinksList( $wikiPage );
Methods:
    $n->delay();  // returns the "older than" delay for this block in
seconds


A CLASS FOR THE OUTGOING EMAILS

class WikiEmail
Constructor:  new WikiEmail( $msg );
Methods:
    $e->sendToBlindly( $addresses[] );

class NotifyAboutChangesEmail extends WikiEmail
Constructor:  new NotifyAboutChangesEmail( $wikiPage );

class NotifyAboutLinksEmail extends WikiEmail
Constructor:  new NotifyAboutLinksEmail( $fromPages[] );


A CLASS FOR DISPATCHING ON EVENTS

class WikiEventHandler
Constructor:  new WikiEventHandler( $eventName, $handlingFunction )
Methods:
    $eh->destroy();  // unregisters the event handler
    $eh->getEventName();
    $eh->trigger( $data[] );

class WikiEventManager
Constructor:  **singleton**
Methods:
    $em->register( $handler );
    $em->unregister( $handler );
    $em->trigger( $eventName );