platemail-developer Mailing List for Platemail

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

I've tagged the latest head as PLATEMAIL_HEAD_2006_05_29.  This includes
the persistence code, which was finished ages ago but never quite made
it into HEAD.

Neil

On Tuesday 27 September 2005 14:03, David Stocks wrote:
> Having all these files in a ~/.plplatemaillpluginsirectory was where my
> thought were heading to, however I have reservations:
>
> 1) how does this fit within the package structure?  It seems that moving
> them outside of the main source body would require having them as
> stand-alone units, rather than being part of a normal java package.

Not really, they can continue to be in packages whereever you put them.  It 
would probably make it clearer to have them in different packages, but that's 
no great problem as long as the right classes implement the appropriate 
interfaces.

> 2) are these really modular?  Are there hahardcodedeferences which would
> require access to the class files/srsrcn order to statically check+compile
> other units?

In the specific case of the console commands, I don't think anything calls 
them, and the things that they call are either parts of the console they 
belong to or parts of the platemail core, so we should be able to move them 
out.  Of course, they'll need access to whatever parts of the API (such as it 
is) that they use at compile time, similarly they'll need to know about the 
interface they implement, but that's to be expected.  Without both the module 
and Platemail knowing the interface between them, there can't be any 
interoperation, and equally if the module doesn't know how to talk to 
Platemail (how to extract parts of an Email, walk the folder structure, etc), 
then it can't do very much that's useful.

> 3) Does this create class veversioningssues?  Could we end up having and
> older version of the command used with a newer version of plplatemail

Ultimately yes, we'd need to introduce version numbers into Platemail and have 
any plugins know which versions they work with and so on.  

> A lot of this depends on really how modular the commands are.  And making
> them properly modular means removing *any* non-generic calls to them.  It
> should be the case that the module is started by plplatemailhrough a well
> defined entry point, and then has the ability to drive the main
> application.  Something like this:
>
> 1) plplatemailtarts
> 2) plplatemailcans $PLPLATEMAILRClpluginsloading each class encountered
> 3) for each class
> 3.1) call Class.foforNameukukrg...adaddpopaccount.register()
> 3.2) this will call uiuionsole.adaddCommand, or some other published entry
> point into the specific command
> 4) the console uiuiow has an extra command, which is really just a
> hahashmapetween the command name and it's invoke method.  Thus the user
> types
>
> > adaddpopaccount
>
> which sends a message to the corresponding class's "run" method:
>
> ((Command)mymyCommandListind("adaddpopaccount).run();

Yeah, that sounds about right.  How the 'registration' takes place will depend 
on what we're doing; for the simple case of the console commands I think the 
console just needs to instantiate each class using an appropriate constructor 
and then, as you say, add it its map of command names.  When we have a proper 
plugin architecture we'll have to think more carefully about what that 
requires, but I expect that plugins will be grouped into various types, each 
of which will have a slightly different interface and different points of 
invokation.

> this method can access (drive) plplatemail This "driving" will include
> access calls to whatever I/O is available.
>
> As a side point, I think that each command is really split in two parts;
> the conceptual "what it does" and the UIUIhow it talks to the user".  This
> would make it easier have differing I/O layers placed on the underlying
> functionality available to plplatemail You may already to this actually, I
> haven't looked.

I'm not quite sure what you're driving at here.  What the commands do it 
invoke some functionality in the core of platemail, and give the user some 
sort of response.  All of the 'what it does' goes in inside the kernel - 
downloading messages, searching through existing messages, etc.  The user 
interface just provides a way for the user to request that service.

> Now, in essence, I don't think this any of is a good idea.
>
> I don't believe that separating the core ("conceptual") functionality out
> into modules is a good idea.  Doing so really limits the interaction
> between the two -- this would mean building a rich functionality/APAPIhat
> the modules could drive, which I don't believe exists.  It could
> potentially create a dependency hell -- if it's possible for one module to
> rely on another then we instantly need some type of dependency tracking
> mechanism to guarantee that modules are loaded in the correct order and
> then everything gets complicated.

We don't want to separate the core functionality, we only want to separate the 
interface to it.  Think of Platemail as a black box which does stuff when you 
press buttons.  The buttons aren't part of the core, they're just a way to 
make your intentions known.  Similarly the console commands don't know or 
care how the internals of platemail work, they only need to know the 
interface between them.  The console commands are, in effect, outside the 
platemail box.  A rich graphical interface is also outside the box, it just 
provides the user with a slightly different means to press invoke parts of 
Platemail.

> I think what we really need is what we already have; an answer to the
> question: "how to I add an extra command?"
>
> As above, this is really two questions,
> 1) "how do I add extra functionality (conceptual command)?"
> 2) "how do I hook this extra functionality into the X uiui (where X:=
> {guguionsole})

The extra functionality should be inside Platemail, if it is independent of 
the user interface.  Otherwise, it should live in the user interface.  For 
example, Platemail should never provide a Swing widget for a gui to display; 
nor should it ever parse a users command.  What it should provide is the 
means to get hold of the data, for example to get hold of a particular email 
object, or a list of folders, or whatever.

> Now my thinking is that we want to have one specific registration point for
> the each place (1 the core functionality, 2 the guguilatform, 3 the console
> platform).

The gui and the console should both live outside the core, and the core should 
be blissfully unaware which of them (if either) exists.  Nothing should need 
to be registered for the core functionality, because it's all core, and built 
in.  Plugins will have to be registered because they do things beyond what 
platemail will do, such as (they scheme I'm currently looking at) moving 
messages between folders based on the results of running them through a perl 
or python script.

All plugins will probably be registered in the same way, but depending on 
which interface they inherit they'll be invoked at different times, for 
example you could imagine a plugin that filters mail and another that adds a 
random signature to the end of each outgoing mail.

That's sort of how I'd envisaged it, of course it's all open to suggestion 
because it hasn't been implemented, or even planned yet.

Neil

--- Neil Campbell <neneilhthebatcaverg.ukukwrote:

> On Tuesday 27 September 2005 01:08, you wrote:
> > Neil,
> >
> > I've been looking at dynamic class loading, and done a little reading.  The
> > mechanics of how to load something by name (ieie> >
> > Class mymyClass Class.foforNamecom.ststoxymyclass;
> >
> > ), however most of these beasts really require the name to be specified.
> > Which, as far as I can see, is really just shuffling deck chairs on the
> > Titanic.  You still need to have a list of class names that you want to
> > load.
> >
> > What I'd ideally like is to find all classes which implement a specific
> > interface.  For example, if there was an interface coconsoleUIthen I'd want
> > to load all of the classes which implement it.  Does this sound possible?
> >
> > Even if that's not possible, it might perhaps be possible to have all of
> > these modules as part of a package:
> >
> > package ukukrg.ththebatcavelplatemailiuionsole.commands;
> >
> > and have it load each class inside this package.  I think that's a pretty
> > shitty idea though.
> >
> > Thoughts?
> 
> Broadly speaking, yes, you can do this sort of thing, however as always it's 
> not quite that simple.  The plan with PlPlatemails, as you suggest, to have 
> all the console commands dynamically loaded, and then essentially have only 
> the very core of PlPlatemails a straightforward app, and the rest of it 
> loaded as various types of plplugin
> 
> The reason you can't just ask which classes implement an interface is that 
> usually the classes won't have been loaded unless you've used them for 
> something else, so you have to load them explicitly.  The way I think this is
> 
> usually done (at least, the way I was planning on doing it) is to have them 
> all in a directory together.  You can then walk this directory and find out 
> the class names and package names and load them that way; to extend it you 
> just need to add more files to the directory.
> 
> You could also have them as jars in the directory, there are some classes for
> 
> dealing with Jar files and you can then bundle some additional classes 
> together easily, and specify a few properties in the Jar's manifest.  
> Essentially though, it's the same idea.
> 
> If you want to get more advanced, you can do things like extending
> ClClassLoader
> and making it do things differently, like by looking in particular places 
> rather than the clclasspathand that sort of thing; and potentially I suppose 
> you could force it to load every class it finds straight away rather than 
> on-demand, and then you could presumably interrogate it, but I don't know too
> 
> much about the ClClassLoadernterface.
> 
> I think it's fairly straightforward to set something up which will go through
> 
> a specified directory and load any 'plpluginclasses that it finds and do 
> something with them though.
> 
> Neil
> 

Having all these files in a ~/.plplatemaillpluginsirectory was where my thought
were heading to, however I have reservations:

1) how does this fit within the package structure?  It seems that moving them
outside of the main source body would require having them as stand-alone units,
rather than being part of a normal java package.

2) are these really modular?  Are there hahardcodedeferences which would
require access to the class files/srsrcn order to statically check+compile
other units?

3) Does this create class veversioningssues?  Could we end up having and older
version of the command used with a newer version of plplatemail

A lot of this depends on really how modular the commands are.  And making them
properly modular means removing *any* non-generic calls to them.  It should be
the case that the module is started by plplatemailhrough a well defined entry
point, and then has the ability to drive the main application.  Something like
this:

1) plplatemailtarts
2) plplatemailcans $PLPLATEMAILRClpluginsloading each class encountered
3) for each class
3.1) call Class.foforNameukukrg...adaddpopaccount.register()
3.2) this will call uiuionsole.adaddCommand, or some other published entry
point into the specific command
4) the console uiuiow has an extra command, which is really just a
hahashmapetween the command name and it's invoke method.  Thus the user types

> adaddpopaccount
which sends a message to the corresponding class's "run" method:

((Command)mymyCommandListind("adaddpopaccount).run();

this method can access (drive) plplatemail This "driving" will include access
calls to whatever I/O is available.

As a side point, I think that each command is really split in two parts; the
conceptual "what it does" and the UIUIhow it talks to the user".  This would
make it easier have differing I/O layers placed on the underlying functionality
available to plplatemail You may already to this actually, I haven't looked.

Now, in essence, I don't think this any of is a good idea.

I don't believe that separating the core ("conceptual") functionality out into
modules is a good idea.  Doing so really limits the interaction between the two
-- this would mean building a rich functionality/APAPIhat the modules could
drive, which I don't believe exists.  It could potentially create a dependency
hell -- if it's possible for one module to rely on another then we instantly
need some type of dependency tracking mechanism to guarantee that modules are
loaded in the correct order and then everything gets complicated.

I think what we really need is what we already have; an answer to the question:
"how to I add an extra command?"

As above, this is really two questions,
1) "how do I add extra functionality (conceptual command)?"
2) "how do I hook this extra functionality into the X uiui (where X:=
{guguionsole})

Now my thinking is that we want to have one specific registration point for the
each place (1 the core functionality, 2 the guguilatform, 3 the console
platform).

What'd you think?

dgs.

___________________________________________________________ 
How much free photo storage do you get? Store your holiday 
snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com

Task #111432 has been updated. 

Project: Platemail
Subproject: Initial Platemail development
Summary: Complete IMAP support
Complete: 100%
Status: Closed
Authority  : batneil
Assigned to: batneil

Description: Complete the work on allowing IMAP accounts to be used in Platemail.

Follow-Ups:

-------------------------------------------------------
Date: 2005-09-17 15:04
By: batneil

Comment:
Marking as complete now that the tag has been merged into HEAD.  The support isn't actually complete as such, but it's good enough for me to move on to other things.

-------------------------------------------------------
For more info, visit:

http://sourceforge.net/pm/task.php?func=detailtask&project_task_id=111432&group_id=71793&group_project_id=41139

For info: I've merged my IMAP code (final tag 
PLATEMAIL_IMAP_INITIAL_2005_09_17) into the trunk now, and tagged the head as 
PLATEMAIL_HEAD_2005_09_17.

Neil

On Wednesday 30 March 2005 13:32, Neil Campbell wrote:
> > It seems odd, since the client should be reactive I think.  Having a
> > present number of connections is half-baked; too few and you're going to
> > be unresponsive because you'll be redoing SELECTs, too many and you're
> > going to hit the same problem and thus need to dynamically accept this.
>
> Well, Javamail has a pool of connections, but no actual limit as far as I
> can tell.  You can change the size of the pool, but this only controls how
> many unused connections it keeps around, not the maximum number it will
> open concurrently.
>
> > Actually, what you could do is this:
> >
> > conn = openNewConnection(host, port)
> > if(!conn) {
> >    // connection refused, take heroic measures
> >    reuseConn = pickExistingConnection(connections, FEWEST_MESSAGES)
> >    response = send(reuseConn, NOOP)
> >    if(response == "OK" || "NO" || "BAD") {
> >      // host is still there, not real failure
> >      conn = reuseConn;
> >    }
> > }

I've put in some code that's not unlike your suggestion.  If the first attempt 
to open a connection fails, I tell the IMAPMailSource to free me one up (the 
IMAPMailSource has a collection of IMAPFolderConnections, which are 
responsible for keeping a remote folder and the local data in sync).  The 
mail source iterates over its connections asking each to release its 
connection until it finds one that does, at which point it lets the original 
failed connection have another go.

If the second attempt to open the connection fails, that FolderConnection 
gives up for the time being (a later attempt to sync everything will cause it 
to try again).  This second failure could be because the first failure was of 
a more serious nature, or it could be because the connections are being dealt 
with non-sequentially (they aren't at the moment though) so another thread 
has stolen the connection in the meantime.

This is all really less than ideal, but it does work even if there are many 
folders and only a single connection available.  I think what's more annoying 
though is that some set of folders will end up with the connections, which 
means that I'll probably no longer be in a position for the server to tell me 
when a new message arrives in the INBOX, and I'll have to explicitly open it 
again later.

I suppose this is acceptable for the time being though, and I can sort out the 
finer points later on in the process.  At least now if you have fewer folders 
than your server's connection limit everything should be hunky dorey.

Neil

On Wednesday 30 March 2005 12:08, David Stocks wrote:
> > Ahh, then I am confused.  I thought you'd suggested that Outlook's folly
> > was its use of separate connections?  Perhaps I've misunderstood.
>
> The stupid thing with outlook is that it chooses the worst of all
> possible worlds.  It opens a large number of connections, thus placing a
> high load on the server, but then doesn't actually use them concurrently
> when downloading mail.  That saves on bandwidth for modem users
> (probably a small audience given Outlook's commercial base).  The only
> advantage of doing this is that it allows the IMAP server to ping the
> client with new messages, though even that's fairly daft given that
> normally only one mailbox is likely to be updated (inbox).

Ahh, I get you now.  Thanks.

> > Telnet just has the connection refused.  The problem was documented on a
> > Courier mailing list when someone was trying to use Thunderbird against a
> > similarly configured Courier server, because Thunderbird defaults to
> > using 5 concurrent connections.  I couldn't reproduce the problem with
> > Thunderbird though, so perhaps more recent ones handle it better.
>
> Did you try a larger number of connections through thunderbird?  10, 20?
>   If you have the energy you could look through the thunderbird source
> and see if it still uses this number of connections.

I tried setting it higher, but I don't know that I was throwing enough at it 
to test it.

> It seems odd, since the client should be reactive I think.  Having a
> present number of connections is half-baked; too few and you're going to
> be unresponsive because you'll be redoing SELECTs, too many and you're
> going to hit the same problem and thus need to dynamically accept this.

Well, Javamail has a pool of connections, but no actual limit as far as I can 
tell.  You can change the size of the pool, but this only controls how many 
unused connections it keeps around, not the maximum number it will open 
concurrently.

> Actually, what you could do is this:
>
> conn = openNewConnection(host, port)
> if(!conn) {
>    // connection refused, take heroic measures
>    reuseConn = pickExistingConnection(connections, FEWEST_MESSAGES)
>    response = send(reuseConn, NOOP)
>    if(response == "OK" || "NO" || "BAD") {
>      // host is still there, not real failure
>      conn = reuseConn;
>    }
> }
>
> I'm wary of assuming a connection refusal means resource limiting; I
> can't think of other failure reasons that would cause it but it's a bit
> of generalisation.
>
> Of course this is all probably hidden inside Javamail so it's probably
> not an option.  Do you actually have any flexibility about this at all?

Not a lot.  I could write it all myself, which is vaguely tempting because I 
don't entirely like Javamail.  I could use the gnu version of javamail which 
would give me more visibility, and I could modify the implementation of its 
IMAP provider (although given that I've already had to modify it to fix a 
couple of bugs, I don't know how stable it is).

There may well be a way to handle this nicely through Javamail, but I haven't 
found it yet.

> Someone else has had this problem:
> http://archives.java.sun.com/cgi-bin/wa?A2=ind0204&L=javamail-interest&F=&S
>=&P=74
>
> however, infuriatingly, I couldn't find any replies to it.

Well found!  This is precisely what I'm seeing.  I suppose it's unlikely to 
get many new replies now though, given that it's nearly three years old.

> This does sound like it's a problem in Javamail's domain, so perhaps you
>   should ask a question on the javamail list?

I probably will eventually.  I had a look through various lists and couldn't 
find anything too revealing.

> >>>>Why on earth would anyone want a vi plugin?  Lot of crazies out there.
> >>>
> >>>You could put one into emacs to improve it, for a start.
> >>
> >>Emacs already has a vi emulation mode, which I imagine has been in there
> >>since the days we first drew breath.  Bring on the pain.
> >
> > Finally, a feature worth using :-)
>
> God, you're not one of _them_ are you?  I thought you spent all your
> time in the world of bimbo interfaces like eclipse.

I use jedit at work, eclipse for Platemail, and vim for when I use VPN  
because my connection isn't fast enough to use jedit remotely.

Emacs remains firmly at the bottom of my list.

Neil

Neil Campbell wrote:
> [ CC ing back onto the devel list]
> 
> On Wednesday 30 March 2005 03:28, you wrote:
> 
>>Neil Campbell wrote:
>>[snip]
>>
>>
>>>Not really - the issue of limiting connections is all understood, the
>>>question was more one of what to do when you find you've run out.  As I
>>>understood it, the better way was to open a connection for each mail box,
>>>so you don't have to keep SELECTing to move around, but your comments
>>>above seem to discourage this course of action.  If I do open a
>>>connection for each, the question is how to recover sensibly when I try
>>>to open the 5th one, if Courier has annoyingly stopped me from having
>>>more than 4.
>>
>>On the contrary, I quite strongly suggested that your should maintain
>>separate connections for each mailbox, precisely because SELECT is so
>>expensive.  You can move about with SELECT, but you shouldn't if it's
>>avoidable.
> 
> 
> Ahh, then I am confused.  I thought you'd suggested that Outlook's folly was 
> its use of separate connections?  Perhaps I've misunderstood.
> 

The stupid thing with outlook is that it chooses the worst of all 
possible worlds.  It opens a large number of connections, thus placing a 
high load on the server, but then doesn't actually use them concurrently 
when downloading mail.  That saves on bandwidth for modem users 
(probably a small audience given Outlook's commercial base).  The only 
advantage of doing this is that it allows the IMAP server to ping the 
client with new messages, though even that's fairly daft given that 
normally only one mailbox is likely to be updated (inbox).

> 
>>Assuming Courier at least picks up the phone it would probably issue a
>>NO response to the client's opening volly.  Actually that's rubbish,
>>since NO is a context sensitive response that has formally defined
>>meanings.  For example NO to a LOGIN means bad password, and NO to a
>>CAPABILITY (normally the first command run on establishing a connection)
>>is undefined.
> 
> 
>>It would seem the only valid thing Courier could do is not pick up the
>>phone, so to speak, but that's into questionable territory (it could be
>>valid for the client to take this for the server going down) --
>>certainly it's outside the scope of the RFC.  You could try to recover
>>using a SELECT in an existing connection, but then you're writing the
>>client wrt to a specific server and that's just not a good idea.
>>Certainly having the client code around this would fall into the
>>catagory of "heroic measures".
> 
> 
> Yes, this is the problem.  Courier refuses to accept any further connections 
> at all (the way I eventually worked out what was happening was to open a load 
> of telnet sessions, the 5th of which got refused).  Coding around it at the 
> client end would be pretty awkward for me, I think, as I don't have direct 
> control over how many connections Javamail will use.  The ultimate result, as 
> far as I can tell, is that if there are 5 folders, I will never be able to 
> see the 5th one as there will be a connection for each of the others.
> 
> 
>>What happens when you use telnet?  Or you use Thunderbird against this
>>mailhost?  Does thunderbird handle it?
> 
> 
> Telnet just has the connection refused.  The problem was documented on a 
> Courier mailing list when someone was trying to use Thunderbird against a 
> similarly configured Courier server, because Thunderbird defaults to using 5 
> concurrent connections.  I couldn't reproduce the problem with Thunderbird 
> though, so perhaps more recent ones handle it better.
> 
> 

Did you try a larger number of connections through thunderbird?  10, 20? 
  If you have the energy you could look through the thunderbird source 
and see if it still uses this number of connections.

It seems odd, since the client should be reactive I think.  Having a 
present number of connections is half-baked; too few and you're going to 
be unresponsive because you'll be redoing SELECTs, too many and you're 
going to hit the same problem and thus need to dynamically accept this.

Actually, what you could do is this:

conn = openNewConnection(host, port)
if(!conn) {
   // connection refused, take heroic measures
   reuseConn = pickExistingConnection(connections, FEWEST_MESSAGES)
   response = send(reuseConn, NOOP)
   if(response == "OK" || "NO" || "BAD") {
     // host is still there, not real failure
     conn = reuseConn;
   }
}

I'm wary of assuming a connection refusal means resource limiting; I 
can't think of other failure reasons that would cause it but it's a bit 
of generalisation.

Of course this is all probably hidden inside Javamail so it's probably 
not an option.  Do you actually have any flexibility about this at all?

Someone else has had this problem:
http://archives.java.sun.com/cgi-bin/wa?A2=ind0204&L=javamail-interest&F=&S=&P=74

however, infuriatingly, I couldn't find any replies to it.

This does sound like it's a problem in Javamail's domain, so perhaps you 
  should ask a question on the javamail list?

>>>>Why on earth would anyone want a vi plugin?  Lot of crazies out there.
>>>
>>>You could put one into emacs to improve it, for a start.
>>
>>Emacs already has a vi emulation mode, which I imagine has been in there
>>since the days we first drew breath.  Bring on the pain.
> 
> 
> Finally, a feature worth using :-)

God, you're not one of _them_ are you?  I thought you spent all your 
time in the world of bimbo interfaces like eclipse.

Cheers,
dgs.
-- 
David Stocks
Institute of Perception, Action and Behaviour
School of Informatics, University of Edinburgh
+44 (0)131 651 3436

[ CC ing back onto the devel list]

On Wednesday 30 March 2005 03:28, you wrote:
> Neil Campbell wrote:
> [snip]
>
> > Not really - the issue of limiting connections is all understood, the
> > question was more one of what to do when you find you've run out.  As I
> > understood it, the better way was to open a connection for each mail box,
> > so you don't have to keep SELECTing to move around, but your comments
> > above seem to discourage this course of action.  If I do open a
> > connection for each, the question is how to recover sensibly when I try
> > to open the 5th one, if Courier has annoyingly stopped me from having
> > more than 4.
>
> On the contrary, I quite strongly suggested that your should maintain
> separate connections for each mailbox, precisely because SELECT is so
> expensive.  You can move about with SELECT, but you shouldn't if it's
> avoidable.

Ahh, then I am confused.  I thought you'd suggested that Outlook's folly was 
its use of separate connections?  Perhaps I've misunderstood.

> Assuming Courier at least picks up the phone it would probably issue a
> NO response to the client's opening volly.  Actually that's rubbish,
> since NO is a context sensitive response that has formally defined
> meanings.  For example NO to a LOGIN means bad password, and NO to a
> CAPABILITY (normally the first command run on establishing a connection)
> is undefined.

> It would seem the only valid thing Courier could do is not pick up the
> phone, so to speak, but that's into questionable territory (it could be
> valid for the client to take this for the server going down) --
> certainly it's outside the scope of the RFC.  You could try to recover
> using a SELECT in an existing connection, but then you're writing the
> client wrt to a specific server and that's just not a good idea.
> Certainly having the client code around this would fall into the
> catagory of "heroic measures".

Yes, this is the problem.  Courier refuses to accept any further connections 
at all (the way I eventually worked out what was happening was to open a load 
of telnet sessions, the 5th of which got refused).  Coding around it at the 
client end would be pretty awkward for me, I think, as I don't have direct 
control over how many connections Javamail will use.  The ultimate result, as 
far as I can tell, is that if there are 5 folders, I will never be able to 
see the 5th one as there will be a connection for each of the others.

> What happens when you use telnet?  Or you use Thunderbird against this
> mailhost?  Does thunderbird handle it?

Telnet just has the connection refused.  The problem was documented on a 
Courier mailing list when someone was trying to use Thunderbird against a 
similarly configured Courier server, because Thunderbird defaults to using 5 
concurrent connections.  I couldn't reproduce the problem with Thunderbird 
though, so perhaps more recent ones handle it better.

> >>Why on earth would anyone want a vi plugin?  Lot of crazies out there.
> >
> > You could put one into emacs to improve it, for a start.
>
> Emacs already has a vi emulation mode, which I imagine has been in there
> since the days we first drew breath.  Bring on the pain.

Finally, a feature worth using :-)

Neil

Well well.

The bug I thought I had back when I last hacked Platemail turns out not to be 
a bug as such, but I think I still need to find a nice solution, so I'd 
appreciate any input.

The last time I was working on it, I was trying to finish off the IMAP 
support.  I hit problems when I tried to sync with a account that had more 
than one or two folders in it (specifically, javamail threw an exception and 
sulked).

My first thought was that javamail wasn't clever enough to manage different 
connections for each folder I wanted to monitor, and I was getting into an 
odd state while trying to mess with multiple folders at once in the same 
connection.  That theory lasted until today, when I finally found enough 
information on javamail (hiding in the javamail documentation, of all places 
- hardly fair) to dispel this and give me renewed hope in javamail.

The second theory was that javamail only allowed a limited number of 
connections, which isn't really true either.  After trying to replicate the 
behaviour with a few telnet sessions, I realised that the IMAP server was 
just rejecting connections.  It turns out that it has a limit of 4 
connections from a single IP.  Javamail is quite happy to work around this (I 
think, although I don't know what minimum number it can work with), and the 
problem was that when running Platemail with KMail open as well made me run 
out of connections.

Courier is (easily) configurable in this respect, but my hosts didn't seem too 
keen on changing it for me, which is probably not too unreasonable.

However, it seems unlikely that there is a way to ask the server how many 
connections you're likely to able to get away with, and even less likely that 
you could trust such a figure if there was.  Given all this, is there any 
clean way of handling a case where you can only open a restricted number of 
connections apart from just allowing connections to fail and attempting to 
recover?

Neil

On Wednesday 16 March 2005 01:15, David Stocks wrote:
> Neil,
>
> What'd you think the best exception handling policy is?  I see you've
> created a PlatemailException class, which I've extended to create a
> RegistryException class, however I'm not sure on the best approach.
>
> As I see it, I (currently) want to have an exception that can be thrown
> when the save/load process fails.  The best behavior here is to have the
> exception pass all the way back to the kernel, which can then perform a
> controlled shutdown.

Yeah, the best thing is probably to throw an exception at the point of failure 
(which includes wrapping any caught IOException or whatever in a 
PlatemailException subclass) and let it bubble up.  We can then catch it at 
the kernel (for now) or catch it somewhere else and try to recover.

> I'm looking to "fail early" here, which I think is the best idea
> generally, however I don't think that a subsystem should do a
> System.exit() call.

Absolutely not.

> Another question is how many such exceptions I should create; one for
> each type of failure?  Should I do something like this:
>
> SaveException extends RegistryException extends PlatemailException
> LoadException extends RegistryException extends PlatemailException
>
> or, should I just have one RegistryException and pass it a string
> specifying the error, which I presume is passed up for
> java.lang.Exception to handle.

It's another of those tradeoff things I guess - we can either have a million 
different classes of exception, or one with a million different strings 
passed in.  Obviously, the middle ground is better.  I'd generally err on the 
side of having different classes, because you can catch them separately.  
Probably the best idea is to have different classes for exceptions that might 
require different handling behaviour, but use the same type for variations on 
a theme that will be handled the same way.

In this case, I'd say that we should handle load exceptions separately from 
store exceptions, because for example we could fail if we can't load a 
settings file, whereas if we can't save it we could still carry on (unless 
the save was part of a shut down).

> I'm leaning toward having an separate exceptions for everything, not
> just things that you might want to differentiate between, though that
> could conceivably be a lot of options.

I think as long as it's part of a hierarchy, we can relatively easily flatten 
it or add levels if the situation requires it in future.  For cases which are 
clearly different though, they should be separate classes.

Neil

Neil,

What'd you think the best exception handling policy is?  I see you've 
created a PlatemailException class, which I've extended to create a 
RegistryException class, however I'm not sure on the best approach.

As I see it, I (currently) want to have an exception that can be thrown 
when the save/load process fails.  The best behavior here is to have the 
exception pass all the way back to the kernel, which can then perform a 
controlled shutdown.

I'm looking to "fail early" here, which I think is the best idea 
generally, however I don't think that a subsystem should do a 
System.exit() call.

Another question is how many such exceptions I should create; one for 
each type of failure?  Should I do something like this:

SaveException extends RegistryException extends PlatemailException
LoadException extends RegistryException extends PlatemailException

or, should I just have one RegistryException and pass it a string 
specifying the error, which I presume is passed up for 
java.lang.Exception to handle.

I'm leaning toward having an separate exceptions for everything, not 
just things that you might want to differentiate between, though that 
could conceivably be a lot of options.

What'd you think?

dgs.

>>I had a brief look at JUnit, however it seems more specialised toward
>>self-contained classses.  Do you know of anything intended for
>>system-wide tests?
> 
> 
> http://www.junit.org/news/extension/index.htm has a few things, none of which 
> I've really looked at in detail.  I suppose it depends on how you want to 
> test the system - JUnit can test larger pieces of functionality, but if you 
> want to run the test from the user's point of view then scripts and expect 
> are probably the way forward (for the CLI at any rate; there are some gui 
> testing tools available too).
> 
> My preference for these would probably be to standardise on expect, or perhaps 
> the python port of expect, which would free us from using tcl.  I don't know 
> how mature the python one is.
> 

I'm not quite sure here.  Expect is probably the best thing for 
user-driven testing, however this seems hard to write tests for, since a 
lot of this will depend on outside sources.

I suppose we could combine this with shell scripts -- ie copy over 
messages store files, perform some operation, and examine the resulting 
files, but then this is really in the domain of shell scripting. 
Perhaps this will be more useful when we have definite use-cases for the 
user, though actually coding the expect scripts would probably depend on 
having a more definite idea of what the input/output would be.

JUnit seems the way to go for less exposed things, like saving settings, 
however again it's hard to check that you're getting the right output 
without effectively reimplementing the code, which can't be right.  It'd 
take a bit of thought to make good tests I think.

dgs.
-- 
David Stocks
Institute of Perception, Action and Behaviour
School of Informatics, University of Edinburgh
+44 (0)131 651 3436

Neil Campbell wrote:
> On Tuesday 15 March 2005 15:07, David Stocks wrote:
> 
>>>>Reconstruction is probably more difficult, since it will mean adding a
>>>>static method to each loadable class.  I'm in favour of making this a
>>>>rigid interface, something like:
>>>>
>>>>static public load(Properties prop, PlatemailKernel kernelRef);
>>>
>>>Can't we have a single method somewhere that does this?  It doesn't seem
>>>quite right to require each class to provide a static method.
>>
>>Perhaps.  Thing is I could imagine some of this getting quite object
>>specific, and I'm not sure it's worth making generic unless we have a
>>more detailed reason to do so.  Certainly, for the types of things that
>>I'm currently thinking about saving/reloading it seems overkill.  As a
>>man with a big beard once said: KISS.
> 
> 
> This method would only really need to decide which object to create - then it 
> can pass the properties object to a constructor to instantiate the 
> appropriate class.  Otherwise, you presumably have a method which selects 
> which class to call this static method on, then you call into that which will 
> basically create you an object in the same way anyway.
> 

This was actually what I was really thinking of anyway.  I don't know 
where the static constructor idea came from.

[...]

> 
> I'm not keen on reflection, generally.  I think once you've determined the 
> type from the string you might be better off deciding yourself which object 
> to create, rather than using the old Class.forName sort of thing.
> 
> Having said that, I appreciate that in the future we may have an unknown 
> number of classes, if it becomes properly configurable.  I'll leave it to you 
> decide which side of the simplicity/extensibility tradeoff we want to sit on 
> for the moment.
> 

I don't think that the reflection really needs to do much; just create 
an instance of the named class.  So all we're doing is creating a 
factory which does this, and passes the

In summary:

public void load(InputStream in) {
   Properties prop;
   PlatemailRegistryFactory factory;

   prop.load(in);
   factory.create(prop, kernelRef);
}

class PlatemailRegistryFactory {

  public static void create(Properties prop, PlatemailKernel kernelRef) {
     Object obj = new (Class.forName(prop.get("type")))(prop, kernelRef);
  }
}

Actually, I don't know much about reflection (that's quite obviously a 
guess at the syntax), so maybe a switch statement would do in the mean time.

> 
>>I had a brief look at JUnit, however it seems more specialised toward
>>self-contained classses.  Do you know of anything intended for
>>system-wide tests?
> 
> 
> http://www.junit.org/news/extension/index.htm has a few things, none of which 
> I've really looked at in detail.  I suppose it depends on how you want to 
> test the system - JUnit can test larger pieces of functionality, but if you 
> want to run the test from the user's point of view then scripts and expect 
> are probably the way forward (for the CLI at any rate; there are some gui 
> testing tools available too).
> 
> My preference for these would probably be to standardise on expect, or perhaps 
> the python port of expect, which would free us from using tcl.  I don't know 
> how mature the python one is.
> 
> 
>>I supose I could expand on my shell scripts, however they aren't
>>currently automatically verified.  Perhaps we also should create a test
>>pop mail account on yahoo for this purpose (thought they don't offer IMAP).
> 
> 
> I've got a test account on thebatcave.org.uk, I can easily create more.  These 
> work for pop and imap.
> 

That would be useful, though it's probably worth sending such account 
details off-list.

> For some more stress-testing, there's also a free IMAP server run by one of 
> the american universities (possibly UW) that you can play with.  I've got it 
> bookmarked at home somewhere.
> 

Yeah, I've head of the UW IMAP server.  Perhaps I should look into that ;)

> 
>>Actually, perhaps I just need to focus unit tests on the correct bit;
>>for example creating unit tests against the console ui should be able to
>>test anything.
> 
> 
> In theory yes, although it's often difficult to test an isolated piece of 
> functionality.
> 
> 
>>>>>If it does genuinely need a MessageStore reference though, I'd like the
>>>>>registry stuff to be able to cope with a dependency like that.  It is
>>>>>conceivable that at some point I'll want to support multiple message
>>>>>stores within a single Platemail instance, so it would be necessary to
>>>>>distinguish between them.  In any case, it seems likely that there will
>>>>>be cause for some sort of dependency information required at some point,
>>>>>because the obejcts won't all remain unrelated.
>>>>
>>>>I'm happy with the MessageStore (MailStore?) needing a reference to the
>>>>kernel, I just think that it's best to have everything it needs
>>>>accessible from that.
>>>
>>>Is that necessarily appropriate in a world with one kernel and multiple
>>>message stores though?
>>
>>I'm not sure as to the point on multiple mailstores.  If each account
>>has it's own "folder" (that's probably an inappropriate term for this)
>>within a hidden root element, then you've got everthing you need.  If
>>you want to attach another type of thing, then you just add in a new
>>child of the root element.
> 
> 
> Yes, but you might want to keep some on disk and some in a database, for 
> example.  There isn't really a facility (nor is there likely to be soon) for 
> 'mounting' stores within one another.
> 
> 

Surely this "mounting" feature is exactly what you're talking about. 
Should this be implemented at the MessageStore level, rather than by 
creating an alternate, MessageStore-level concept.  Indeed, the 
underlying format should be hidden behind a driver abstraction, so all 
you'd need was to specify the type of driver associated with each 
MessageStore.

Probably not important right now though.

>>>>Again, since we can't guarantee the order in which objects are reloaded,
>>>>it's probably also important that the reloading process is random
>>>>access, which means that no object can make explicit assumptions about
>>>>the kernel having loaded anything.
>>>
>>>Presumably you can arrange for all objects to be written after anything
>>>that they depend on, can't you?
>>
>>Yes, but I suspect that way lies madness.  Can you write me a test case
>>where this is strictly necessary?
> 
> 
> Well, if you have a mail source that needs a message store, wouldn't you need 
> to create the former after the latter?
> 

Yes, but you don't need dependency checking for that.  You just need a 
text reference in the MailSource and runtime checking, since these two 
datatypes are separate anyway (we're not storing messages inside the .rc 
file).  This would only break if the message store didn't exist, in 
which case something's been corrupted and all bets are off anyway.

The issue is how we add the MessageStores, which is probably independent 
of other things.  Thus, we have the kernel do it's own setup;

KernelConstructor() {
   reload mail objects (sources etc)
   scan all files in ~/mail
   for each file do
     try to identify it heuristically for each driver
     if a driver claims it, use the driver to reload relevant state data
   end for
}

dgs.
-- 
David Stocks
Institute of Perception, Action and Behaviour
School of Informatics, University of Edinburgh
+44 (0)131 651 3436

On Tuesday 15 March 2005 15:07, David Stocks wrote:
> >>Reconstruction is probably more difficult, since it will mean adding a
> >>static method to each loadable class.  I'm in favour of making this a
> >>rigid interface, something like:
> >>
> >>static public load(Properties prop, PlatemailKernel kernelRef);
> >
> > Can't we have a single method somewhere that does this?  It doesn't seem
> > quite right to require each class to provide a static method.
>
> Perhaps.  Thing is I could imagine some of this getting quite object
> specific, and I'm not sure it's worth making generic unless we have a
> more detailed reason to do so.  Certainly, for the types of things that
> I'm currently thinking about saving/reloading it seems overkill.  As a
> man with a big beard once said: KISS.

This method would only really need to decide which object to create - then it 
can pass the properties object to a constructor to instantiate the 
appropriate class.  Otherwise, you presumably have a method which selects 
which class to call this static method on, then you call into that which will 
basically create you an object in the same way anyway.

> >>Thus, when we do the reload operation inside PlatemailRegistry, it just
> >>iterates through each saved object and calls the appropriate class.
> >>Something like this:
> >>
> >>for each object in InStream do
> >>   prop = deserialise Properties object
> >>   mailObj = Reflection.getObjectType(prop.type)
> >>   mailObject.load(prop, kernelRef)
> >>end
> >
> > Could you expand on the 'Reflection.getObjectType(prop.type)' line
> > please?
>
> The idea was that the object would have a field:
>
> type=POP3MailSource
>
> and then we would need a factory which created the suitable thing using
> reflection.  This we do:
>
> MailObj mailObj; // pointer to generic mail structure
> mailObj = MailObjFactory(prop.get("type")) // create new POP3MailSource
> mailObj.load(prop, kernelRef) // call local setup mechanism
>
> Though we'd need a common hierarchy for this, which I've just realised
> that we don't have.  That could make things less clean, since we'd need
> to push the load() call into the code that does object-specific things
> (the factory).

I'm not keen on reflection, generally.  I think once you've determined the 
type from the string you might be better off deciding yourself which object 
to create, rather than using the old Class.forName sort of thing.

Having said that, I appreciate that in the future we may have an unknown 
number of classes, if it becomes properly configurable.  I'll leave it to you 
decide which side of the simplicity/extensibility tradeoff we want to sit on 
for the moment.

> I had a brief look at JUnit, however it seems more specialised toward
> self-contained classses.  Do you know of anything intended for
> system-wide tests?

http://www.junit.org/news/extension/index.htm has a few things, none of which 
I've really looked at in detail.  I suppose it depends on how you want to 
test the system - JUnit can test larger pieces of functionality, but if you 
want to run the test from the user's point of view then scripts and expect 
are probably the way forward (for the CLI at any rate; there are some gui 
testing tools available too).

My preference for these would probably be to standardise on expect, or perhaps 
the python port of expect, which would free us from using tcl.  I don't know 
how mature the python one is.

> I supose I could expand on my shell scripts, however they aren't
> currently automatically verified.  Perhaps we also should create a test
> pop mail account on yahoo for this purpose (thought they don't offer IMAP).

I've got a test account on thebatcave.org.uk, I can easily create more.  These 
work for pop and imap.

For some more stress-testing, there's also a free IMAP server run by one of 
the american universities (possibly UW) that you can play with.  I've got it 
bookmarked at home somewhere.

> Actually, perhaps I just need to focus unit tests on the correct bit;
> for example creating unit tests against the console ui should be able to
> test anything.

In theory yes, although it's often difficult to test an isolated piece of 
functionality.

> >>>If it does genuinely need a MessageStore reference though, I'd like the
> >>>registry stuff to be able to cope with a dependency like that.  It is
> >>>conceivable that at some point I'll want to support multiple message
> >>>stores within a single Platemail instance, so it would be necessary to
> >>>distinguish between them.  In any case, it seems likely that there will
> >>>be cause for some sort of dependency information required at some point,
> >>>because the obejcts won't all remain unrelated.
> >>
> >>I'm happy with the MessageStore (MailStore?) needing a reference to the
> >>kernel, I just think that it's best to have everything it needs
> >>accessible from that.
> >
> > Is that necessarily appropriate in a world with one kernel and multiple
> > message stores though?
>
> I'm not sure as to the point on multiple mailstores.  If each account
> has it's own "folder" (that's probably an inappropriate term for this)
> within a hidden root element, then you've got everthing you need.  If
> you want to attach another type of thing, then you just add in a new
> child of the root element.

Yes, but you might want to keep some on disk and some in a database, for 
example.  There isn't really a facility (nor is there likely to be soon) for 
'mounting' stores within one another.

> >>Again, since we can't guarantee the order in which objects are reloaded,
> >>it's probably also important that the reloading process is random
> >>access, which means that no object can make explicit assumptions about
> >>the kernel having loaded anything.
> >
> > Presumably you can arrange for all objects to be written after anything
> > that they depend on, can't you?
>
> Yes, but I suspect that way lies madness.  Can you write me a test case
> where this is strictly necessary?

Well, if you have a mail source that needs a message store, wouldn't you need 
to create the former after the latter?

Neil

Neil Campbell wrote:
> On Tuesday 15 March 2005 13:14, David Stocks wrote:
> 
>>>>That's pretty much it.  It's currently only implemented for
>>>>addpopaccount, since that's the only thing I currently use.  Which other
>>>>objects do you believe need saved?
>>>
>>>I think the best approach will be to put the code in with just that
>>>object saved, and we can add code as needed for other objects.  For
>>>example, in my current piece of work I'll need the IMAPMailSource data to
>>>get saved, but I've changed that quite a bit so there's not much point in
>>>you adding a method to it at this stage.
>>
>>It's really simple to add, since all you need to do is call
>>Properties.put(String, String) for each data item you want saved.
>>
>>Reconstruction is probably more difficult, since it will mean adding a
>>static method to each loadable class.  I'm in favour of making this a
>>rigid interface, something like:
>>
>>static public load(Properties prop, PlatemailKernel kernelRef);
> 
> 
> Can't we have a single method somewhere that does this?  It doesn't seem quite 
> right to require each class to provide a static method.
> 

Perhaps.  Thing is I could imagine some of this getting quite object 
specific, and I'm not sure it's worth making generic unless we have a 
more detailed reason to do so.  Certainly, for the types of things that 
I'm currently thinking about saving/reloading it seems overkill.  As a 
man with a big beard once said: KISS.

> 
>>Thus, when we do the reload operation inside PlatemailRegistry, it just
>>iterates through each saved object and calls the appropriate class.
>>Something like this:
>>
>>for each object in InStream do
>>   prop = deserialise Properties object
>>   mailObj = Reflection.getObjectType(prop.type)
>>   mailObject.load(prop, kernelRef)
>>end
> 
> 
> Could you expand on the 'Reflection.getObjectType(prop.type)' line please?
> 

The idea was that the object would have a field:

type=POP3MailSource

and then we would need a factory which created the suitable thing using 
reflection.  This we do:

MailObj mailObj; // pointer to generic mail structure
mailObj = MailObjFactory(prop.get("type")) // create new POP3MailSource
mailObj.load(prop, kernelRef) // call local setup mechanism

Though we'd need a common hierarchy for this, which I've just realised 
that we don't have.  That could make things less clean, since we'd need 
to push the load() call into the code that does object-specific things 
(the factory).

> 
>>This means that the object can't require any information that isn't
>>either saved or accessible from an initialised but otherwise blank
>>kernel, which I think is a semantically correct approach anyway.
> 
> 
> Yes, that sounds perfectly reasonable.
> 
> 
[...]
> 
> I've never been convinced that running it in Eclipse really gives you any 
> benefit.  My preferred approach is to keep an editor on one screen and a 
> terminal for running it on the other.
> 

I had a brief look at JUnit, however it seems more specialised toward 
self-contained classses.  Do you know of anything intended for 
system-wide tests?

I supose I could expand on my shell scripts, however they aren't 
currently automatically verified.  Perhaps we also should create a test 
pop mail account on yahoo for this purpose (thought they don't offer IMAP).

Actually, perhaps I just need to focus unit tests on the correct bit; 
for example creating unit tests against the console ui should be able to 
test anything.

> 
>>>If it does genuinely need a MessageStore reference though, I'd like the
>>>registry stuff to be able to cope with a dependency like that.  It is
>>>conceivable that at some point I'll want to support multiple message
>>>stores within a single Platemail instance, so it would be necessary to
>>>distinguish between them.  In any case, it seems likely that there will
>>>be cause for some sort of dependency information required at some point,
>>>because the obejcts won't all remain unrelated.
>>
>>I'm happy with the MessageStore (MailStore?) needing a reference to the
>>kernel, I just think that it's best to have everything it needs
>>accessible from that.
> 
> 
> Is that necessarily appropriate in a world with one kernel and multiple 
> message stores though?
> 

I'm not sure as to the point on multiple mailstores.  If each account 
has it's own "folder" (that's probably an inappropriate term for this) 
within a hidden root element, then you've got everthing you need.  If 
you want to attach another type of thing, then you just add in a new 
child of the root element.

> 
>>Again, since we can't guarantee the order in which objects are reloaded,
>>it's probably also important that the reloading process is random
>>access, which means that no object can make explicit assumptions about
>>the kernel having loaded anything.
> 
> 
> Presumably you can arrange for all objects to be written after anything that 
> they depend on, can't you?
> 

Yes, but I suspect that way lies madness.  Can you write me a test case 
where this is strictly necessary?

> 
>>Accordingly, we should have a definite idea about what state an
>>initialised but "blank" kernel is in; mainly I'm thinking that this
>>holds a reference to all of the functionally registered things (which
>>interfaces are available, which commands for the cli etc).
>>
>>I'm inclined to think that there should be a definite "root" to the
>>mailstore, that isn't going to change anytime soon.  This should
>>probably be invisible to the user (ie not a part of a swing JTree or
>>anything).  This would allow for later addition of multiple mail stores
>>(a la Thunderbird, which separates IMAP accounts, POP, news, local etc)
> 
> 
> This is already how it works in Platemail (except for news support and so on, 
> of course).
> 
> 
>>I'm also thinking that the kernel itself should be savable, since it
>>should save as a generic saving point for all of these "misc" items
>>(preferred UI, default mail account etc).
> 
> 
> Yep.
> 
> 
>>Right, I'm off to learn fascinating things about how Rhinolophus
>>ferrumequinm's use of constant frequency echo's allows it to use the
>>doppler effect to detect moth flight vectors.  More interesting than I
>>imagine it sounds.
> 
> 
> You should have asked me - I'm an expert in that field.
> 

Turns out it wasn't that interesting.  Meh.

dgs.
-- 
David Stocks
Institute of Perception, Action and Behaviour
School of Informatics, University of Edinburgh
+44 (0)131 651 3436

On Tuesday 15 March 2005 13:14, David Stocks wrote:
> >>That's pretty much it.  It's currently only implemented for
> >>addpopaccount, since that's the only thing I currently use.  Which other
> >>objects do you believe need saved?
> >
> > I think the best approach will be to put the code in with just that
> > object saved, and we can add code as needed for other objects.  For
> > example, in my current piece of work I'll need the IMAPMailSource data to
> > get saved, but I've changed that quite a bit so there's not much point in
> > you adding a method to it at this stage.
>
> It's really simple to add, since all you need to do is call
> Properties.put(String, String) for each data item you want saved.
>
> Reconstruction is probably more difficult, since it will mean adding a
> static method to each loadable class.  I'm in favour of making this a
> rigid interface, something like:
>
> static public load(Properties prop, PlatemailKernel kernelRef);

Can't we have a single method somewhere that does this?  It doesn't seem quite 
right to require each class to provide a static method.

> Thus, when we do the reload operation inside PlatemailRegistry, it just
> iterates through each saved object and calls the appropriate class.
> Something like this:
>
> for each object in InStream do
>    prop = deserialise Properties object
>    mailObj = Reflection.getObjectType(prop.type)
>    mailObject.load(prop, kernelRef)
> end

Could you expand on the 'Reflection.getObjectType(prop.type)' line please?

> This means that the object can't require any information that isn't
> either saved or accessible from an initialised but otherwise blank
> kernel, which I think is a semantically correct approach anyway.

Yes, that sounds perfectly reasonable.

> > Yay!  Anything you can do by way of automatable testing would be very
> > good. As for Eclipse, I tend to run everything in a normal shell anyway,
> > because I'm not keen on how Eclipse handles it (and it doesn't colour my
> > prompt either).
>
> To be honest I'm always a little annoyed that I loose so much screen
> space; even in xemacs's compile frame you loose the bottom 5 lines of
> the screen (which is why I normally have another windows setup to
> compile/run).  You loose code space, and you don't really get to see
> error messages either.  Perhaps a bigger monitor would fix this, though
> I already have a 19" and that should be fine.  I imagine that this would
> be useful to work dual-headed, but I doubt that eclipse will like itself
> being split like that.

I've never been convinced that running it in Eclipse really gives you any 
benefit.  My preferred approach is to keep an editor on one screen and a 
terminal for running it on the other.

> > If it does genuinely need a MessageStore reference though, I'd like the
> > registry stuff to be able to cope with a dependency like that.  It is
> > conceivable that at some point I'll want to support multiple message
> > stores within a single Platemail instance, so it would be necessary to
> > distinguish between them.  In any case, it seems likely that there will
> > be cause for some sort of dependency information required at some point,
> > because the obejcts won't all remain unrelated.
>
> I'm happy with the MessageStore (MailStore?) needing a reference to the
> kernel, I just think that it's best to have everything it needs
> accessible from that.

Is that necessarily appropriate in a world with one kernel and multiple 
message stores though?

> Again, since we can't guarantee the order in which objects are reloaded,
> it's probably also important that the reloading process is random
> access, which means that no object can make explicit assumptions about
> the kernel having loaded anything.

Presumably you can arrange for all objects to be written after anything that 
they depend on, can't you?

> Accordingly, we should have a definite idea about what state an
> initialised but "blank" kernel is in; mainly I'm thinking that this
> holds a reference to all of the functionally registered things (which
> interfaces are available, which commands for the cli etc).
>
> I'm inclined to think that there should be a definite "root" to the
> mailstore, that isn't going to change anytime soon.  This should
> probably be invisible to the user (ie not a part of a swing JTree or
> anything).  This would allow for later addition of multiple mail stores
> (a la Thunderbird, which separates IMAP accounts, POP, news, local etc)

This is already how it works in Platemail (except for news support and so on, 
of course).

> I'm also thinking that the kernel itself should be savable, since it
> should save as a generic saving point for all of these "misc" items
> (preferred UI, default mail account etc).

Yep.

> Right, I'm off to learn fascinating things about how Rhinolophus
> ferrumequinm's use of constant frequency echo's allows it to use the
> doppler effect to detect moth flight vectors.  More interesting than I
> imagine it sounds.

You should have asked me - I'm an expert in that field.

Neil Campbell wrote:
> On Tuesday 15 March 2005 00:05, David Stocks wrote:
> 
>>Well, I've fixed the problem with saving the properties, and I can now
>>save anything string like that you fancy to ~/.platemailrc.
> 
> 
> Excellent stuff.
> 
> 
>>To add an object:
>>1) implement the interface platemail.registry.RegistryInterface
>>   - this basically means adding the function register(), and a
>>Java.Util.Properties object
>>
>>2) have something call this, namely the cli interface
>>
>>That's pretty much it.  It's currently only implemented for
>>addpopaccount, since that's the only thing I currently use.  Which other
>>objects do you believe need saved?
> 
> 
> I think the best approach will be to put the code in with just that object 
> saved, and we can add code as needed for other objects.  For example, in my 
> current piece of work I'll need the IMAPMailSource data to get saved, but 
> I've changed that quite a bit so there's not much point in you adding a 
> method to it at this stage.
> 

It's really simple to add, since all you need to do is call 
Properties.put(String, String) for each data item you want saved.

Reconstruction is probably more difficult, since it will mean adding a 
static method to each loadable class.  I'm in favour of making this a 
rigid interface, something like:

static public load(Properties prop, PlatemailKernel kernelRef);

Thus, when we do the reload operation inside PlatemailRegistry, it just 
iterates through each saved object and calls the appropriate class. 
Something like this:

for each object in InStream do
   prop = deserialise Properties object
   mailObj = Reflection.getObjectType(prop.type)
   mailObject.load(prop, kernelRef)
end

This means that the object can't require any information that isn't 
either saved or accessible from an initialised but otherwise blank 
kernel, which I think is a semantically correct approach anyway.

> 
>>I think that a testing framework would be useful here; I've created a
>>test script which just pipes the input in but I have to do this outside
>>Eclipse since I don't know how to make eclipse do this.  Actually, I've
>>pretty much moved back to xemacs anyway, but I'd be interested to look
>>at unit testing.
> 
> 
> Yay!  Anything you can do by way of automatable testing would be very good.  
> As for Eclipse, I tend to run everything in a normal shell anyway, because 
> I'm not keen on how Eclipse handles it (and it doesn't colour my prompt 
> either).
> 

To be honest I'm always a little annoyed that I loose so much screen 
space; even in xemacs's compile frame you loose the bottom 5 lines of 
the screen (which is why I normally have another windows setup to 
compile/run).  You loose code space, and you don't really get to see 
error messages either.  Perhaps a bigger monitor would fix this, though 
I already have a 19" and that should be fine.  I imagine that this would 
be useful to work dual-headed, but I doubt that eclipse will like itself 
being split like that.

> 
>>The next, more complex, thing to do is to implement the reloading code.
>>  I'm thinking that this should be done by adding a constructor method
>>that takes a Properties object.  This gets more complex if there are
>>dependencies; for example POP3MailSource taking a MessageStore argument.
>>  It'd be possible to save such dependancies, however this could get
>>into loops etc. and probably is best avoided if it's possible to keep
>>things simple.
> 
> 
> I think if there are any cyclic dependencies, it's a problem with the object 
> hierarchy anyway, so the persistence code shouldn't have to worry about that.  
> The structure of the objects that need to be saved should be fairly simple, 
> and if this isn't the case lets refactor them rather than muddying the 
> load/save code.  I'm guessing that simple dependencies shouldn't be too 
> tricky to handle.
> 
> 
>>What's the point behind the MessageStore argument?  Is it possible to
>>have more than one of them?  It seems that the kernel has one of these,
>>which I presume is the root of all mail storage.  If there is only on
>>mailstore then it shouldn't be necessary to pass it as an argument to
>>the constructor.  If you've just done this to allow the created
>>mailsource (ie POP3MailSource) to hold a reference to the root, then
>>it's probably worth doing this some other way, since when the registry
>>does the restore it doesn't naturally have this reference.  It could
>>probably derive it from the global kernel reference, however the child
>>POP3MailSource could just do that itself anyway; it seems redundant and
>>unless there's something else going on here then I think that we should
>>bin that argument.
> 
> 
> Yes, at present there will only be one MessageStore, and it will be the one 
> the kernel has a reference to.  I can't recall exactly why the POP3MailSource 
> has a reference to it (and sourceforge's CVS seems to playing up at the 
> moment), but it's possible that it can be factored out - if you can do that 
> cleanly then by all means feel free.
> 
> If it does genuinely need a MessageStore reference though, I'd like the 
> registry stuff to be able to cope with a dependency like that.  It is 
> conceivable that at some point I'll want to support multiple message stores 
> within a single Platemail instance, so it would be necessary to distinguish 
> between them.  In any case, it seems likely that there will be cause for some 
> sort of dependency information required at some point, because the obejcts 
> won't all remain unrelated.
> 

I'm happy with the MessageStore (MailStore?) needing a reference to the 
kernel, I just think that it's best to have everything it needs 
accessible from that.

Again, since we can't guarantee the order in which objects are reloaded, 
it's probably also important that the reloading process is random 
access, which means that no object can make explicit assumptions about 
the kernel having loaded anything.

Accordingly, we should have a definite idea about what state an 
initialised but "blank" kernel is in; mainly I'm thinking that this 
holds a reference to all of the functionally registered things (which 
interfaces are available, which commands for the cli etc).

I'm inclined to think that there should be a definite "root" to the 
mailstore, that isn't going to change anytime soon.  This should 
probably be invisible to the user (ie not a part of a swing JTree or 
anything).  This would allow for later addition of multiple mail stores 
(a la Thunderbird, which separates IMAP accounts, POP, news, local etc)

I'm also thinking that the kernel itself should be savable, since it 
should save as a generic saving point for all of these "misc" items 
(preferred UI, default mail account etc).

Right, I'm off to learn fascinating things about how Rhinolophus 
ferrumequinm's use of constant frequency echo's allows it to use the 
doppler effect to detect moth flight vectors.  More interesting than I 
imagine it sounds.

dgs.
-- 
David Stocks
Institute of Perception, Action and Behaviour
School of Informatics, University of Edinburgh
+44 (0)131 651 3436

On Tuesday 15 March 2005 00:05, David Stocks wrote:
> Well, I've fixed the problem with saving the properties, and I can now
> save anything string like that you fancy to ~/.platemailrc.

Excellent stuff.

> To add an object:
> 1) implement the interface platemail.registry.RegistryInterface
>    - this basically means adding the function register(), and a
> Java.Util.Properties object
>
> 2) have something call this, namely the cli interface
>
> That's pretty much it.  It's currently only implemented for
> addpopaccount, since that's the only thing I currently use.  Which other
> objects do you believe need saved?

I think the best approach will be to put the code in with just that object 
saved, and we can add code as needed for other objects.  For example, in my 
current piece of work I'll need the IMAPMailSource data to get saved, but 
I've changed that quite a bit so there's not much point in you adding a 
method to it at this stage.

> I think that a testing framework would be useful here; I've created a
> test script which just pipes the input in but I have to do this outside
> Eclipse since I don't know how to make eclipse do this.  Actually, I've
> pretty much moved back to xemacs anyway, but I'd be interested to look
> at unit testing.

Yay!  Anything you can do by way of automatable testing would be very good.  
As for Eclipse, I tend to run everything in a normal shell anyway, because 
I'm not keen on how Eclipse handles it (and it doesn't colour my prompt 
either).

> The next, more complex, thing to do is to implement the reloading code.
>   I'm thinking that this should be done by adding a constructor method
> that takes a Properties object.  This gets more complex if there are
> dependencies; for example POP3MailSource taking a MessageStore argument.
>   It'd be possible to save such dependancies, however this could get
> into loops etc. and probably is best avoided if it's possible to keep
> things simple.

I think if there are any cyclic dependencies, it's a problem with the object 
hierarchy anyway, so the persistence code shouldn't have to worry about that.  
The structure of the objects that need to be saved should be fairly simple, 
and if this isn't the case lets refactor them rather than muddying the 
load/save code.  I'm guessing that simple dependencies shouldn't be too 
tricky to handle.

> What's the point behind the MessageStore argument?  Is it possible to
> have more than one of them?  It seems that the kernel has one of these,
> which I presume is the root of all mail storage.  If there is only on
> mailstore then it shouldn't be necessary to pass it as an argument to
> the constructor.  If you've just done this to allow the created
> mailsource (ie POP3MailSource) to hold a reference to the root, then
> it's probably worth doing this some other way, since when the registry
> does the restore it doesn't naturally have this reference.  It could
> probably derive it from the global kernel reference, however the child
> POP3MailSource could just do that itself anyway; it seems redundant and
> unless there's something else going on here then I think that we should
> bin that argument.

Yes, at present there will only be one MessageStore, and it will be the one 
the kernel has a reference to.  I can't recall exactly why the POP3MailSource 
has a reference to it (and sourceforge's CVS seems to playing up at the 
moment), but it's possible that it can be factored out - if you can do that 
cleanly then by all means feel free.

If it does genuinely need a MessageStore reference though, I'd like the 
registry stuff to be able to cope with a dependency like that.  It is 
conceivable that at some point I'll want to support multiple message stores 
within a single Platemail instance, so it would be necessary to distinguish 
between them.  In any case, it seems likely that there will be cause for some 
sort of dependency information required at some point, because the obejcts 
won't all remain unrelated.

Neil

Well, I've fixed the problem with saving the properties, and I can now 
save anything string like that you fancy to ~/.platemailrc.

To add an object:
1) implement the interface platemail.registry.RegistryInterface
   - this basically means adding the function register(), and a 
Java.Util.Properties object

2) have something call this, namely the cli interface

That's pretty much it.  It's currently only implemented for 
addpopaccount, since that's the only thing I currently use.  Which other 
objects do you believe need saved?

I think that a testing framework would be useful here; I've created a 
test script which just pipes the input in but I have to do this outside 
Eclipse since I don't know how to make eclipse do this.  Actually, I've 
pretty much moved back to xemacs anyway, but I'd be interested to look 
at unit testing.

The next, more complex, thing to do is to implement the reloading code. 
  I'm thinking that this should be done by adding a constructor method 
that takes a Properties object.  This gets more complex if there are 
dependencies; for example POP3MailSource taking a MessageStore argument. 
  It'd be possible to save such dependancies, however this could get 
into loops etc. and probably is best avoided if it's possible to keep 
things simple.

What's the point behind the MessageStore argument?  Is it possible to 
have more than one of them?  It seems that the kernel has one of these, 
which I presume is the root of all mail storage.  If there is only on 
mailstore then it shouldn't be necessary to pass it as an argument to 
the constructor.  If you've just done this to allow the created 
mailsource (ie POP3MailSource) to hold a reference to the root, then 
it's probably worth doing this some other way, since when the registry 
does the restore it doesn't naturally have this reference.  It could 
probably derive it from the global kernel reference, however the child 
POP3MailSource could just do that itself anyway; it seems redundant and 
unless there's something else going on here then I think that we should 
bin that argument.

Cheers,
dgs.

On Monday 31 January 2005 16:10, David Stocks wrote:
> I'd forgotten about the static kernel accessor.  I'll use that instead.
>   I have the natural distrust of static things, however that will save a
> lot of argument passing.  Lovely.

I was thinking about removing a load of stuff from Kernel and putting it into 
a PlatemailContext, because most of what the kernel does at the moment is 
hold things.

> > OK.  Let me know when your OK to use the list again.
>
> Hopefully this will get to you.  If so, then everything is peachy.

All OK.

> >>Right.  I need to book myself some flights to Madrid.
> >
> > Madrid, eh?  Off jet-setting?
>
> Project workshop.  Plus some sight seeing, of course.  I love working
> for the EU.

Very nice.

> I'm off to Dublin to see Jenny the week after, however I need to pay for
> that myself unfortunately.

Damn stingy universities.  Who do they think they are?

>>>
>>>>>I'm going to start a bit of Platemail hacking now.
>>>>
>>>>Good, good.  I did some poking, and I think I've gotten the basic idea
>>>>for how it's going to work.  I've implemented a little sample which
>>>>lists the Vector for Properties for pop3mailsource.  That's quite
>>>>straight forward, however I really think we need a more
>>>>organised/formal way of adding the classes that are to be saved.
>>>>
>>>>To whit;
>>>>
>>>>1) Modifying the constructors to add a reference to the registry
>>>
>>>I'd be careful with doing this in the constructor; there may be cases
>>>where you don't want to add them (though none that immediately occur). 
>>>Perhaps you
>>>
>>>could just have a register() method to be called after construction, and
>>>optionally a static method on the class which creates you a new object
>>>and registers it for you.  The other alternative is to pass a boolean
>>>flag to the
>>>
>>>constructor, but it seems to me that registration isn't really a
>>>constructor task.
>>
>>I know what you're coming from, however I'm not sure what the best thing to
>>do is.  I think that the objects need to have a reference to the registry
>>somehow, and once they've got that then they can register/deregister
>>themselves all day long.  That's the important thing which I want put into
>>the constructor, so that an outside called can just say obj.register();
>>rather than the C-style obj.register(registry).
> 
> 
> There's a static accessor method on PlatemailKernel that gives you the current 
> kernel instance, so if your registry hangs off PlatemailKernel you can get it 
> from there.  PlatemailKernel is (currently) a singleton, and if that ever 
> changes I'll change the static accessor to do the right thing.  This is 
> probably easier than passing around a reference to the registry all the time.
> 

I'd forgotten about the static kernel accessor.  I'll use that instead. 
  I have the natural distrust of static things, however that will save a 
lot of argument passing.  Lovely.

> 
>>As to adding the call to register() outside the constructor, I'm not really
>>that bothered.  It's probably a bit icky (error prone) to require a manual
>>call to the register, however without multiple inheritance there's not a
>>lot we can really do.  It would be nice to have this handled automatically.
>>
>>Maybe.
>>
>>Creating a static constructor and having that do the registration doesn't
>>seem to solve any of the problems as best I can see though, that just seems
>>to be moving the problem around.
> 
> 
> Not really, I think it gives you a separation between object construction and 
> what you do with the object once you've made it.  If you want it registered, 
> you can call, say POP3MailSource.getRegisteredMailSource(args), and if you 
> don't you can always call new POP3MailSource(args).  I imagine that the 
> former will be the common case, so you can make the constructor itself 
> private until we decide it's necessary to expose it; that'll prevent anyone 
> accidentally forgetting to register the object in the meantime.
> 

Sure.  I don't think it's that important, but you know, whatever.

> 
>>>>2) Adding
>>>>   - a Vector of properties
>>>
>>>Why not just a Properties object?
>>
>>Because I didn't realise that it's actually a list (hashtable) itself. 
>>I'll change this.
> 
> 
> Ah, OK.
> 
> 
>>>>   - a method called by the constructor (?) to populate that list
>>>>   - a method to register the properties vector
>>>>   - a method to unregister the properties vector
>>>>
>>>>Currently, I just call these methods "manually" from the class
>>>>corresponding to "addpopaccount", however these should really be
>>>>self-contained within each class.  Adding an interface to specify these
>>>>properties would probably be the best plan.
>>>
>>>OK.  Are you going to specify a common interface (PlatemailObject or
>>>RegisteredObject or something) that all filters, mail sources and so on
>>>can implement?
>>
>>Yeah, that sounds like the best plan.  I'll have to do a bit of reading up
>>about how interfaces work in Java, but I think that's reasonable.
> 
> 
> Cool.
> 
> 
>>Oh, this is off-list but that's because I've had some mail issues.  I've
>>changed my sourceforge address to work, which will be easier for me to
>>access from now on.
> 
> 
> OK.  Let me know when your OK to use the list again.
> 

Hopefully this will get to you.  If so, then everything is peachy.

> 
>>Right.  I need to book myself some flights to Madrid.
> 
> 
> Madrid, eh?  Off jet-setting?
> 

Project workshop.  Plus some sight seeing, of course.  I love working 
for the EU.

I'm off to Dublin to see Jenny the week after, however I need to pay for 
that myself unfortunately.

dgs.
-- 
David Stocks
Institute of Perception, Action and Behaviour
School of Informatics, University of Edinburgh
+44 (0)131 651 3436

On Monday 31 January 2005 10:00, you wrote:
> > I'm going to start a bit of Platemail hacking now.
>
> Good, good. =A0I did some poking, and I think I've gotten the basic idea
> for how it's going to work. =A0I've implemented a little sample which
> lists the Vector for Properties for pop3mailsource. =A0That's quite
> straight forward, however I really think we need a more organised/formal
> way of adding the classes that are to be saved.
>
> To whit;
>
> 1) Modifying the constructors to add a reference to the registry

I'd be careful with doing this in the constructor; there may be cases where=
=20
you don't want to add them (though none that immediately occur).  Perhaps y=
ou=20
could just have a register() method to be called after construction, and=20
optionally a static method on the class which creates you a new object and=
=20
registers it for you.  The other alternative is to pass a boolean flag to t=
he=20
constructor, but it seems to me that registration isn't really a constructo=
r=20
task.

> 2) Adding
> =A0 =A0- a Vector of properties

Why not just a Properties object?

> =A0 =A0- a method called by the constructor (?) to populate that list
> =A0 =A0- a method to register the properties vector
> =A0 =A0- a method to unregister the properties vector
>
> Currently, I just call these methods "manually" from the class
> corresponding to "addpopaccount", however these should really be
> self-contained within each class. =A0Adding an interface to specify these
> properties would probably be the best plan.

OK.  Are you going to specify a common interface (PlatemailObject or=20
RegisteredObject or something) that all filters, mail sources and so on can=
=20
implement?

Cheers,
Neil

Task #111442 has been updated. 

Project: Platemail
Subproject: Future development
Summary: Developer docs/API spec
Complete: 0%
Status: Open
Authority  : batneil
Assigned to: nobody

Description: Document the packages and interfaces, document the interface between the Platemail core and the user interfaces.

-------------------------------------------------------
For more info, visit:

http://sourceforge.net/pm/task.php?func=detailtask&project_task_id=111442&group_id=71793&group_project_id=41140

Task #111441 has been updated. 

Project: Platemail
Subproject: Future development
Summary: User documentation
Complete: 0%
Status: Open
Authority  : batneil
Assigned to: nobody

Description: Create user docs, tutorials, installation guides, etc.

-------------------------------------------------------
For more info, visit:

http://sourceforge.net/pm/task.php?func=detailtask&project_task_id=111441&group_id=71793&group_project_id=41140

2003	Jan (1)	Feb	Mar	Apr	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec
2004	Jan	Feb	Mar	Apr	May (1)	Jun	Jul	Aug	Sep	Oct	Nov (1)	Dec (8)
2005	Jan (16)	Feb	Mar (15)	Apr	May	Jun	Jul	Aug	Sep (4)	Oct	Nov	Dec
2006	Jan	Feb	Mar	Apr	May (1)	Jun	Jul	Aug	Sep	Oct	Nov	Dec

platemail-developer Mailing List for Platemail

platemail-developer — Mailing list for discussion of Platemail development