pclasses-devel — P::Classes developer mailing-list

Re: [pclasses-devel] ??? move include/pclasses to include/pclasses2?

[Christian P. <cp...@se...>]

Am Freitag 24 Dezember 2004 00:38 schrieb stephan beal:
> Yo,
>
> Christian - yesterday you said you'd like the build to install the
> headers to prefix/include/pclasses2.
>
> If you still want to do that, can we rename include/pclasses? That would
> mean sed'ing the rest of the files to fix the path, but i think it is
> worth it.
>
> Optionally, keep include/pclasses and install under
> prefix/include/pclasses. This will of course break 1.x clients...
>
> Wait - no it won't: 1.x and 2.x use 100% different file names! All
> headers are now upper-case and libs are now libpclasses_xxx.so.
>
> So we could keep the headers under include/pclasses and <pclasses/...>,
> instead of pclasses2.
>
> Just let me know which one you prefer so i can change the build
> accordingly.
>
> See ya!

Lets stick with "include/pclasses" for now.

Greetings,
and have a nice Christmas


Christian

Re: [pclasses-devel] Mime ported - please approve or disapprove!

[Christian P. <cp...@se...>]

Am Freitag 24 Dezember 2004 12:32 schrieb stephan beal:
> On Friday 24 December 2004 03:52, stephan beal wrote:
> > Just comitted:
> >
> > P::System::MimeType - slightly different interface (no file
> > extensions). Container-friendly, via operator< and operator==.
>
> ...
>
> > It compiles and links, but is untested.
>
> As you probably saw via pclasses-cvs, this is now working. Some notable
> changes from the p1 code are:
>
> - MimeType is now default constructable (will need for serialization).
>
> - MimeType is now container-friendly, and can be used as a key.
>
> - MimeTypeDb is no longer a singleton, but a shared instance is provided
> via the conventional instance() interface. i changed this for a couple
> reasons, but one of them is because there are some cool potential uses
> for local Mime DBs in other parts of the code (later on) and in client
> code.
>
> - MimeTypeDb no longer reads the system-wide mime db at ctor type.
> Instead this is delegated to the instance() object - only that object
> loads mime.types by default.
>
> - MimeTypeDb::instance() is now post-main() safe, and the db is
> repopulated if instance() is called after the shared object has been
> destroyed post-main().
>
>
> - Added MimeTypeDb::FileExtensionsMap typedef, plus related functions.
> File extensions mappings are now done once per MimeType media-subtype
> combination, instead of once per instance of MimeType. This change made
> making MimeType container-friendly straightforward, and also eases
> client-side use of MimeType (IMO).


Great work!

but... we should move MimeType and MimeTypeDb to a different module, since it 
is on a higher level than the System abstraction classes.
How about IO/ or Util/ or even Net/ ?

I would like to see a MimeMagic (detection of the Mimetype by reading the 
contents of the file) feature added to MimeType/MimeTypeDb.



Greetings,

Christian

[pclasses-devel] toc build tree HOWTO

[stephan b. <st...@s1...>]

Yo!

Here's an overview of how to try out the toc-based build tree...

First, update your pclasses2 tree to HEAD.

Some of the new files:

./configure - check this out from CVS, replacing the AT-generated copy. 
PLEASE do NOT commit an AT-generated configure script over this one! 
(But i know it's gonna happen by accident at some point.) The only time 
this file is edited is to change the PACKAGE_VERSION.

./configure.pclasses2 - this is the main configure script. It is called 
by the toc framework after toc does it's own bootstrapping. configure 
customizations go here.

./configure.toc is a stub file for working with AT. Ignore it - you'll 
never need to know it exists.

./toc.pclasses.help contains the help text for P-specific configure 
options.

./toc/... The entire build system lives here, which means it's always 
compatible with the version of the tree it is shipped with. (This is 
where AT sucks so badly.)

Makefile.toc (in every dir): the toc-based makefiles. They are called by 
Makefile stubs which get created by ./configure. Normally toc does not 
generate 'Makefile' (they are hand-implemented), but in this case we 
generate them to allow it to cooperate with AT. Put your Makefile code 
in Makefile.toc. toc will not modify your makefile at all. To generate 
a Makefile for a dir, run configure or copy one from any other dir 
(they are all 100% identical).


As usual, run ./configure --help for info.

Switching between toc and AT:

To use the AT support, simply use AT to rebuild the configure script and 
Makefiles. To use toc, replace the AT configure with the one from CVS 
and run configure. That's all there is to it.


ACHTUNG: the toc port of the build tree is not 100% complete:

- It builds the posix-compatible sources only, because i've only got a 
Suse Linux box. For the SharedLib support it currently uses libltdl or 
libdl, whichever the configure script finds (preferring ltdl).

- It builds all of the libs, but ./configure does not yet provide all of 
the necessary linker flags for linking against those libs. There are 
lots of fixes to be done here.

- Many of the config values are hard-coded at the moment. See 
configure.pclasses2 for notes.



Some things worth knowing about toc:

- toc = "the other ./configure"

- ./configure --help

- ./configure --help-tests

- If you export the env var 'prefix' then you don't need to pass it 
to ./configure. Useful when editing the configure script and re-running 
it a lot of times.

- Only works on systems hosting GNU versions of tools like Make, Tar, 
Bash, Awk, etc. Only common system tools are needed, but Solaris 
versions of Make or tar (and other apps) will not work.

- See ./toc/tests for the sources for the config tests run by 
configure.pclasses2.

- make dist. Works, but is probably still missing some files.

- toc's shell API is defined/documented in ./toc/sbin/toc_core.sh. Only 
the API which is actually used in configure.pclasses2 
and ./toc/tests/*.sh are of interest to clients.

- The makefile framework is quite modular. See ./toc/make for the 
various snippets (not all of which are used). See, e.g., 
toc/make/SHARED_LIBS.make for the snippet which does all DLL building. 
We can swap out these snippets for platforms which need custom 
handling.

- You do NOT have to run configure when you change your makefiles. You 
only need to run it when:
a) Initial config
b) You want to, because AT has trained you to.
c) You've added a subdir. You do NOT need to add the subdir to the 
config script, but you do need to generate toc.make (and friends) for 
that dir. configure generates a toc.make for any dir containing a 
Makefile{.toc} with the line "include toc.make".

- The make code explicitely takes advantage of features of GNU make, 
making it more powerful than portable Makefile.am code.

- "Project-global" makefile code should go into ./toc.pclasses2.make.at,
then re-run configure to regenerate ./toc.pclasses2.make. It then gets 
included into any Makefile.toc automatically.

- If you have lyx installed, try 'cd doc/manual; make LYX_EXPORT'.

- When using DIST_FILES in your Makefile.toc, NEVER use 
'DIST_FILES=...', but always use 'DIST_FILES+=...'. (Note the '+='.)

- The full toc docs (a bit outdated, but still mostly accurate) are 
available via toc.sourceforge.net.

- If you have a dir full of files and want to generate a mostly-usable 
Makefile.toc automatically, do this from the top srcdir:
	./toc/bin/create_makefile_stubs.sh ./my/subdir
	cd ./my/subdir
	mv Makefile.suggestion Makefile.toc
	emacs Makefile.toc ...
Normally the generated makefile contains 90%+ of what you need to build 
the dir, and for simple dirs it can do all the work itself.

- If we ever decide to drop AT then we can rename Makefile.toc ==> 
Makefile.

- Ask me if you have questions about what toc can/cannot do, or how to 
do a specific thing in toc. If something can't be currently done, it 
can be added. i've spent almost 2 years refining toc, so nobody knows 
it better than i ;).



Okay, that was more than i wanted to say, but i guess i just want to 
convince everyone that toc is pretty flexible. i've used it for ALL of 
my source trees since almost 2 years, so i know it easy to 
hack/extend/customize, and MUCH easier to maintain than autotools.

:)

See ya!

-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

Re: [pclasses-devel] Mime ported - please approve or disapprove!

[stephan b. <st...@s1...>]

On Friday 24 December 2004 03:52, stephan beal wrote:
> Just comitted:
>
> P::System::MimeType - slightly different interface (no file
> extensions). Container-friendly, via operator< and operator==.
...
> It compiles and links, but is untested.

As you probably saw via pclasses-cvs, this is now working. Some notable 
changes from the p1 code are:

- MimeType is now default constructable (will need for serialization).

- MimeType is now container-friendly, and can be used as a key.

- MimeTypeDb is no longer a singleton, but a shared instance is provided 
via the conventional instance() interface. i changed this for a couple 
reasons, but one of them is because there are some cool potential uses 
for local Mime DBs in other parts of the code (later on) and in client 
code.

- MimeTypeDb no longer reads the system-wide mime db at ctor type. 
Instead this is delegated to the instance() object - only that object 
loads mime.types by default.

- MimeTypeDb::instance() is now post-main() safe, and the db is 
repopulated if instance() is called after the shared object has been 
destroyed post-main().


- Added MimeTypeDb::FileExtensionsMap typedef, plus related functions. 
File extensions mappings are now done once per MimeType media-subtype 
combination, instead of once per instance of MimeType. This change made 
making MimeType container-friendly straightforward, and also eases 
client-side use of MimeType (IMO).


-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

Re: Fwd: Re: [pclasses-devel] P::Classes experimental code

[stephan b. <st...@s1...>]

On Friday 24 December 2004 10:29, Marc Duerner wrote:
> Prefs is an API for loading saving/loading application preferences as
> well as for all sorts of config files. The important thing is that it
> doesnt build on simple string maps but on concrete Entry and Section
> types. So when you change someting in the Prefs and forget to change
> it in your program you are more likely to realise it at compile time.
>

If Entry and Section are made Serializable (automatically the case if 
they are i/ostreamable) then serialzing a Prefs is as simple as 
serializing a map.

Let's take a look...

> class Section {
...
> template <class ValueT>
> class ValueEntry : public Entry {
...
> The Prefs class holds the root section and a PrefsStore class which
> knows how to load and save the data. This would be where I have to
> look at s11n.

So far i see absolutely no problem with s11n compatibility here. Your 
ValueType holds one key element: lexical castability. When a type has 
that it is simpler to serialize, because we can internally simply treat 
it as a string.

> @Stephan what would you suggest? wrapping up s11n in IniStore,
> XmlStore etc? Or kick out PrefsStore and use s11n? It would also be
> interesting to have the parsers as seperate classes in P2.

i would first suggest writing an s11n proxy for the Prefs tree. The only 
thing which might need to change in Prefs (don't yet know) is adding 
some accessors so that s11n can get/set all of the internal data.
i will be happy to experiment with this for you. Caveats:

a) P2 isn't at a state where i can build ps11n against it.
b) i will either use ps11n (with P1) or skip the ps11n support for now 
and simply use s11nlite.

The advantage of a proxy is:

a) Your type doesn't know it's a Serializable.
b) We can swap out proxies later without touching Prefs. That said, i 
*may* need to tweak prefs a bit to make it more s11n-friendly. For 
example, types which are not Default Constructable can't yet be handled 
by s11n without some arcane back-end work involving a custom object 
factory for that type.
c) If the code fails miserably, we can through it away without editing 
Prefs. 


> One other idea I had was to have Iterators that stride (hope that is
> the correct term) over one type of entry. 

"stride" is technically correct, but "traverse" or "iterate" would be 
the normal words, i think. :)

> - Impelment ResolvConf derived from Prefs with its store class for
> the resolv.conf config file format with a NameserverEntry class
> derived from i.e. StringEntry
> - Use "stride" iterator for NameserverEntries to print all nameserver
> entries.

One of the few conventions that s11n really enforces on data is that it 
be structurable similarly to a DOM. That doesn't mean the data file HAS 
to be like a DOM, it just means that as long as we can represent it in 
memory as a DOM-like structure, s11n can work with it.

> The "stride" or "select" iterator was rather easy to implement as a
> template class encapsulating a normal "linear" iterator. operator++
> has to do some type checking though. But maybe that should not be
> part of Prefs but part of the the containers, with SelectTraits or
> something.

That is a good point. i'm a big fan of type traits.

Is the current Prefs interface the one from the P1 CVS tree? If so i 
will take a look and let you know what s11n can or cannot do using the 
current structure.

-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

Re: Fwd: Re: [pclasses-devel] P::Classes experimental code

[Marc D. <bdu...@gm...>]

Excellent!
Prefs is an API for loading saving/loading application preferences as well as 
for all sorts of config files. The important thing is that it doesnt build on 
simple string maps but on concrete Entry and Section types. So when you 
change someting in the Prefs and forget to change it in your program you are 
more likely to realise it at compile time.

Preference data is organised in sections. Each section has a list of 
subsections and a list of entries yielding a tree like structure. 

class Section {
	/* ... */
	private:
		std::list<Entriy*> _entries;
		std::list<Section*> _sections
};

Entries and Sections have a name and a parent. The both derive from Node:

class Node {
	/*  ... */												
	private:
		Node* _parentNode;
		std::string _name;
};  

Entry and Section are base classes and you CAN add derived ones or use one of 
the simple predefined Entry types (String, int, float etc...) made available 
by the ValueEntry template class for all streamables.

class Entry : public Node {
	public:
		Entry(const std::string& name);
		virtual std::string str() const = 0;	
};

template <class ValueT>
class ValueEntry : public Entry {
	public:
		ValueEntry(const std::string& name, const ValueT& value);
		virtual std::string str() const; // stream conversion of _value;
		const ValueT& value() const;
		void setValue(const ValueT& value);

	private:
		ValueT _value; 
};

// examples
typedef ValueEntry<std::string> StringEntry;
typedef ValueEntry<int> IntEntry; 

The Prefs class holds the root section and a PrefsStore class which knows how 
to load and save the data. This would be where I have to look at s11n.

class PrefsStore { 
	public:
		PrefsStore() {}		
		virtual ~PrefsStore() {}    		
		virtual void load(Prefs::Section& root) throw(P::IOError) = 0;
		virtual void update(Prefs::Section& root) throw(P::IOError) = 0;
};

class Prefs {
	/* ... */
	private:
		Section _root;
		PrefsStore* _store;
};

We want to be able to use Prefs with predefined strores like INI, xml or even 
some network location.
@Stephan what would you suggest? wrapping up s11n in IniStore, XmlStore etc? 
Or kick out PrefsStore and use s11n? It would also be interesting to have the 
parsers as seperate classes in P2.


One other idea I had was to have Iterators that stride (hope that is the 
correct term) over one type of entry. So if you want to have only the 
nameserver entries from a resolv.conf file you would do something like this:

- Impelment ResolvConf derived from Prefs with its store class for the 
resolv.conf config file format with a NameserverEntry class derived from i.e. 
StringEntry
- Use "stride" iterator for NameserverEntries to print all nameserver entries.

Now for a simple file like resolv.conf it doesnt look like much, but imagine 
to simply iterate through all InputDevice sections in a XF86Config.

The "stride" or "select" iterator was rather easy to implement as a template 
class encapsulating a normal "linear" iterator. operator++ has to do some 
type checking though. But maybe that should not be part of Prefs but part of 
the the containers, with SelectTraits or something.


happy cristmas,
Marc


On Friday 24 Dec 2004 00:43, stephan beal wrote:
> On Thursday 23 December 2004 22:52, Marc Duerner wrote:
> > Hello,
> > I still want to look at P::Prefs and some containers. Specially the
> > first one may benefit from s11n.
>
> Any std container is usably out of the box by s11n:
>
> #include <s11n.net/s11n/s11nlite.hpp>
> #include <s11n.net/s11n/list.hpp>
> #include <s11n.net/s11n/map.hpp>
> #include <s11n.net/s11n/pods_streamable.hpp>
>
> ...
>
> typedef map<string,list<map<int,string> > > Geez;
>
> Geez g;
> ... populate it ...
> bool worked = s11nlite::save( g, stream|file );
>
> Loading has 2 options:
>
> #1:
> Geez * g = s11nlite::load_serializable<Geez>( stream|file );
>
> #2:
> s11nlite::node_type * n = s11nlite::load_node( stream|file );
> if( ! n ) { ... error ... }
> Geez g;
> bool worked= = s11nlite::deserialize( *n, g );
>
>
> P::s11n works identically, but also supports URLs (whatever is supported
> by IOManager).

[pclasses-devel] Mime ported - please approve or disapprove!

[stephan b. <st...@s1...>]

Just comitted:

P::System::MimeType - slightly different interface (no file extensions). 
Container-friendly, via operator< and operator==.

P::System::MimeTypeDb - a non-singleton db. This type provides the file 
extension-to-MimeType mapping which used to be part of MimeType.

Dependencies: System::PathFinder

It compiles and links, but is untested.

If the changes offend anyone, please speak up!

:)

-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

[pclasses-devel] IOHandler and IORequest deps

[stephan b. <st...@s1...>]

Yo,

Question: why do IOH and IOR have circular deps?

It seems to me that the IOR dep on IOH is unnecessary. The IOH funcs 
take a URL which returns an IOR. Does the IOR really need to know which 
IOHandler is handling it?


-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

[pclasses-devel] supermacro-based factory registration mini-demo

[stephan b. <st...@s1...>]

Yo,

Consider the 3 classes we had earlier:

AType ==> extended by BType ==> x by CType

Here's how we can plug in factories for a set of classes:

#define PFACREG_TYPE AType
#define PFACREG_TYPE_NAME "AType"
#include <pclasses/FactoryReg.h>

#define PFACREG_TYPE BType
#define PFACREG_TYPE_INTERFACE AType
#define PFACREG_TYPE_NAME "BType"
#include <pclasses/FactoryReg.h>

#define PFACREG_TYPE CType
#define PFACREG_TYPE_INTERFACE AType
#define PFACREG_TYPE_NAME "CType"
#include <pclasses/FactoryReg.h>

That causes some back-end template code to register the classes with the 
AType factory. The registration happens during the static init phase of 
the app, which means before main() or during dlopen(). Thus all types, 
regardless of their linkage, have the exact same registration 
technique.

To use the default factories a type must be Default Constructable using 
'new', but clients can easily install their own back-end factories 
(using templates) which the factory API will pick up.

To re-use the FactoryTest example from earlier, it now works using this 
type of registration. The main implications/points are:

a) Same reg technique for all classes, regardless of linkage.
b) We have the same client API for loading types, regardless of linkage 
of loaded class: classload<T>(name). Clients automatically benefits 
from any DLL support which might be hidden behind the P API.
c) Also works with abstract types (add #define 
PFACREG_TYPE_IS_ABSTRACT).

The code for the supermacro is in include/pclasses/FactoryReg.h. Factory 
itself did not need to change to accomodate this.


See ya!

-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

[pclasses-devel] App module?

[stephan b. <st...@s1...>]

Christian, may i add an App module? This would cover at least the 
following areas:
- argv parser
- environment, including env-var/string expansion.
- per-app config data
- basic app interface (once we agree on one).



-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

Re: Fwd: Re: [pclasses-devel] P::Classes experimental code

[stephan b. <st...@s1...>]

On Thursday 23 December 2004 22:52, Marc Duerner wrote:
> Hello,
> I still want to look at P::Prefs and some containers. Specially the
> first one may benefit from s11n.

Any std container is usably out of the box by s11n:

#include <s11n.net/s11n/s11nlite.hpp>
#include <s11n.net/s11n/list.hpp>
#include <s11n.net/s11n/map.hpp>
#include <s11n.net/s11n/pods_streamable.hpp>

...

typedef map<string,list<map<int,string> > > Geez;

Geez g;
... populate it ...
bool worked = s11nlite::save( g, stream|file );

Loading has 2 options:

#1:
Geez * g = s11nlite::load_serializable<Geez>( stream|file );

#2:
s11nlite::node_type * n = s11nlite::load_node( stream|file );
if( ! n ) { ... error ... }
Geez g;
bool worked= = s11nlite::deserialize( *n, g );


P::s11n works identically, but also supports URLs (whatever is supported 
by IOManager).


-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

[pclasses-devel] ??? move include/pclasses to include/pclasses2?

[stephan b. <st...@s1...>]

Yo,

Christian - yesterday you said you'd like the build to install the 
headers to prefix/include/pclasses2.

If you still want to do that, can we rename include/pclasses? That would 
mean sed'ing the rest of the files to fix the path, but i think it is 
worth it.

Optionally, keep include/pclasses and install under 
prefix/include/pclasses. This will of course break 1.x clients...

Wait - no it won't: 1.x and 2.x use 100% different file names! All 
headers are now upper-case and libs are now libpclasses_xxx.so.

So we could keep the headers under include/pclasses and <pclasses/...>, 
instead of pclasses2.

Just let me know which one you prefer so i can change the build 
accordingly.

See ya!

-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

Fwd: Re: [pclasses-devel] P::Classes experimental code

[Marc D. <bdu...@gm...>]

Hello,
I still want to look at P::Prefs and some containers. Specially the first one
may benefit from s11n.
There is also PSI (device stuff like pci, usb, cdrom, serialdev etc) which
may be included if we get around to implement the other platforms than
Linux/posix. I have looked briefly at freebsd.

Marc

On Wednesday 22 Dec 2004 19:45, stephan beal wrote:
> On Wednesday 22 December 2004 19:39, Christian Prochnow wrote:
> > Things im currently on:
> >
> > - Transition of the System abstraction library.
> > - Make Unicode String class ready to use.
> > - SerialDevice class
> > - NamedPipe, NamedPipeServer class
>
> If i may add to that, things i will work on over the next 4 days:
>
> - Implement a couple patches i made on the 1.x CVS tree. A fix for URL
> and mysql driver.
>
> - Add Request::read( string &, size_t ). Doh - Request isn't ported to
> pclasses2 yet.
>
> - Port some utility functions out of ps11n into P, like
> iomanager_read(string url), which reads in input from any
> IOManager-supported type.
>
> - Maybe port the build tree to toc (the tools s11n uses). This depends
> on Christian's decision.
>
> - Add a new shell-app module:
>   - add argv handling.
>   - add command dispatcher
>   - add eshell's argv type ('arguments').
>
> > There is not much in the tree right now. Old features should be
> > transitioned from the pclasses CVS.
>
> Please let me know as you start to work on moving parts over, so i don't
> duplicate any there. As i need classes i will start porting them to the
> new conventions, as needed. i will avoid any of the code you mention
> above.
>
> > Everyone is welcome to hack around.
>
> Thanks again! :)
>
> Something Christian didn't mention: subscribe to pclasses-cvs to get the
> CVS commits via email.
>
> Happy hacking!

-------------------------------------------------------

Re: [pclasses-devel] SharedLib dtor

[stephan b. <st...@s1...>]

On Thursday 23 December 2004 23:55, Christian Prochnow wrote:
> Am Donnerstag 23 Dezember 2004 22:08 schrieb stephan beal:
> > Yo!
> >
> > i'm adding SharedLib.ltdl.cpp (uses GNU libltdl, portable libdl
> > replacement) and i noticed this code in SharedLib.dl.cpp:
>
> which other DLL APIs does libltdl support ? i thought we catched
> every DLL API with the existing dl, shl, dyld and win32
> implementations !?

It's not a question of whether libltdl supports another lib loader. 
Suppose i open Foo.so. Foo.so can run arbitrary code and IT can use, 
e.g., dlopen(). If SharedLib dlcloses() in the dtor then we close 
Foo.so, but the libs dlopen()ed by Foo.so might not be shut down. Those 
dlls might leave artefacts in memory which we still rely on, like 
object factories.

> While this is a problem with the CL architecture .. in general it is
> not. In fact, when DLLs are unloaded which register themselves upon
> loading, they should also deregister themselves from the CL.

That assumes that the DLL knows how to do so. AFAIK this is 
loader-dependent. e.g., libdl looks for the symbol _fine() (i think 
that's the name) and calls if it exists. We can't ask loadable classes 
to know what dlopen implementation they're using.

> Anyway for objects created within the loaded DLL you're right, it
> would be unpractial to keep track of ownership and to destroy all
> instances created form within the DLL we're unloading. Do you think
> it is possible to "transfer" ownership of objects allocated in DLLs
> to the app ?

"Jein".

This is difficult to answer without giving some details about the CL 
model. Here's how a CL lookup works:

a) Do we have a factory for KEY? If so, run that factory.

b) Can we find a DLL for KEY.so? If so, open it. The static-init phase 
of the DLL triggers factory registrations living in that DLL. They 
simply call Factory<InterfaceT>::registerxxxx("myclassname",myfactory).

c) Repeat (a). If we have no factory, return 0. The DLL did not have 
what we wanted, so we could ARGUABLY dlclose() it, but the fact is that 
we cannot know what other DLLs it has opened, and this HAS causes 
crashes for me in real use cases.

What this means is that the DLLs register themselves with the factory, 
without the factory actually having to know. Opening ANY DLL can 
trigger an arbitrary number of factory registrations. Consider doing 
this in your app's init code:

P::System::openSharedLib( "/path/to/my_common_classes.so" );

That might have just registered 27 classes across 5 different factories. 
And it might not have done anything at all!

As far as SharedLib ownership goes: see the impl of openSharedLibrary() 
SharedLib.generic.cpp. i believe that one is relatively sane, and it 
works using the platform-defined SharedLib.

> This is not a problem. In fact SharedLib owns the pointers one can
> retrieve with the operator[]. If this is documented everything is
> fine.

i'm not thinking about symbol-level access. i'm thinking about the 
factories which were registered via static-init. If dlclose() is 
called, those factories point to nirgendwo, or maybe even point to some 
other valid memory after the next dlopen! Eeeek!

> DLLs have a "init" and "shutdown" function. Maybe the Factory
> pointers can be removed by the shutdown function when unloading the
> DLL.

That's lib-loader-dependent. That means the static init code must know 
what loader it's using. The static init code normally looks like the 
following global-scope code, and is 100% symbol/load-independent:

void my_init() {
	... register my factories ...
}

int my_init_placeholder = (my_init(),0);

That is how we achieve 100% symbol-ignorance in the CL registration 
layer.


-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

Re: [pclasses-devel] SharedLib dtor

[Christian P. <cp...@se...>]

Am Donnerstag 23 Dezember 2004 22:08 schrieb stephan beal:
> Yo!
>
> i'm adding SharedLib.ltdl.cpp (uses GNU libltdl, portable libdl
> replacement) and i noticed this code in SharedLib.dl.cpp:

which other DLL APIs does libltdl support ? i thought we catched every DLL API 
with the existing dl, shl, dyld and win32 implementations !?

> SharedLib::~SharedLib() throw()
> {
>  dlclose((void*)_handle);
> }
>
> i strongly recommend removing that dlclose()!
>
> In my experience that's a Bad Idea because it is *impossible* for us to
> track all DLLs/objects brought into the system by any given dlopen().
> Each is free to call arbitrary code which can use other - non-dlopen()
> - techniques for opening additional DLLs. By extension, it is
> impossible for us to accurately know what dlls we later need to close.
> What this means is random crashes whenever a SharedLib goes out of
> scope. The 'man dlopen' docs imply that dlclose() is safe, but in my
> experience it's not.

While this is a problem with the CL architecture .. in general it is not.
In fact, when DLLs are unloaded which register themselves upon loading, they 
should also deregister themselves from the CL.

Anyway for objects created within the loaded DLL you're right, it would be 
unpractial to keep track of ownership and to destroy all instances created 
form within the DLL we're unloading. Do you think it is possible to 
"transfer" ownership of objects allocated in DLLs to the app ?

>
> Let the OS shut down the dlls when the app exits - that is the only 100%
> safe thing to do.
>
> Consider this code, where a client innocently uses SharedLib to open a
> DLL:
>
> void myfunc() {
>  SharedLib lib( "/my.so" );
> ...
> }
>
> after myfunc exits we have a major problem: we are expecting to be able
> to get at types loaded via that dll. We only used SharedLib to open the
> DLL for us, not expecting it to shut down the DLL when the SharedLib
> object dies!

This is not a problem. In fact SharedLib owns the pointers one can retrieve 
with the operator[]. If this is documented everything is fine.

> The current dtor WILL break the classloading layer if the CL uses
> SharedLib to open DLLs. Remember that the CL model doesn't require any
> symbol lookups - it only opens a DLL and then ignores it, expecting the
> types in the DLL to register themselves with the classloader. Their
> factories exist in DLL-space, and shutting down the DLLs leaves our
> Factory with handles pointing to closed DLLs.

DLLs have a "init" and "shutdown" function. Maybe the Factory pointers can be 
removed by the shutdown function when unloading the DLL.

> See ya!

Greetings

[pclasses-devel] SharedLib dtor

[stephan b. <st...@s1...>]

Yo!

i'm adding SharedLib.ltdl.cpp (uses GNU libltdl, portable libdl 
replacement) and i noticed this code in SharedLib.dl.cpp:

SharedLib::~SharedLib() throw()
{
	dlclose((void*)_handle);
}

i strongly recommend removing that dlclose()!

In my experience that's a Bad Idea because it is *impossible* for us to 
track all DLLs/objects brought into the system by any given dlopen(). 
Each is free to call arbitrary code which can use other - non-dlopen() 
- techniques for opening additional DLLs. By extension, it is 
impossible for us to accurately know what dlls we later need to close. 
What this means is random crashes whenever a SharedLib goes out of 
scope. The 'man dlopen' docs imply that dlclose() is safe, but in my 
experience it's not.

Let the OS shut down the dlls when the app exits - that is the only 100% 
safe thing to do.

Consider this code, where a client innocently uses SharedLib to open a 
DLL:

void myfunc() {
 SharedLib lib( "/my.so" );
...
}

after myfunc exits we have a major problem: we are expecting to be able 
to get at types loaded via that dll. We only used SharedLib to open the 
DLL for us, not expecting it to shut down the DLL when the SharedLib 
object dies!

The current dtor WILL break the classloading layer if the CL uses 
SharedLib to open DLLs. Remember that the CL model doesn't require any 
symbol lookups - it only opens a DLL and then ignores it, expecting the 
types in the DLL to register themselves with the classloader. Their 
factories exist in DLL-space, and shutting down the DLLs leaves our 
Factory with handles pointing to closed DLLs.

See ya!

-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

[pclasses-devel] factory mini-demo

[stephan b. <st...@s1...>]

Yo,

Consider the following 3 classes:

AType ==> extended by ==> BType ==> x by ==> CType

Here's the current P2 code to get at them via Factory:

First, register them:

	P::CL::registerBase<AType>( "AType" );
	P::CL::registerSubtype<AType,BType>( "BType" );
	P::CL::registerSubtype<AType,CType>( "CType" );

Here we use the default factories, but we can also swap those out with 
our own via at least 3 different techniques. In standard usage the 
default factories serve very well. There will also be a technique to 
allow DLLs to *register themselves* with their factory(ies), meaning P 
won't have to do any more symbol lookups in DLLs to load classes.

Part 2, load them:

#define LOAD(CN) a = P::CL::classload<AType>( CN ); \
	CERR << CN << " loaded? == " << a << "\n"; \
	if( a ) CERR << "classname="<<a->classname()<<"\n"; \
	delete( a ); a = 0;

	LOAD("AType");
	LOAD("BType");
	LOAD("CType");
	LOAD("NoType");

Output:
stephan@owl:~/cvs/pclasses2/test> ./FactoryTest
FactoryTest.cpp:59 : Factory tests...
FactoryTest.cpp:20 : AType()
FactoryTest.cpp:75 : AType loaded? == 0x804e3d0
FactoryTest.cpp:75 : classname=AType
FactoryTest.cpp:24 : ~AType()
FactoryTest.cpp:20 : AType()
FactoryTest.cpp:35 : BType()
FactoryTest.cpp:76 : BType loaded? == 0x804e3d0
FactoryTest.cpp:76 : classname=BType
FactoryTest.cpp:39 : ~BType()
FactoryTest.cpp:24 : ~AType()
FactoryTest.cpp:20 : AType()
FactoryTest.cpp:35 : BType()
FactoryTest.cpp:48 : CType()
FactoryTest.cpp:77 : CType loaded? == 0x804e3d0
FactoryTest.cpp:77 : classname=CType
FactoryTest.cpp:52 : ~CType()
FactoryTest.cpp:39 : ~BType()
FactoryTest.cpp:24 : ~AType()
FactoryTest.cpp:78 : NoType loaded? == 0



The reasons their addresses repeat is because we delete the object 
between each call - that is not a bug.


-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

Re: [pclasses-devel] P::Classes experimental code

[Marc D. <bdu...@gm...>]

Hello,
I still want to look at P::Prefs and some containers. Specially the first one 
may benefit from s11n.
There is also PSI (device stuff like pci, usb, cdrom, serialdev etc) which may 
be included if we get around to implement the other platforms than 
Linux/posix. I have looked briefly at freebsd.

Marc


On Wednesday 22 Dec 2004 19:45, stephan beal wrote:
> On Wednesday 22 December 2004 19:39, Christian Prochnow wrote:
> > Things im currently on:
> >
> > - Transition of the System abstraction library.
> > - Make Unicode String class ready to use.
> > - SerialDevice class
> > - NamedPipe, NamedPipeServer class
>
> If i may add to that, things i will work on over the next 4 days:
>
> - Implement a couple patches i made on the 1.x CVS tree. A fix for URL
> and mysql driver.
>
> - Add Request::read( string &, size_t ). Doh - Request isn't ported to
> pclasses2 yet.
>
> - Port some utility functions out of ps11n into P, like
> iomanager_read(string url), which reads in input from any
> IOManager-supported type.
>
> - Maybe port the build tree to toc (the tools s11n uses). This depends
> on Christian's decision.
>
> - Add a new shell-app module:
>   - add argv handling.
>   - add command dispatcher
>   - add eshell's argv type ('arguments').
>
> > There is not much in the tree right now. Old features should be
> > transitioned from the pclasses CVS.
>
> Please let me know as you start to work on moving parts over, so i don't
> duplicate any there. As i need classes i will start porting them to the
> new conventions, as needed. i will avoid any of the code you mention
> above.
>
> > Everyone is welcome to hack around.
>
> Thanks again! :)
>
> Something Christian didn't mention: subscribe to pclasses-cvs to get the
> CVS commits via email.
>
> Happy hacking!

[pclasses-devel] Initial Factory impl in CVS

[stephan b. <st...@s1...>]

include/pclasses/Factory.h

i haven't tested it, nor fully instantiated all of the template code yet 
(my build tree only halfway works), but it is syntactically okay and 
almost fully documented.

The P-side interface is essentially 3 functions:

  template <class Base>
  void P::CL::registerBase( string [, factory_func] );

  template <class Base, class SubT>
  void P::CL::registerSubtype( string [, factory_func] );

  template <class Base>
  Base * P::CL::classload( string );

Those funcs use a Factory<>::instance(). The factory uses 
P::Hook::FactoryInstanceHook<FactoryT> to provide a shared object. This 
allows clients to specialize that type to customize the object returned 
by Factory::instance(). e.g., they can add DLL support to the core API 
without the core knowing it. Clients can also specialize 
P::Hook::FactoryCreateHook<Base,Sub> to completely override the default 
new-object implementations. This is necessary when, e.g, Base is 
abstract (and therefor 'new Base' won't work). See the docs for full 
details.

i don't like the CL namespace, and may rename registerXXX() to 
P::cl_registerXXX(). Opinions?

i did a complete re-think of the s11n CL layer in this port, so there 
may be several bugs laying around.

-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

[pclasses-devel] rethinking the classloader/core deps

[stephan b. <st...@s1...>]

Responding to an IRC discussion Christian and i had, where we decided to 
put DLL loading into the System module and provide a basic Factory 
interface via the core API...

This just occurred to me:

In my CL's [CL==classloader] model there is no difference, API-wise, to 
loading a type from a DLL and loading a statically-linked type:

MyT * t = classload<MyT>( "MySubT" );

That MIGHT load MySubT.so, it might not. The client doesn't care if 
MySubT was generated over http and JIT-compiled on a cluster in 
Botswana. All the client wants is his object.

That flexibility is where the problem is: that means the CL interface 
MUST depend on the System module if it is to do any dynamic lookups, 
since System provides all file- and DLL-related stuff (rightfully so). 
That means i can't directly offer dynamic lookups through the same API 
transparency as the CL currently does.

After thinking about it for a while, and looking over the core API more, 
i'm gonna install the CL entirely in System. i will provide a basic 
Factory in the core (done, but not yet commited), but that means the 
default classloader won't do DLL loads by default. i will provide a 
functor-driven way to do that via the core API, though, by passing 
functors which know how to do dynamic lookups. i have a template-driven 
idea which should let clients replace the default (static) factory's 
internals to redirect it to the System one without the core knowing 
(similar to s11n's custom factory installation approach).

The PathFinder class i mentioned a few days ago now lives in System, but 
i haven't compiled it after sed'ing the functions to follow P 
conventions. Still working on the build port - the core lib compiles 
now, but i've gotta sleep before porting the others...

See ya!

-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

Re: [pclasses-devel] dual-system build tree notes

[stephan b. <st...@s1...>]

On Thursday 23 December 2004 01:05, stephan beal wrote:
> The Makefile.toc implementations are your standard Makefiles.
> Autotools will overwrite toc's Makefiles, which is exactly what we
> want. Both trees can co-exist while i piddle around with toc and
> figure out if it's really what we want to use. :)

Doh! The top-level autotools files were already removed from CVS by the 
time i did this. i'll leave it as-is now, and move Makefile.toc ==> 
Makefile later.


-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

[pclasses-devel] dual-system build tree notes

[stephan b. <st...@s1...>]

Yo, Christian!

Okay, i've got toc installed and configuring, but won't have all of the 
required tests done before late Thursday or maybe Friday.

Here are things which you need to know:

a) ./configure is the toc-powered configure script. When using autotools 
you will overwrite configure. This is exactly what we want, as it means 
"by default use toc, otherwise fall back to autotools".
Be sure not to commit your configure script if you auto-gen it!

b) All configure tests go into configure.pclasses2. That is called by 
configure. configure.toc is a middleman stub - ignore it.

c) toc now creates the Makefiles, with Makefile.toc holding the "real" 
make code. toc just generates stubs which look like:
	#!/usr/bin/make -f
	# Automatically generated by configure.
	# Do not edit this file: put your code in Makefile.toc.
	include Makefile.toc
	DIST_FILES += Makefile.toc
	DIST_FILES := $(filter-out Makefile,$(DIST_FILES))
	# ^^^^ toc automatically adds Makefile to DIST_FILES.
	DISTCLEAN_FILES += Makefile

The Makefile.toc implementations are your standard Makefiles. Autotools 
will overwrite toc's Makefiles, which is exactly what we want. Both 
trees can co-exist while i piddle around with toc and figure out if 
it's really what we want to use. :)


So, what this all essentially means is:

You may continue working on the tree as if toc didn't exist, and i'll 
let you know once it's fully buildable. The transition from AT to toc 
will then be transparent for the builder.

See ya!

-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

Re: [pclasses-devel] P::Classes experimental code

[stephan b. <st...@s1...>]

On Wednesday 22 December 2004 20:05, Christian Prochnow wrote:
> Place URL in Net/.  

Will do.

> SQL stuff isn't ported yet and needs major 
> rework. However .. i think the String class must be done before
> starting with the URL class. 

Will the string support implicit conversion from (char *) or 
std::string? If not, i would prefer to see URL based off of 
std::string. If there's an implicit conversion, i see no problem with 
basing URL off of it.

> What about support for 
> internationalization in the URL class (don't remember the correct
> name)?

i know very little about i18n, so i can't say much about it.


> > - Add Request::read( string &, size_t ). Doh - Request isn't ported
> > to pclasses2 yet.
>
> We first need a Plugin library. Since Plugin loading is not a core
> component, and isn't a System abstraction class either it should go
> into it's own P library. Stephan .. how about your excellent
> class_loder stuff ? Shall i integrate it into P and drop the old
> Plugin stuff ? your code is much more elegant and cleaner.

*blush*. Actually, earlier today i was thinking about how horrible one 
of the classloader's "features" is, and want to rewrite it. i like the 
cllite interface a lot, though, and i get TONS of usage out of it. That 
said, i will be more than happy to add a classloading layer to P. i'd 
be honored, in fact, to be allowed to port my code into the tree :).

> > - Port some utility functions out of ps11n into P, like
> > iomanager_read(string url), which reads in input from any
> > IOManager-supported type.
>
> Should go into P::IO library.

Agreed. i will generally follow the policy of "put it wherever it'd deps 
say it should." If new dependencies would be introduced i will first 
ask for your approval.

> > - Maybe port the build tree to toc (the tools s11n uses). This
> > depends on Christian's decision.
> >
> > - Add a new shell-app module:
> >   - add argv handling.
> >   - add command dispatcher
> >   - add eshell's argv type ('arguments').
>
> Hmm .. should this go into Util/ or shall we add a "application
> support" library ?

i think argv belongs in the Util section, and the rest as a standalone 
module or plugin, but i'm not sure what the plugin abstraction might 
look like yet.

> For example SimpleApp should go into "application support" lib, i
> think argv handling as well.

That'd work, too. That makes more sense, actually.

> If argv handling is "generic" and doesn't rely on input from
> "main(int, char*[])" it should go into Util/.

The argv PARSER does not strictly rely on main(), but is not much use 
without it. The argv TYPE is 100% independent. The argv TYPE is simpy a 
sequence container of args with some features to ease it's use as an 
argc/argv replacements. The features like $var/alias expansion depend 
on the app layer (Environment and Aliases objects), so the argv list 
probably belongs there as well.

That's a long way of saying: the ArgvParser class belongs in util, the 
Arguments (Argv?) class belongs in the app utils.

> Seems that this Christmas becomes a funny hacking session ...

That's how i usually spend them. :)


-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts

Re: [pclasses-devel] P::Classes experimental code

[Christian P. <cp...@se...>]

Am Mittwoch 22 Dezember 2004 19:45 schrieb stephan beal:
> On Wednesday 22 December 2004 19:39, Christian Prochnow wrote:
> > Things im currently on:
> >
> > - Transition of the System abstraction library.
> > - Make Unicode String class ready to use.
> > - SerialDevice class
> > - NamedPipe, NamedPipeServer class
>
> If i may add to that, things i will work on over the next 4 days:
>
> - Implement a couple patches i made on the 1.x CVS tree. A fix for URL
> and mysql driver.

Place URL in Net/.  SQL stuff isn't ported yet and needs major rework.
However .. i think the String class must be done before starting with the URL 
class. What about support for internationalization in the URL class (don't 
remember the correct name)?

> - Add Request::read( string &, size_t ). Doh - Request isn't ported to
> pclasses2 yet.

We first need a Plugin library. Since Plugin loading is not a core component, 
and isn't a System abstraction class either it should go into it's own P 
library. Stephan .. how about your excellent class_loder stuff ? 
Shall i integrate it into P and drop the old Plugin stuff ? your code is much 
more elegant and cleaner.

> - Port some utility functions out of ps11n into P, like
> iomanager_read(string url), which reads in input from any
> IOManager-supported type.

Should go into P::IO library.

> - Maybe port the build tree to toc (the tools s11n uses). This depends
> on Christian's decision.
>
> - Add a new shell-app module:
>   - add argv handling.
>   - add command dispatcher
>   - add eshell's argv type ('arguments').

Hmm .. should this go into Util/ or shall we add a "application support" 
library ?
For example SimpleApp should go into "application support" lib, i think argv 
handling as well. 
If argv handling is "generic" and doesn't rely on input from "main(int, 
char*[])" it should go into Util/.

> > There is not much in the tree right now. Old features should be
> > transitioned from the pclasses CVS.
>
> Please let me know as you start to work on moving parts over, so i don't
> duplicate any there. As i need classes i will start porting them to the
> new conventions, as needed. i will avoid any of the code you mention
> above.
>
> > Everyone is welcome to hack around.
>
> Thanks again! :)
>
> Something Christian didn't mention: subscribe to pclasses-cvs to get the
> CVS commits via email.
>
> Happy hacking!

Seems that this Christmas becomes a funny hacking session ...

Re: [pclasses-devel] P::Classes experimental code

[stephan b. <st...@s1...>]

On Wednesday 22 December 2004 19:39, Christian Prochnow wrote:
> Things im currently on:
>
> - Transition of the System abstraction library.
> - Make Unicode String class ready to use.
> - SerialDevice class
> - NamedPipe, NamedPipeServer class

If i may add to that, things i will work on over the next 4 days:

- Implement a couple patches i made on the 1.x CVS tree. A fix for URL 
and mysql driver.

- Add Request::read( string &, size_t ). Doh - Request isn't ported to 
pclasses2 yet.

- Port some utility functions out of ps11n into P, like 
iomanager_read(string url), which reads in input from any 
IOManager-supported type.

- Maybe port the build tree to toc (the tools s11n uses). This depends 
on Christian's decision.

- Add a new shell-app module:
  - add argv handling.
  - add command dispatcher
  - add eshell's argv type ('arguments').


> There is not much in the tree right now. Old features should be
> transitioned from the pclasses CVS.

Please let me know as you start to work on moving parts over, so i don't 
duplicate any there. As i need classes i will start porting them to the 
new conventions, as needed. i will avoid any of the code you mention 
above.

> Everyone is welcome to hack around.

Thanks again! :)

Something Christian didn't mention: subscribe to pclasses-cvs to get the 
CVS commits via email.

Happy hacking!

-- 
----- st...@s1...   http://s11n.net
"...pleasure is a grace and is not obedient to the commands
of the will." -- Alan W. Watts