zoolib-dev — Development list for the ZooLib cross-platform application framework

[ZooLib-dev] More thoughts on stream positions and sizes

[Andrew G. <ag...@em...>]

I've had a few more days to think about the SetPosition issue that I 
recognized in revamping the ZStreamXPos API to use 64 bit offsets. And 
I've realized that (again) UNIX had it right all along. To recap, code 
that takes a ZStreamRPos and tries to position it beyond its physical 
end will see different behavior when it's actually passed a 
ZStreamRWPos than if it's passed a ZStreamRPos. To whit, the former 
will quietly resize the stream if physically possible, whereas the 
latter will now throw an exception (previously its behavior was 
undefined).

The UNIX (and Win32) approach is to allow SetPosition to take any 
value, and to report that value when GetPosition is called. If 
ZStreamRPos::Read is called when the position is at or beyond the end 
of stream then it just reads zero bytes and leaves the position 
unchanged, and higher level code can make of that what it will. If 
ZStreamRWPos::Write is called then the underlying stream is extended 
appropriately, with any gap between the end of stream and the current 
position being filled with undefined data.

I've gone through all of ZooLib, and other code, and updated it to 
reflect these semantics. It's an almost completely safe change in that 
it just _removes_ a constraint on what can be passed to SetPosition. 
The only difference in behavior code would see is if it called 
SetPosition and then GetSize and required that the size be at least as 
big as the position. That's not a pattern that shows up anywhere in 
ZooLib, but let me know if this is something you need to do or indeed 
can think of a scenario in which it makes sense.

Code that calls SetPosition without first calling SetSize will 
generally be doing so in order to write something, in which case the 
stream will be extended as necessary when the write occurs. If it's 
calling SetPosition to read something, and that position is at or 
beyond the end of stream, then in either case, under the old scheme or 
the new scheme, it's going to see zero bytes -- the old scheme only 
extended the size to *match* the position, and there were still zero 
bytes beyond the position.

The only ZooLib code I haven't updated to reflect these semantics is in 
ZBlockStore_SlotStore. The ZBlockStore_SlotStore::Stream updates the 
secondary slot referenced by a position as it reads, writes and when 
SetPosition is called. Maintaining a logical and physical position 
would be straightforward, but I don't want to futz with that code for 
now -- it's working fine for current purposes, and new code should 
probably use ZBlockStore_PhaseTree.

A+
-- 
Andrew Green                           mailto:ag...@em...
Electric Magic Co.               vox: +1 (831) 621 1822

[ZooLib-dev] Mike's doing GUI work again

[Michael D. C. <cra...@go...>]

Hi Friends,

I'm happy to say I'm writing C++ graphical user interface code again, on 
the Mac even.

It's in PowerPlant, but I've been making the case to my client that he 
should pay me to port it to ZooLib so he can offer a Windows version. 
We'll have to see.

I haven't had much occasion to work with ZooLib for a while because I've 
been doing embedded systems development since the fall of 2002.  I've 
done well at it, and I have no problem finding new clients, but it is 
also really painful: I've written a number of custom firmware images for 
the Oxford 911 FireWire/IDE bridge chip, but have had to be very 
scrupulous because the 911 has only 1700 bytes of RAM.  Not kilobytes, 
BYTES.

It seemed like things would be easier when I moved to the 922 because it 
has 8 kb of RAM, but it hasn't worked out too well because it's a MUCH 
more complex chip.

I spent the last couple days running Spotlight 
(http://www.onyxtech.com/) under OS 9 to debug my client's product, and 
I found myself thinking "This is a lot more fun than bitbanging yet 
another electronic widget".

I tested ZooLib with Spotlight too, but that was back in 2000, and a lot 
of work has been done on it since.  I expect I can do it again soon, I 
have lots of source that exercises ZooLib.

          Spotlight: the Best Argument for Supporting OS 9.
          There is No Equivalent in OS X, Not for Any Price.

Well, I'll get back to work now.

And no, I haven't forgotten that I've been promising to complete the 
ZooLib Cookbook for Several Years:

http://www.goingware.com/zoolib/cookbook/

Here's a book I recommend to anyone who will listen, it has made my work 
life make a lot more sense to me, and I'm able to spend more time on my 
music as a result: "How to Get Control of Your Time and Your Life" by 
Alan Lakein.

It's just a little book, I think six bucks.  It's not the sort of time 
management book your boss wants you to read though.  Lakein says that if 
your job isn't helping you achieve your life's goals, maybe what you 
need to do is get a better job.

Indeed.

More Soon.

Mike Crawford
cra...@go...

     Tilting at Windmills for a Better Tomorrow.

[ZooLib-dev] Support for 64 bit offsets in ZStream

[Andrew G. <ag...@em...>]

I'm about to check in support for 64 bit offsets in ZStream. Most 
ZStream derivatives hang off ZStreamR or ZStreamW and so are mostly 
unaffected by this change, except where they implement copy from/to.

The affected entry points are:
ZStreamR:
void CopyAllTo(const ZStreamW& iStreamW, uint64* oCountRead, uint64* 
oCountWritten) const;
void CopyTo(const ZStreamW& iStreamW, uint64 iCount) const;
void CopyTo(const ZStreamW& iStreamW, uint64 iCount, uint64* 
oCountRead, uint64* oCountWritten) const;

void Skip(uint64 iCount) const;
void Skip(uint64 iCount, uint64* oCountSkipped) const;
void SkipAll(uint64* oCountSkipped) const;

ZStreamW:
void CopyAllFrom(const ZStreamR& iStreamR, uint64* oCountRead, uint64* 
oCountWritten) const;
void CopyFrom(const ZStreamR& iStreamR, uint64 iCount) const;
void CopyFrom(const ZStreamR& iStreamR, uint64 iCount, uint64* 
oCountRead, uint64* oCountWritten) const;

ZStreamXXPos:
uint64 GetPosition() const;
void SetPosition(uint64 iPosition) const;
uint64 GetSize() const;
void SetSize(uint64 iSize) const;

And of course the actual Imp_XXX methods. Read and write methods still 
take and return size_t parameters, as you can't read or write more data 
in one hit than your address space can accomodate.

I've also tightened up the behavior of SetSize and SetPosition. A 
SetSize operation that detectably fails will throw a 
ZStreamWPos::ExBadSize exception. A SetPosition that detectably fails 
will throw a ZStreamRPos::ExBadPosition. Both inherit from 
std::range_error. This did highlight a weakness in how SetPosition is 
defined to behave. If SetPosition is called on a ZStreamRPos outside of 
the range of bytes available then the behavior was supposed to be 
undefined -- that is, it was not defined what the actual position would 
be after the call. It now throws an exception. If SetPosition is called 
on a ZStreamWPos or ZStreamRWPos outside of the range of bytes 
available then it's supposed to extend the range of bytes available, 
with the new bytes having undefined values, and if the size extension 
fails then a ZStreamWPos::ExBadSize is thrown. So code that's passed a 
ZStreamRWPos expecting only a ZStreamRPos has would always have behaved 
differently if it ever strayed outside the defined range depending on 
what actual stream was passed to it. The fix would be to have a 
separate entry point for ZStreamWPos/ZStreamRWPos that sets the 
position and extends the size if necessary. I'm not sure if that's 
warranted right now.

A+
-- 
Andrew Green                           mailto:ag...@em...
Electric Magic Co.               vox: +1 (831) 621 1822

Re: [ZooLib-dev] Problems using ZooLib with VS 2005

[<pa...@id...>]

> This is legal C++. If the compiler is unable to actually inline the 
> function it will compile it and put it in the .o of the including 
> source file, tagged to indicate that multiple definitions are to be 
> expected. In googling for this issue, and what you were finding with 
> asm syntax, it seems that you might be using managed C++. In which case 
> the code is compiling down to CLI, the asm syntax is different and 
> we've got some work to do:)

Nope, not unless the compiler is forcing it on me, I don't see any mention 
of it in the project command line.  The project was imported from an 
existing VC6 project.
 
> So, the first thing is to determine if you're actually building managed 
> C++, in which case we'll need to use quite different techniques, and 
> ideally should just leverage the much better .net API. If you're not 
> then I'll need to look at the source for the MS runtime. Would you be 
> able to zip it up and email it to me?

I`m looking through the source now, and I`m finding references to _getptd, 
I don't know why they didn't show up when I searched everything in 
/Program Files/, the VC6 ones did, weird.  OK, looks like it just needs to 
have extern "C" added, now all I seem to have left are the multiply 
defined symbols.

Thanks,

Paul Nolan, CEO Idruna Software Inc.
http://www.idruna.com

Re: [ZooLib-dev] Problems using ZooLib with VS 2005

[Andrew G. <ag...@em...>]

On Feb 19, 2005, at 11:22, Paul Nolan wrote:
> I`m back, trying to use the cvs version of zoolib, but have to fix the
> inline functions in the headers causing multply defined symbols again.
> What is the best way for me to fix them, so that the changes can be 
> merged
> back?  It's mainly things like this line in ZAtomic.h.  I`m still 
> learning
> C++, so I`m very tempted to make the functions macros..
>
> inline int ZAtomic_Get(const ZAtomic_t* iAtomic)
> 	{ return iAtomic->fValue; }

This is legal C++. If the compiler is unable to actually inline the 
function it will compile it and put it in the .o of the including 
source file, tagged to indicate that multiple definitions are to be 
expected. In googling for this issue, and what you were finding with 
asm syntax, it seems that you might be using managed C++. In which case 
the code is compiling down to CLI, the asm syntax is different and 
we've got some work to do:)

On Feb 19, 2005, at 12:18, Paul Nolan wrote:
> One more problem I`m afraid, the CVS version of ZThread.cpp has been
> changed to call _getptd(), which from what I can tell was an internal 
> CRT
> function from VC6 that no longer exists, any ideas?
>
> static HANDLE sCurrentThreadHANDLE()
> 	{
> 	if (_tiddata* theData = _getptd())
> 		return theData->_thandle;
> 	return 0;
> 	}

ZSemaphore has stronger semantics than many native semaphore 
implementations. In particular it is safe to dispose a semaphore whilst 
threads are blocked on it. Those threads are woken up and 
ZThread::errorDisposed is returned to them. This was an essential 
feature for some of the UI stuff, although I've generally used other 
mechanisms elsewhere in the code and have given serious thought to 
implementing a parallel set of much lighter synchronization primitives 
that map more closely to what we can expect from the OS.

To support this capability on Win32, we block a thread by recording its 
real thread handle in a waiter structure and calling SuspendThread. 
When it needs to be unblocked we call ResumeThread. GetCurrentThread 
returns a pseudo thread handle, one that means "the current thread" 
rather than a value that has meaning from another thread (the one 
that's calling ResumeThread). In the past we'd look to see if we're 
executing in a ZThread (using TLS) and if so would pull a value from 
fThreadHANDLE. If we created the thread any other way we'd use 
DuplicateHandle to get a real globally-meaningful handle, and then 
dispose it when we were woken up. This is pretty wasteful, as the 
runtime libraries record the thread handle in the structure that lets 
the C library work correctly (set up by beginthread or beginthreadex).

So, the first thing is to determine if you're actually building managed 
C++, in which case we'll need to use quite different techniques, and 
ideally should just leverage the much better .net API. If you're not 
then I'll need to look at the source for the MS runtime. Would you be 
able to zip it up and email it to me?

A+
-- 
Andrew Green                           mailto:ag...@em...
Electric Magic Co.               vox: +1 (831) 621 1822

Re: [ZooLib-dev] Problems using ZooLib with VS 2005

[<pa...@id...>]

Andrew,

One more problem I`m afraid, the CVS version of ZThread.cpp has been 
changed to call _getptd(), which from what I can tell was an internal CRT 
function from VC6 that no longer exists, any ideas?  

static HANDLE sCurrentThreadHANDLE()
	{
	if (_tiddata* theData = _getptd())
		return theData->_thandle;
	return 0;
	}
	
Thanks,

Paul Nolan, CEO Idruna Software Inc.
http://www.idruna.com

Re: [ZooLib-dev] Problems using ZooLib with VS 2005

[Joshua J. <wan...@me...>]

On Feb 15, 2005, at 4:58 PM, Paul Nolan wrote:

>> wrt smart pointers in general -- the boost library has come on a long
>> way in the last few years, and their smart_ptr will likely be in the
>> next version of the standard library. It's not thread-safe, but might
>> suffice for your purposes. Generally, unless you're assigning *to* a
>> shared smart pointer you only need to update the count manipulation
>> with atomic ops. If you are assigning to a shared smart pointer it's
>> often better to simply protect it with some synchronization entity --
>> it's possible to update a couple of pointers and an associated count
>> safely on PPC using address reservation instructions, and apparently
>> it's also possible on x86, but I suspect it'll need some post Pentium 
>> I
>> instruction to do so.
>
> Ah, I think I see what they were talking about on the boost list now 
> when
> they said smart_ptrs are not thread safe, would I be right in thinking 
> a
> ZRef would also need synchronization when assigning?  I was hoping to 
> find
> a reallysmart_ptr I could just use without having to worry about, do 
> you
> have any recommendations?  I`d happily trade some speed in the non 
> speed
> critical parts of my software if it reduced the amount of time I spent
> debugging multithreaded issues.

You could (copy and) modify a smart pointer template to perform the 
unsafe operations inside a critical region.  This might be an example 
of policy that's a candidate for being a template parameter.  Now that 
I think of it, Alexandrescu may have discussed this already in his 
book.

Personally, I've been using cooperative threads, so the dynamic scope 
between every consecutive pair of yield calls is automatically a 
critical region.  :-)

Josh

-- 
Joshua Juran
Metamage Software Creations - Mac Software and Consulting
http://www.metamage.com/

                * Creation at the highest state of the art *

Re: [ZooLib-dev] Problems using ZooLib with VS 2005

[<pa...@id...>]

> Hmmm. I just started writing directions on how to browse the cvs 
> repository, how to check out etc, and in checking my instructions I 
> find that nothing's working. It looks like sourceforge is having some 
> difficulties. In the meantime I'm sending you current copies of 
> ZAtomic.cpp/h in a separate message.

Ah, that would explain things, thanks!

> I'm afraid I don't know of differences between VC++ 6-ish and VS 2005 
> that would account for the problems in compiling the assembly code -- 
> it may be that it's stricter in its interpretation of mnemonics, and 
> perhaps the cmpxchg instructions have their operands the wrong way 
> round. I haven't used VC++ et al for years, and nor are any of my 
> current colleagues so ZooLib support for MS dev tools relies on helpful 
> souls pitching in from time to time.

Gotcha.
 
> wrt smart pointers in general -- the boost library has come on a long 
> way in the last few years, and their smart_ptr will likely be in the 
> next version of the standard library. It's not thread-safe, but might 
> suffice for your purposes. Generally, unless you're assigning *to* a 
> shared smart pointer you only need to update the count manipulation 
> with atomic ops. If you are assigning to a shared smart pointer it's 
> often better to simply protect it with some synchronization entity -- 
> it's possible to update a couple of pointers and an associated count 
> safely on PPC using address reservation instructions, and apparently 
> it's also possible on x86, but I suspect it'll need some post Pentium I 
> instruction to do so.

Ah, I think I see what they were talking about on the boost list now when 
they said smart_ptrs are not thread safe, would I be right in thinking a 
ZRef would also need synchronization when assigning?  I was hoping to find 
a reallysmart_ptr I could just use without having to worry about, do you 
have any recommendations?  I`d happily trade some speed in the non speed 
critical parts of my software if it reduced the amount of time I spent 
debugging multithreaded issues.

Thanks,

Paul Nolan, CEO Idruna Software Inc.
http://www.idruna.com

Re: [ZooLib-dev] Problems using ZooLib with VS 2005

[Andrew G. <ag...@em...>]

On Feb 15, 2005, at 13:10, Paul Nolan wrote:
> I`m getting an error trying to browse the CVS online, is there a more
> recent version of ZAtomic, or any other tips for using ZooLib and VS 
> 2005?

Hmmm. I just started writing directions on how to browse the cvs 
repository, how to check out etc, and in checking my instructions I 
find that nothing's working. It looks like sourceforge is having some 
difficulties. In the meantime I'm sending you current copies of 
ZAtomic.cpp/h in a separate message.

I'm afraid I don't know of differences between VC++ 6-ish and VS 2005 
that would account for the problems in compiling the assembly code -- 
it may be that it's stricter in its interpretation of mnemonics, and 
perhaps the cmpxchg instructions have their operands the wrong way 
round. I haven't used VC++ et al for years, and nor are any of my 
current colleagues so ZooLib support for MS dev tools relies on helpful 
souls pitching in from time to time.

wrt smart pointers in general -- the boost library has come on a long 
way in the last few years, and their smart_ptr will likely be in the 
next version of the standard library. It's not thread-safe, but might 
suffice for your purposes. Generally, unless you're assigning *to* a 
shared smart pointer you only need to update the count manipulation 
with atomic ops. If you are assigning to a shared smart pointer it's 
often better to simply protect it with some synchronization entity -- 
it's possible to update a couple of pointers and an associated count 
safely on PPC using address reservation instructions, and apparently 
it's also possible on x86, but I suspect it'll need some post Pentium I 
instruction to do so.


> I also noticed an old post on using ZooLib for PhotoShop plugins, was 
> any
> progress ever made?

Not that I'm aware of, at least for PhotoShop plugins. I did do about 
90% of what's needed for hosting ZooLib stuff in a browser plugin. It 
took a week or so and is checked in at src_other/browser. That code 
uses the original NetScape API, and might be instructive if someone 
were to want to do something similar for PhotoShop plugins. In the 
ancient past I had such wrappers for HyperCard, Director and MacOS 
Control Panels -- such varied hosting environments were one of the 
initial motivations for developing ZooLib at all.


A+
-- 
Andrew Green                           mailto:ag...@em...
Electric Magic Co.               vox: +1 (831) 621 1822

[ZooLib-dev] Problems using ZooLib with VS 2005

[<pa...@id...>]

Hi,

I came across ZooLib while looking for a thread safe smart pointer, but  
I`m having a bit of trouble compiling under VS 2005.
The compiler didn't like inline functions outside of class definitions in 
.hpp files, or the inline assembly, so I moved things around a bit to see 
if I could just get it working enough to see how good these ZRefs are.  
The snag now is that trying to use a ZRef with the dumb versions of the 
code causes infinite recursion:

.exe!ZAtomic_Add(ZAtomic_t * inValueAddress=0x00a879c4, int inParam=1)  
Line 405	C++
.exe!ZMutexNR::Acquire()  Line 986 + 0xe bytes	C++
.exe!ZAtomic_Add(ZAtomic_t * inValueAddress=0x00a879c4, int inParam=1)  
Line 405	C++
.exe!ZMutexNR::Acquire()  Line 986 + 0xe bytes	C++
.exe!ZAtomic_Add(ZAtomic_t * inValueAddress=0x00a879c4, int inParam=1)  
Line 405	C++
.exe!ZMutexNR::Acquire()  Line 986 + 0xe bytes	C++
.exe!ZAtomic_Inc(ZAtomic_t * inValueAddress=0x0744014c)  Line 449	
C++
.exe!ZRefCounted::sIncRefCount(ZRefCounted * inObject=0x07440148)  Line 35 
+ 0xc bytes	C++
.exe!ZRef<Ng::Action>::operator=(Ng::Action * other=0x07440148)  Line 189 
+ 0x9 bytes	C++

I went back to trying to get the inline assembly working in ZAtomic.cpp, 
changing asm{ to __asm{, but it fails on lines such as:
lock xadd [edx], inParam, with error C2415: improper operand type.  The 
compiler compiles my SSE code OK, but I`m not that knowledgable on 
assembly.  

I`m getting an error trying to browse the CVS online, is there a more 
recent version of ZAtomic, or any other tips for using ZooLib and VS 2005?

I also noticed an old post on using ZooLib for PhotoShop plugins, was any 
progress ever made?

Thanks,

Paul Nolan, CEO Idruna Software Inc.
http://www.idruna.com

[ZooLib-dev] ZFile API changes

[Andrew G. <ag...@em...>]

I've added Size(), TimeCreated() and TimeModified() methods to 
ZFileSpec, and made concomitant additions to ZFileLoc and its various 
concrete subclasses. I've also changed the names of some of the 
navigational methods on ZFileSpec, partly for aesthetics and partly to 
invalidate existing code because of a change in the interpretation of 
path parameters. Finally I've made most of the code more friendly 
towards C++ libraries where vector<XX>::iterator is a real type and not 
just a typedefed pointer.

The new ZFileSpec method names are Parent, Child, Sibling and Trail. 
They replace GetParent, GetChild, GetSibling and GetDescendant.

Parent returns the parent of the ZFileSpec, just as GetParent did.

Child takes a *filename* as a parameter and returns a ZFileSpec 
referencing the child node with that name. This differs from GetChild, 
which took a ZFileTrail which generally was conversion-constructed from 
a passed-in string. By taking just a name you can now reference 
arbitrary node names, including those containing the posix file 
separator.

Trail works like the old GetChild, in that takes a ZFileTrail. The 
ZFileTrail can be constructed from a string, in which case posix path 
notation is assumed and interpreted. It can also be constructed step by 
step, or by passing iterators referencing a string container, in which 
case there are no restrictions on the names used at each step of the 
trail (except for the empty string, indicating a bounce).

Sibling effectively calls this->Parent().Child(const string&). If you 
want to walk a trail starting at the parent then you should explicitly 
call theFileSpec.Parent().Trail(theTrail).

I've also changed the default values on the various Open and Create 
methods' iPreventWriters parameter. The rule now is that if you open or 
create with any write access then iPreventWriters by default is true. 
Otherwise it defaults to false. I wonder if perhaps there should not be 
any default values provided at all, but they remain until I hear a good 
argument for losing them.

A+
-- 
Andrew Green                           mailto:ag...@em...
Electric Magic Co.               vox: +1 (831) 621 1822

Re: [ZooLib-dev] Re: [ZooLib-checkins] CVS: zoolib/src_other/tuplebase ZTS_BlockStore.cpp,1.4,NONE ZTS_BlockStore.h,1.3,NONE

[Andrew G. <ag...@em...>]

On Aug 26, 2004, at 01:50, Richard Clark wrote:
> I had to back away earlier due to workload issues. I'm making a new 
> attempt using the latest shipping xcode, though it's slow going. There 
> will need to be a couple of casts to hush the compiler (it doesn't 
> like "return nil" when the return type is ZRef<...>).

Here's the comment from zconfigl.h that explains nil's provenance:

/* We use the macro 'nil' extensively in ZooLib. It's something of a 
historical accident, basically
the result of an early collaborator Cyrus Ghalambor having previously 
worked in a pascal-centric
environment. NULL might more normally be expected, but its official C 
definition makes it problematic
as a subsitute for a null pointer. And there's a problem with having 
nil be simply zero, in that it
then cannot be distinguished from a zero integer. Herb Sutter's column 
in C/C++ User's Journal, May 2004
discusses 'nullptr', which will likely end up part of C++ in the next 
standardization cycle.
In the meantime I'm re-defining nil from being zero to something like 
nullptr. */

I've sucked all of ZooLib into XCode and compiled everything, although 
I haven't actually built a running app of any sort yet. Doing so threw 
up a few mistakes, and also a few situations where I had not corrected 
the use of nil. Some places we were passing nil when a zero integer was 
expected. In other cases we were returning nil when a 
default-constructed ZRef<XX> was required.

The code that's in cvs should already have all those fixes in place (I 
just now checked in a handful more).

A+
-- 
Andrew Green                           mailto:ag...@em...
Electric Magic Co.               vox: +1 (831) 621 1822

[ZooLib-dev] Re: [ZooLib-checkins] CVS: zoolib/src_other/tuplebase ZTS_BlockStore.cpp,1.4,NONE ZTS_BlockStore.h,1.3,NONE

[Richard C. <rd...@ne...>]

On Aug 23, 2004, at 15:24, Andrew Green wrote:

 > Hi. Nice to hear from you again. How did you get on in the end with 
xcode and zoolib?

I had to back away earlier due to workload issues. I'm making a new 
attempt using the latest shipping xcode, though it's slow going. There 
will need to be a couple of casts to hush the compiler (it doesn't like 
"return nil" when the return type is ZRef<...>).

  ...Richard

[ZooLib-dev] Re: [ZooLib-checkins] CVS: zoolib/src_other/tuplebase ZTS_BlockStore.cpp,1.4,NONE ZTS_BlockStore.h,1.3,NONE

[Andrew G. <ag...@em...>]

Hi. Nice to hear from you again. How did you get on in the end with 
xcode and zoolib?

On Aug 18, 2004, at 23:22, Richard Clark wrote:

> The only thing to look out for is that BerkeleyDB breaks compatibility 
> with old data stores whenever they upgrade the software. There needs 
> to be as good dump/restore capability, perhaps as a serialization 
> mechanism in ZBlockStore.

Where things stand at the moment with the tuplebase stuff is that the 
application-level API is provided by ZTB, ZTxn, ZTBSpec, ZTBQuery, 
ZTBIter and ZTuple. That stuff has been stable even in most of the 
details for the last 8 months or so, and the fundamental stucture has 
been unchanged for the last 20 months.

The implementation resides in subclasses of ZTBRep. There are two of 
those right now, ZTBRep_Client and ZTBRep_TS. ZTBRep_Client uses a 
ZStreamR/ZStreamW pair to talk to an instance of ZTBServer which itself 
talks to an instance of a ZTBRep subclass. Because the ZTBRep API is 
async the ZTBRep_Client/ZTBServer combo is able to be more efficient 
than if it were simply proxying calls across a comms link.

ZTBRep_TS is where most of the real tuplebase work happens, but it 
delegates the actual storage and basic searching of tuples to an 
instance of a ZTS subclass. The only extant subclass of ZTS is ZTS_RAM, 
which keeps all the tuples in a map<> and maintains a number of indices 
over the tuples. ZTS_RAM can dump the tuples it holds to a stream in 
binary format, or to a strim in parseable text format. It can also load 
its tuples from a stream or a strim, regenerating its indices as it 
does so.

The ZTS API is where we could make use of BerkeleyDB, mainly so the 
indices are persistent. Loading the tuples themselves is pretty quick; 
on my laptop it does at about 10,000/second in binary format, 
1,000/second in text format. The general index code runs about 1,000 
tuples per index per second. So If you have four indices generating the 
index takes 80% of the load up time for tuples coming from a strim, 99% 
for a stream.

> One web host I know basically tells people not to use BerkeleyDB for 
> their blogs and to configure for MySQL instead.

If the BerkeleyDB format is that volatile we can continue to store 
tuples in a flat stable file format (maybe some log-structured thing) 
and only use BerkeleyDB for the indices.

A+
-- 
Andrew Green                           mailto:ag...@em...
Electric Magic Co.               vox: +1 (831) 621 1822

[ZooLib-dev] I Need Your Screenshots!

[Michael D. C. <cra...@go...>]

Help!  I need you to send me screenshots of your zoolib applications. 
Anything that shows any of the capabilities of zoolib will do.

I'm trying to convince a potential client that he should have me build 
his product from zoolib.  However, he asked his business partner what he 
thought of it, and this was his reply:

> Well, that's interesting - but I was unable to see a complex, UI-rich example of Zo
> oLib.

I'm afraid his poor first impression is entirely my fault, as ZooLib's 
website was hastily slapped together when it was first released, and 
hasn't been worked on much since.

He went on to suggest I develop the application in Microsoft .NET, to be 
run on the Mac using the OS X .NET CLR.

Over My Dead Body.

I sent him some screenshots from a graphic editor called Instant 
Makeover that I wrote back in 2000, but I'd like to show off a wider 
variety of capabilities.

I'll be sending the screenshots privately to my client, but it would be 
helpful if I could post some of the screenshots on ZooLib's website. 
When you send them to me, let me know if that would be OK with you.

Thanks for your help!

Mike

Re: [ZooLib-dev] Zoolib 0.9 install on OS X

[Andrew G. <ag...@em...>]

On May 18, 2004, at 13:04, daniel civello wrote:

> Hey all, so I'm very eager to jump into ZooLib, but I can't get the 
> samples
> compiled.  Specifically, it's dying on the linker...the compiler gives 
> no

> ld: Undefined symbols:
> std::char_traits<unsigned short>::eq(unsigned short const&, unsigned 
> short
> const&)
> std::char_traits<unsigned short>::copy(unsigned short*, unsigned short 
> const*,
> unsigned long)
> std::char_traits<unsigned short>::find(unsigned short const*, unsigned 
> long,
> unsigned short const&)
> std::char_traits<unsigned short>::move(unsigned short*, unsigned short 
> const*,
> unsigned long)
> std::char_traits<unsigned short>::assign(unsigned short*, unsigned 
> long,
> unsigned short)
> std::char_traits<unsigned short>::assign(unsigned short&, unsigned 
> short
> const&)
> std::char_traits<unsigned short>::length(unsigned short const*)
> std::char_traits<unsigned short>::compare(unsigned short const*, 
> unsigned short
> const*, unsigned long)
> make: *** [ButtonMessage-debug] Error 1
>
> .....
>
> I'm running 10.3.3.   Any ideas?

Yes. Do a CVS checkout of the current source tree. The std C++ lib on 
MacOS X does not include definitions of specializations of char_traits 
for anything other than char, and there are places in ZooLib where 
we're using 16 and 32 bit code units (depending on your compiler 
settings one or other may be a wchar_t). I think that the necessary 
code might even be in the 0.9 release, but just isn't switched on when 
building gcc/mach for macos X.

A+
-- 
Andrew Green                           mailto:ag...@em...
Electric Magic Co.               vox: +1 (831) 621 1822

[ZooLib-dev] Zoolib 0.9 install on OS X

[daniel c. <ci...@pa...>]

Hey all, so I'm very eager to jump into ZooLib, but I can't get the samples
compiled.  Specifically, it's dying on the linker...the compiler gives no
errors.   

~/zoolib-0.9/samples/buttonmessage/buttonmessage_build_posix> make
g++  -I../../../samples/buttonmessage/buttonmessage_src 
-I../../../src/_deprecated  -I../../../src/asset  -I../../../src/compat 
-I../../../src/config  -I../../../src/file  -I../../../src/foundation 
-I../../../src/graphics  -I../../../src/net  -I../../../src/platform/beos 
-I../../../src/platform/macos  -I../../../src/platform/posix 
-I../../../src/platform/win32  -I../../../src/printing  -I../../../src/stream 
-I../../../src/text  -I../../../src/thread  -I../../../src/uicore 
-I../../../src/uiextra  -I../../../src_other/misc  -I./  -pipe -D_REENTRANT -g
-DDEBUG -c ../../../samples/buttonmessage/buttonmessage_src/BMApp.cpp -o
objects-debug/BMApp.o

[edited for space]

debug/TLSinside.o   -L./  -L/usr/X11R6/lib/   -lstdc++  -lpthread  -lc  -lX11 
-lXext  -lz  -lpng
ld: Undefined symbols:
std::char_traits<unsigned short>::eq(unsigned short const&, unsigned short
const&)
std::char_traits<unsigned short>::copy(unsigned short*, unsigned short const*,
unsigned long)
std::char_traits<unsigned short>::find(unsigned short const*, unsigned long,
unsigned short const&)
std::char_traits<unsigned short>::move(unsigned short*, unsigned short const*,
unsigned long)
std::char_traits<unsigned short>::assign(unsigned short*, unsigned long,
unsigned short)
std::char_traits<unsigned short>::assign(unsigned short&, unsigned short
const&)
std::char_traits<unsigned short>::length(unsigned short const*)
std::char_traits<unsigned short>::compare(unsigned short const*, unsigned short
const*, unsigned long)
make: *** [ButtonMessage-debug] Error 1

.....

I'm running 10.3.3.   Any ideas?

thanks,
-daniel

[ZooLib-dev] FASD project: Online survey launched

[Benno L. <ben...@id...>]

Dear Open Source developer

I am doing a research project on "Fun and Software Development" in which I
kindly invite you to participate.
You will find the online survey under http://fasd.ethz.ch/qsf/. The
questionnaire consists of 53 questions and you will need about 15 minutes to
complete it.

With the FASD project (Fun and Software Development) we want to define the
motivational significance of fun when software developers decide to engage
in Open Source projects. What is special about our research project is that
a similar survey is planned with software developers in commercial firms.
This procedure allows the immediate comparison between the involved
individuals and the conditions of production of these two development
models. Thus we hope to obtain substantial new insights to the phenomenon of
Open Source Development.


With many thanks for your participation,
Benno Luthiger


PS:
The results of the survey will be published under
http://www.isu.unizh.ch/fuehrung/blprojects/FASD/.
We have set up the mailing list fa...@we... for this study. Please
see http://fasd.ethz.ch/qsf/mailinglist_de.html for registration to this
mailing list.

_______________________________________________________________________

Benno Luthiger
Swiss Federal Institute of Technology Zurich
8092 Zurich

Mail: benno.luthiger(at)id.ethz.ch
_______________________________________________________________________

[ZooLib-dev] Will be working on Cookbook again

[Michael D. C. <cra...@go...>]

Greetings, ZooLib Fans,

I plan to start work on the ZooLib Cookbook again, and hopefully this time work 
on it steadily until it's complete.

I'm going to start by getting DocBook configured on my iBook using fink, so I 
can take my writing anywhere.

What my plan has always been is for each chapter to illustrate a few important 
concepts about ZooLib by explaining the source code of a demo application.  But 
we don't have that many demos yet.  I could use suggestions for what the demos 
could be.  I can code the demos myself, but I'm not too sure what sort of apps 
would suit this purpose.

If you want to read what I've done on the cookbook so far, I have a draft 
available at http://www.goingware.com/zoolib/cookbook/

Also, I'm not so sure now that I should be using the GNU Free Documentation 
License for The ZooLib Cookbook.  There is some controversy over it.  The debian 
linux developers have decided that the GFDL is not compliant with the Debian 
Free Software Guidelines, and won't include anything licensed with the GFDL in 
their main distribution.  I wrote more about this in "Which License for Free 
Documentation" at http://advogato.org/article/682.html

Do you think I should use a different license, or keep the GFDL?  Why do you 
think so, whichever you prefer?

I have written some non-technical articles that have Creative Commons licenses. 
   One has a choice of license terms with Creative Commons:

http://creativecommons.org/licenses/

If I were to use Creative Commons, most likely I would choose 
Attribution-Sharealike.

The zoolib codebase has the MIT license.  Would it be preferable for the ZooLib 
Cookbook to have that?

Thanks for your help,

Mike
-- 
Michael D. Crawford
GoingWare Inc. - Expert Software Development and Consulting
http://www.goingware.com
cra...@go...

   Tilting at Windmills for a Better Tomorrow.

     "I give you this one rule of conduct. Do what you will, but speak
      out always. Be shunned, be hated, be ridiculed, be scared,
      be in doubt, but don't be gagged."
      -- John J. Chapman, "Make a Bonfire of Your Reputations"
         http://www.goingware.com/reputation/

[ZooLib-dev] Buttonmessage example

[Thorsten M. <ma...@mp...>]

i have tried to compile the button example on visual c++ 6.0
but i got thie error:
_MT is not defined. ZooLib requires thread safe libraries and headers.


i dont know what to do

[ZooLib-dev] Compilation failure on POSIX (FreeBSD)

[Richard C. <rd_...@sb...>]

I was building the paintertest sample of FreeBSD, using the POSIX build=
 environment, and it fails in ZFile_POSIZ.cpp, with a long run of errors=
 claiming there's an unterminated #ifdef

Try as I might, i can't see what's gone wrong...can one of you gurus give it=
 a shot?

Thanks,

 ...Richard

Re: [ZooLib-dev] GCC Headers "false alarm", but a linking problem now...

[Richard C. <rd_...@sb...>]

>=A0However, it's not linking properly -- buttonmessage_src/realmake
>=A0lacks a libraries definition for POSIX (just one for LINUX). When I
>=A0duplicated the LINUX line, I had to remove the reference to "png"
>=A0(not in standard BSD) and the reference to "pthread" wasn't found.

Changing "c" to "c_r" in my POSIX library definition line did the trick, but=
 this is FreeBSD-specific.

 ...R

[ZooLib-dev] GCC Headers "false alarm", but a linking problem now...

[Richard C. <rd_...@sb...>]

It turns out the headers issue (no NAN defined) was a false alarm...the=
 FreeBSD gcc ports package didn't install the latest headers. I moved over=
 to FreeBSD 5.1 (with a proper GCC 3.2.2 toolchain) and it's all compiling.

However, it's not linking properly -- buttonmessage_src/realmake lacks a=
 libraries definition for POSIX (just one for LINUX). When I duplicated the=
 LINUX line, I had to remove the reference to "png" (not in standard BSD)=
 and the reference to "pthread" wasn't found. Here's what the man page has=
 to say:

"The current FreeBSD POSIX thread implementation is built in the library
  libc_r which contains both thread-safe libc functions and the thread
  functions.  This library replaces libc for threaded applications."

You can get the same effect by specifying the -pthread option to GCC, as=
 some other *BSD versions did, though I think everyone else is past that=
 now.

So, what are your thoughts on this? Where should the FreeBSD threading/build=
 options go? (Or is it time to grag in GNU Autoconf?)

 ...Richard

[ZooLib-dev] Compiler glitches on *NIX

[Richard C. <rd_...@sb...>]

It's me again...

I'm setting up a new UNIX box (FreeBSD) as a development station, in part=
 tocheck the Zoolib example(s) I'm writing and I ran into compilation=
 =A0problems running the "POSIX" build style:
- FreeBSD 4.9 uses the GCC 2.95 compiler chain, which wouldn't compile=
 the=A0code (no <sysint.h>).
- Upgrading to GCC 3.3.3 compiled more of the code, but failed to find=
 the=A0NAN used in ZTIme.cpp (it's not in <math.h>, though _isnan is.)

(FYI, I ran a find for \*math.h and grepped the results for NAN -- no=
 such=A0luck.)

So, my questions are:
1. Which GCC did you test with (and on which platforms?)
2. How can/should we handle an uninitialized time on platforms that don't=
 have NAN defined as a constant? (Perhaps a flag in ZTIme?)

=A0...Richard

Re: [ZooLib-dev] Finding a ZooLib "exemplar"

[Andrew G. <ag...@em...>]

On Tuesday, Dec 9, 2003, at 15:25 US/Central, Richard Clark wrote:
> I was thinking of a "contact manager" program -- one of those tools 
> that sales professionals use that combines an address book with 
> reminders to make contact (and provides a place to store notes.) That 
> calls for a multi-part layout -- a list of addresses, a selected 
> address' details, and notes on those calls. It's a natural for the 
> database, and could be extended to use a local "customer info" server. 
> I could even see a use for threads (as you type in a name, the DB 
> lookup happens concurrently on a thread.)

I like the idea. I had been thinking of a picture browser, akin to 
gqview, but that entails a lot of image-processing stuff which would 
swamp the more general aspects of an app.

However, rather than using ZDBase I'd very strongly suggest looking at 
the tuplebase stuff. It manages freeform tuples rather than records 
from fixed-schema tables and all access is transacted, whereas ZDBase 
requires careful locking. The ZTB API and semantics remain the same 
regardless of the actual underlying implementation (locally hosted or 
via network socket to another process or host).

It's been a couple of years since I've had a GUI app in development, so 
this examplar would very helpful in getting the ZooLib UI code updated 
-- it's become a little crusty in places (see TODO.html for specifics), 
and some architectural issues arose that I think I've now got solutions 
for; it would be good to work them out with new code that's still 
maleable.

A+
-- 
Andrew Green                           mailto:ag...@em...
Electric Magic Co.           Vox/Fax: +1 (408) 907 2101