nitric-development — Discussion about the development of Nitrogen

[Nitric-dev] On version control

[Joshua J. <jj...@gm...>]

I've finally converted my active SourceForge projects from CVS to  
Git.  Now I propose the same for Nitrogen.

There are numerous advantages to using Git over CVS.  Off the top of  
my head:

* local repository
	* offline access (consult history or check in on an airplane!)
	* speculative checkins that are later published or discarded
* cheap, easy branching and merging
* cheap, easy renaming (retaining history)
* history repair -- include changes from Pending but pretend it never  
existed.
* fast
* user-friendly -- e.g. git status suggests what to do next

The converted Nitrogen repository could be hosted at SourceForge,  
GitHub, or both.

Thoughts?  Objections?

Josh

Re: [Nitric-dev] Owned filesystem objects

[Joshua J. <wan...@me...>]

Good to hear from you!

On Jun 13, 2007, at 3:16 PM, Lisa Lippincott wrote:

> On Jun 1, 2007, at 12:35 AM, Joshua Juran wrote:
>
>> Nitrogen/Files.h defines a class FileSystemDisposer capable of
>> deleting an FSRef or FSSpec, which can be used for temporary files.
>> Currently, Owned<FSRef> is used by several functions in Nitrogen/
>> Files -- it's consumed by FSDeleteObject(), and referenced by
>> FSMoveObject() and FSRenameUnicode().  Similarly, Owned<FSForkRef> is
>> returned from FSCreateFork() and consumed by FSDeleteFork().  The
>> equivalent FSSpec routines do not work with Owned<FSSpec> at all, so
>> we have an inconsistency.
>>
>> To resolve it, I'm going to modify the FSRef interfaces to use plain
>> FSRef and FSForkRef.  I have several reasons for this, but primary
>> among them is that the purpose of smart resource management (as
>> provided in Nitrogen by Owned and Shared) is to prevent leaks of
>> resources which are no longer used or are members of an object whose
>> construction was terminated by exception, and to prevent double-
>> destruction of resources -- and filesystem objects are not leakable
>> (since they are enumerated by the filesystem itself) or doubly
>> destructable (since attempts to delete a nonexistent file are
>> detected and simply yield fnfErr or the like, rather than a crash).
>
> I think this is the wrong direction.  The motivation for using
> Owned<FSRef> was not only uniformity (creations always return Owned)
> but also because it makes correct behavior easier.

The current design needs to be reexamined.  As is, it has issues that  
make it more dangerous than the risk of leaving a stray file.

One problem is the need to seize files just to perform operations.   
If I write:

	NN::Owned<FSRef> file = NN::Owned<FSRef>::Seize( ... );
	N::FSMoveObject( file, destDir );
	file.Release();

...and FSMoveObject() throws, my file gets deleted.  So as not to  
throw the baby out with the bathwater, from here on I'll construe  
your design as applying to creation and deletion, and not to  
manipulation.

We need to come up with use cases.  The most obvious is temporary  
files, to which Owned is well-suited, regardless of the level at  
which we apply it.  Others include creating a new document, saving an  
old document, downloading a file, converting files, and archiving/ 
dearchiving.  Less common in typical applications but also requiring  
consideration are file management primitives, as in the Finder or a  
command shell.

The shell is an interesting subtopic:  Although it uses POSIX filing  
rather than the File Manager, the same arguments apply to POSeven.   
The problem here is that creat() is a special case of open() -- open 
() might or might not create a file, just like AESend() may or may  
not return a reply descriptor, depending on the send mode.  Also,  
unlike AESend() which can always return an empty Owned<AEDesc>,  
P7::Open() can't return an Owned<std::string?> because it has to  
return an Owned<FileDescriptor>.

> (How the FSSpec routines ended up the other way is a mystery to me.
> I think that's a bug.)

The FSSpec routines were added without guaranteed destruction  
semantics because I wrote them that way, based on concerns which have  
since developed into the arguments I present here.

> Even with non-temporary files, it's not a good idea to immediately
> release the FSRef.  It's better to create files transactionally:  
> create,
> write, flush and close, then release.  That way you don't leak turd
> files onto the disk when writing a file goes wrong.

I agree that this is a good practice.  But I would like to see its  
enforcement be a separate mechanism, not comingled with filing  
primitives.

> What's more, adding a Seize is not a remedy for not returning an Owned
> object in the first place.  It leaves code open to the classic order
> of operation problem in nested calls like this:
>
>             Foo( Seize( Create<X>() ), Bar() )
>
> Here Bar() could be called between Create<X>() and Seize(), and if  
> Bar()
> fails, the created item would be leaked.

We can write Foo( Create_Tentative<X>(), Bar() ) instead.  We don't  
have to make the non-seizing primitives unavailable.

> The exception to this is files that grow by appending.  For them, a
> truncated file is still useful, if the truncation goes back to a
> clean break.  I think that a tentative-append class would be best for
> these, but that seems beyond the scope of Nitrogen.

This requires its own solution, an append transaction based on  
Tentative which on failure truncates the file back to its previous  
length.

My overall point here is that I don't object to rollback-on-failure  
semantics, but I want them separate from the usual Nitrogen throw- 
exception-on-error semantics.  And I think 'roll back' vs. merely  
'dispose' is a very important distinction.  One is destructive in a  
way that the other is not.

Josh

Re: [Nitric-dev] Owned filesystem objects

[Lisa L. <li...@ha...>]

On Jun 1, 2007, at 12:35 AM, Joshua Juran wrote:

> Nitrogen/Files.h defines a class FileSystemDisposer capable of
> deleting an FSRef or FSSpec, which can be used for temporary files.
> Currently, Owned<FSRef> is used by several functions in Nitrogen/
> Files -- it's consumed by FSDeleteObject(), and referenced by
> FSMoveObject() and FSRenameUnicode().  Similarly, Owned<FSForkRef> is
> returned from FSCreateFork() and consumed by FSDeleteFork().  The
> equivalent FSSpec routines do not work with Owned<FSSpec> at all, so
> we have an inconsistency.
>
> To resolve it, I'm going to modify the FSRef interfaces to use plain
> FSRef and FSForkRef.  I have several reasons for this, but primary
> among them is that the purpose of smart resource management (as
> provided in Nitrogen by Owned and Shared) is to prevent leaks of
> resources which are no longer used or are members of an object whose
> construction was terminated by exception, and to prevent double-
> destruction of resources -- and filesystem objects are not leakable
> (since they are enumerated by the filesystem itself) or doubly
> destructable (since attempts to delete a nonexistent file are
> detected and simply yield fnfErr or the like, rather than a crash).

I think this is the wrong direction.  The motivation for using
Owned<FSRef> was not only uniformity (creations always return Owned)
but also because it makes correct behavior easier.

(How the FSSpec routines ended up the other way is a mystery to me.
I think that's a bug.)

Even with non-temporary files, it's not a good idea to immediately
release the FSRef.  It's better to create files transactionally: create,
write, flush and close, then release.  That way you don't leak turd
files onto the disk when writing a file goes wrong.

What's more, adding a Seize is not a remedy for not returning an Owned
object in the first place.  It leaves code open to the classic order
of operation problem in nested calls like this:

            Foo( Seize( Create<X>() ), Bar() )

Here Bar() could be called between Create<X>() and Seize(), and if Bar()
fails, the created item would be leaked.

The exception to this is files that grow by appending.  For them, a
truncated file is still useful, if the truncation goes back to a
clean break.  I think that a tentative-append class would be best for
these, but that seems beyond the scope of Nitrogen.

                                                  --Lisa

[Nitric-dev] Owned filesystem objects

[Joshua J. <jj...@gm...>]

Nitrogen/Files.h defines a class FileSystemDisposer capable of  
deleting an FSRef or FSSpec, which can be used for temporary files.   
Currently, Owned<FSRef> is used by several functions in Nitrogen/ 
Files -- it's consumed by FSDeleteObject(), and referenced by  
FSMoveObject() and FSRenameUnicode().  Similarly, Owned<FSForkRef> is  
returned from FSCreateFork() and consumed by FSDeleteFork().  The  
equivalent FSSpec routines do not work with Owned<FSSpec> at all, so  
we have an inconsistency.

To resolve it, I'm going to modify the FSRef interfaces to use plain  
FSRef and FSForkRef.  I have several reasons for this, but primary  
among them is that the purpose of smart resource management (as  
provided in Nitrogen by Owned and Shared) is to prevent leaks of  
resources which are no longer used or are members of an object whose  
construction was terminated by exception, and to prevent double- 
destruction of resources -- and filesystem objects are not leakable  
(since they are enumerated by the filesystem itself) or doubly  
destructable (since attempts to delete a nonexistent file are  
detected and simply yield fnfErr or the like, rather than a crash).

You can still create an Owned<FSRef> temporary file by seizing an FSRef.

Josh

Re: [Nitric-dev] FSRead() behavior

[Marshall C. <mar...@id...>]

At 1:55 AM -0800 1/16/07, Joshua Juran wrote:
>Unless I'm overinterpreting, we seem to have consensus for FSRead
>[Fork].  FSRead() has already been revised for POSIX semantics 
>(return all non-zero byte counts).  FSReadFork() is still throwing 
>EOFErr on short reads, and a year after raising the issue, I'd say 
>its time has finally come.

I agree.
At the very least, FSRead and FSReadFork should behave the same ;-)

>If no one objects, I'll modify FSReadFork() similarly to FSRead().  
>The difference in behavior will be that short reads return the byte 
>count instead of throwing EOFErr.  Chances are that you aren't
>relying on the old behavior, since it's basically broken.

No objection from me.
-- 
-- Marshall

Marshall Clow     Idio Software   <mailto:mar...@id...>

It is by caffeine alone I set my mind in motion.
It is by the beans of Java that thoughts acquire speed,
the hands acquire shaking, the shaking becomes a warning.
It is by caffeine alone I set my mind in motion.

Re: [Nitric-dev] FSRead() behavior

[Joshua J. <wan...@me...>]

On Jan 19, 2005, at 8:06 AM, Marshall Clow wrote:

>>> The general case "where a call partially succeeds" needs to be  
>>> addressed.
>>>
>>> Since there are several cases of this in Carbon (FSRead,  
>>> FSReadFork, ConvertUnicodeToText, ConvertTextToUnicode, to name  
>>> just four) - they should all behave the same, and there should
>>> be an explicit "Nitrogen Rule" for this kind of thing.
>>
>> I'd like there to be a rule, but I'm not sure how to phrase it.   
>> Here are
>> some things I think are worth considering:
>
> I think that this is a general enough case that we should handle it  
> consistently.
>
>> Does the result code really indicate a failure?  Or did some  
>> function-result
>> information get mixed into the result code?
>
> In the cases I indicated, the Nitrogen result is the number of  
> bytes "processed".
> We're talking about the cases where a call processes some of the  
> bytes, and then fails;
> and the calling application needs to be (IMHO) notified of both (a)  
> how much succeeded,
> and (b) why the "rest of the data" was not processed.
>
>> Is there a side effect that isn't rolled back, and typically won't  
>> be ignored
>> by a program?
>
> Yes - in both these cases. In FSReadXXX, the "current mark" in the  
> file has moved,
> and in ConvertFromXXXX, some of the data has been converted, and  
> the "# of bytes
> processed" function return is necessary to continue processing.
>
>> Will throwing an exception typically encourage local try-blocks,  
>> or is it
>> practical to catch the exception far from where it was thrown?
>
> Throwing an N::OSStatus will lose important information. Throwing a  
> custom
> exception-type will encourage local exception handling, because....
>
>> Is the exception self-contained, or will the catcher need access  
>> to the
>> parameters to the original function call?
>>
>> My feeling is that these questions point toward not throwing in  
>> the FSRead
>> case.  I don't know enough about ConvertTextToUnicode to judge.

Unless I'm overinterpreting, we seem to have consensus for FSRead 
[Fork].  FSRead() has already been revised for POSIX semantics  
(return all non-zero byte counts).  FSReadFork() is still throwing  
EOFErr on short reads, and a year after raising the issue, I'd say  
its time has finally come.

If no one objects, I'll modify FSReadFork() similarly to FSRead().   
The difference in behavior will be that short reads return the byte  
count instead of throwing EOFErr.  Chances are that you aren't  
relying on the old behavior, since it's basically broken.

Josh

[Nitric-dev] Wrapped Integral Type Safety

[Joshua J. <wan...@me...>]

My proposed solution to wrapping integral types is to use enums  
generated by templates.  There don't exist enum templates, but a  
class template can have an enum member.  The basis of this plan is  
struct Enumeration<>:

namespace Nucleus
{
	
	template < class Tag, class Int >
	struct Enumeration
	{
		typedef Int IntegralType;
		
		const static bool isSigned = IntegralType( -1 ) < IntegralType( 0 );
		
		const static int byteWidth = sizeof (IntegralType);
		const static int bitWidth  = byteWidth * 8;
		
		const static int binaryDigits = bitWidth - int( isSigned );
		
		const static IntegralType min = isSigned ? 1ULL << binaryDigits : 0;
		const static IntegralType max = IntegralType(~0) ^ min;
		
		enum Type
		{
			kMin = min,
			kMax = max
		};
	};
	
}

The Int parameter specifies the underlying built-in type, e.g.  
UInt32, and Tag distinguishes between different types with the same  
underlying type (e.g. DescType vs. ResType).  Defining kMin and kMax  
gurantees that the enum's storage size will be at least as large as  
the underlying type (although it may be larger, if enums are treated  
as ints by the compiler).

The new replacements for FlagType, IDType, and SelectorType are Flag,  
ID, and Selector.  Here's one example:

namespace Nucleus
{
	
	template < class Tag, class Int >
	struct Selector
	{
		typedef typename Enumeration< Tag, Int >::Type Type;
	};
	
}

The heavy lifting is done by Enumeration.  The only thing Selector  
adds is documentative value.

Here's how it's used:

namespace Nitrogen
{

	// Old SelectorType usage
	//typedef Nucleus::SelectorType< class DescType_Tag, ::DescType >  
DescType;

	// New Selector::Type usage
	typedef Nucleus::Selector< class DescType_Tag, ::DescType >::Type  
DescType;

}

Since Enumeration::Type is an enum, not a class, you can't specify a  
default value (since enums don't have constructors).  You *can*,  
however, use it as a template parameter:

	// Can't use SelectorType-based N::DescType as template parameter;  
fall back to underlying type
	//template < ::DescType > struct DescType_Traits;

	// Selector::Type is ok
	template < DescType > struct DescType_Traits;

[Flag needs additional commentary to cover its operators, to be  
provided later.]

The consequence of implementing type-safety is having to adhere to  
it.  Having to write N::DescType( typeChar ) everywhere is not anly  
annoying, but won't save you from mistakes like writing N::DescType 
( pVersion ) instead of N::DescType( typeVersion ).  Instead, my plan  
includes providing constants which have the types built in:

	typedef Nucleus::Selector< class DescType_Tag, ::DescType >::Type  
DescType;
	
	static const DescType typeBoolean                = DescType 
( ::typeBoolean                );
	static const DescType typeChar                   = DescType 
( ::typeChar                   );
	
	static const DescType typeSInt16                 = DescType 
( ::typeSInt16                 );
	static const DescType typeSInt32                 = DescType 
( ::typeSInt32                 );
	static const DescType typeUInt32                 = DescType 
( ::typeUInt32                 );
	static const DescType typeSInt64                 = DescType 
( ::typeSInt64                 );
	static const DescType typeIEEE32BitFloatingPoint = DescType 
( ::typeIEEE32BitFloatingPoint );
	static const DescType typeIEEE64BitFloatingPoint = DescType 
( ::typeIEEE64BitFloatingPoint );
	static const DescType type128BitFloatingPoint    = DescType 
( ::type128BitFloatingPoint    );
	static const DescType typeDecimalStruct          = DescType 
( ::typeDecimalStruct          );

You get the idea.  Now, the Nitrogen user has the option of either  
writing N::typeChar, or preceding usage of typeChar with using  
N::typeChar; inside a namespace or function block.  Inside namespace  
Nitrogen, no change is necessary -- typeChar refers to the  
N::DescType constant, and gracefully degrades to ::DescType as  
necessary.

Josh

[Nitric-dev] Wrapped Integral Types: A History

[Joshua J. <wan...@me...>]

Since its inception, Nitrogen has had means of creating new distinct  
types which corresponded to conflated built-in types, such as those  
used by Carbon (e.g. OSType, ResType, DescType, etc.).  There are  
some special cases like OSStatus and the dually-signed types in  
Files.h (FSDirID, FSSharingFlags, and FSUserPrivileges), but the  
primary mechanism for defining Nitrogen's counterparts to Carbon's  
aliases of integer builtins has been the trio FlagType, IDType, and  
SelectorType, which I'll collectively label IntType.

The goal of type-safety is elusive -- creating a new N::DescType is  
easy enough, but having it be constructible from typeWildCard and not  
from keyDirectObject is tricky at best.

IntType's strategy is to allow implicit construction from the  
underlying type and refuse the rest.  The tendency of anonymous enum  
constants to evaluate as 'int', while the underlying types are signed  
or unsigned short or long, leaves the Nitrogen user writing  
N::DescType( typeWildCard ), or DescType( typeWildCard ) within  
namespace Nitrogen.  With constants at least, there's no safety  
added, just inconvenience.  (I concede that the extra verbiage could  
have documentative value, but at the expense of readability.)

The mechanics of IntType requires defining many operators in the  
class template, some to provide for allowed operations that normally  
would occur automatically, and more to prohibit operations with the  
wrong types.  One side effect is the 'hijacking' of built-in  
operations on built-in types; one example that actually occurred was  
char_foo & int_bar -- this expression became illegal, and had to be  
qualified with otherwise superfluous type coercion.  (See September  
7, 2004 list email.)

Additionally, as Nitrogen grew, the volume of IntType instantiations  
overwhelmed CodeWarrior's capacity to process it in any reasonable  
length of time.  CW Pro 9.3 addressed the issue, but older versions  
remain hosed.  Speaking personally, this was unacceptable as I have a  
business requirement for building code resources and a very strong  
desire to be able to develop on Mac OS 9 -- capping my minimum  
supported development environment at CW Pro 6.  All of that  
notwithstanding, IntType's monumental verbosity is (in this  
developer's humble opinion) a maintenance concern, an eyesore, and at  
best, too much effort for too little benefit.

(I should note at this point that I mean no offense -- I know Lisa is  
brilliant, and I'm only here criticising her work because I  
absolutely love Nitrogen.)

Finally, since an IntType is a class, it can't be used as a template  
parameter and instances of it can't be used as subjects of switch  
statements.

In summer 2004, prior to the critical CodeWarrior update, Lisa  
proffered new versions of IntType which used constructor templates,  
to address the compile time problem, but these turned out to be  
unworkable, at least for me (for reasons I described in a September  
6, 2004 email to the list) and shortly followed by CW Pro 9.3, which  
left Marshall happy with the original code.  My solution was to gut  
IntType of all but the basics, which I checked in guarded by  
JOSHUA_JURAN_EXPERIMENTAL, effectively forking Nitrogen.  My fork  
avoided slow compiles in all compilers and the aforementioned  
hijacked operators, at the cost of type-safety.  The state of affairs  
has been unchanged since then.

...Until now.  Last year I described the problem to Sean Parent, who  
explained that named enums have properties we may find useful...

TO BE CONTINUED

Josh

Re: [Nitric-dev] Nitrogen and NavServices

[Lisa L. <li...@ha...>]

> Since there is no scoping relationship between the place where the
> dialog is created and where it is destroyed, a solution like Owned
> will not suffice. Same for the UPPs, whose lifetimes (probably) match
> the dialogs.

The problem here is thinking Owned is only about scoped
ownership.  I think what you need here is an object which
represents "the thing you're doing that's requires all this
stuff."  That object can own (through Owned<T> member objects
or auto_ptrs) the things it needs, such as NavServices
dialogs.

That task object might be owned in turn by a longer-lived
object -- say, a document interface object for a document
you're saving.  Or, for a document-opening dialog, it might
be owned by an application object, like other modeless
windows.

                                             --Lisa

[Nitric-dev] Nitrogen and NavServices

[Marshall C. <mar...@id...>]

I am attempting to write some Nitrogen wrappers for NavServices, and 
running into some difficulties.
So, I thought I would share my troubles here, in the hope that either:
1) Writing them down would suggest a solution, or
2) Someone else could suggest an answer.

[ If a good answer does emerge, this could act as the basis for some 
documentation! Win-win! ]

So, if all we had were modal NavServices dialogs, then we would be golden.
The basic idea for them (taken from the NavServices overview doc, and 
Apple sample code) is:

1) Create the dialog options.
2) Create between zero and three UPPs
3) Create the dialog, with some UPPs and the options
4) Run the dialog.
5) Get the results.
.... do something with the results ....
6) Dispose the results
7) Dispose the UPPs.
8) Dispose of the dialog.

The Nitrogen "Owned" paradigm takes care of this just fine. The 
dialog, UPPs, and results
can be wrapped in Nucleus::Owned<> classes, and steps #6,7, and 8 
will happen automatically.

We can create templated functions that will give us typed refCons as 
well (see the CarbonEventHanders for examples).

Exception safety is two-fold:
1) Making sure that things are cleaned up in the case of an exception 
being thrown. This is handled by the Owned<> variables.
2) Making sure that an exception does not escape from the callbacks 
(the UPPs) passed into NavServices.
This can be handled by making template functions that wrap a routine 
in a try/catch block.

That's all well and good - but Mac OS X is all about flexibility; and 
modal NavServices dialogs are not flexible. Most dialogs will be 
modeless (or sheets, which are effectively modeless). For these, the 
flow
of control is somewhat different:

1) Create the dialog options.
2) Create between zero and three UPPs
3) Create the dialog, with some UPPs and the options
4) Run the dialog.
... which returns and the program goes on with it's life; the dialog 
is now running its own event loop.

In one of the callbacks (NavEventHandler), on a certain event, the 
program is supposed to:

	5) Get the results.
	.... do something with the results ....
	6) Dispose the results
	7) Dispose the UPPs.
	8) Dispose of the dialog.

(However, I don't know what is supposed to happen if you dispose of a 
UPP while that UPP is being invoked. I'm guessing it's a moot point, 
since, these days, a UPP is just a procedure pointer, and 
DisposeXXXUpp is just a no-op) But I digress.

So, how do we simplify/automate this for Nitrogen?

Since there is no scoping relationship between the place where the 
dialog is created and where it is destroyed, a solution like Owned 
will not suffice. Same for the UPPs, whose lifetimes (probably) match 
the dialogs.

Any ideas?
-- 
-- Marshall

Marshall Clow     Idio Software   <mailto:mar...@id...>

It is by caffeine alone I set my mind in motion.
It is by the beans of Java that thoughts acquire speed,
the hands acquire shaking, the shaking becomes a warning.
It is by caffeine alone I set my mind in motion.

[Nitric-dev] Flatteners vs. DescType_Traits

[Joshua J. <wan...@me...>]

Since its inception, Nitrogen has had specialized struct templates  
DescType_Traits (defined largely in Nitrogen/AEDataModel.h) allowing  
such constructions as { std::string str = N::AEGetDescData< typeChar > 
( desc ); } for all AE{Get,Put}XXX() functions.  DescType_Traits are  
also used for {Get,Set}{ControlData,EventParameter}() in Controls.h  
and CarbonEvents.h.

However, they are broken.  Handle_DescType_Traits is impossible to  
implement safely -- the handle must be locked during the call to the  
Get or Put routine, since there's no guarantee they won't move  
memory, so we must call HLock() in SetInputBufferLength() and  
PrepareOutputBuffer(), and HUnlock() or HSetState() in  
ProcessInputBuffer() and ReleaseOutputBuffer() respectively, of which  
the latter case gives us a problem:  The Put call may throw, leaving  
the handle locked.  (If Get throws, the handle gets disposed because  
it's Owned.)

Also, the terminology is very confusing since input/output is  
relative to one's point of view, which isn't specified.  The output  
of AEGetDescData() is input to the application.  You can't tell which  
the code means without reverse-engineering it.

The fundamental problem is having an object with two entry points  
that must both be called to ensure integrity.  Lisa's Flattener  
protocol solves this problem by inverting the call chain -- instead  
of the Put function calling Traits::Prepare, upstream Put, and  
Traits::Release, we  create a Putter function object (which calls  
upstream Put) and pass that as a parameter to Flattener::Put, which  
calls it when appropriate.  If the Putter throws, regular stack  
unwinding restores the previous handle state via Scoped.

A Getter or Putter has knowledge of how, given a byte buffer, to  
populate it from or copy it to somewhere.  A Flattener has knowledge  
of how to flatten a particular data type into and reassamble one from  
a byte sequence.

In February I added some missing Flatteners, made the DescType_Traits  
classes inherit from the appropriate Flatteners, and revised the  
templated AEGet/Put functions to use the new protocol.  Just recently  
I noted that a few DescTypes_Traits specializations had no  
corresponding Flatteners (Unicode and Seizing) so I wrote them.  I've  
also written Getters and Putters for Get/SetControlData and Get/ 
SetEventParameter, and wired them in but left them disabled.

The flags are gUseFlattenersForEventParameter on line 332 of  
CarbonEvents.h and gUseFlattenersForControlData on line 265 of  
Controls.h, set to false.  Set them to true to try out the new code.   
If there are no problems, I'll remove the old implementations.

Assorted notes:

Seizing_DescType_Traits is perhaps misleading.  It's only used for  
CFStringRef, which has the property that you can Seize one given to  
you by the operating system without otherwise allocating anything.   
(Compare with AliasHandle, for which you have to create your own  
Handle and copy the data.)  I don't see how this can work for  
anything other than a Core Foundation type.  Still, eliminating  
mention of CoreFoundation allows us to store the code in Nucleus/ 
Flattener.h instead of distributed in yet another Nitrogen header  
(e.g. MacMemory.h), so I met myself in the middle and named the new  
template SeizingPODFlattener, indicating that aside from Seize-on- 
Get, the type is otherwise treated the same as POD (Plain Old Data).

Style questions:  Should we bring back the underscores, a la  
'Seizing_POD_Flattener'?  DescType_Traits uses Result/Parameter  
whereas Flattener uses Get_Result/Set_Parameter (which is more  
verbose but also more descriptive).  Should we settle on one, the  
other, or both?  Should empty classes be declared class, or struct,  
or does it depend?

Josh

Re: [Nitric-dev] Nitrogen source rearrangement

[Joshua J. <wan...@me...>]

On Aug 20, 2006, at 10:35 AM, Marshall Clow wrote:

> As time as moved on, and the Nitrogen project has evolved, I am
> finding that the arrangement of the sources in CVS is ... confused.
> Therefore, I suggest that the sources be reorganized in the following
> manner (and yes, I am volunteering to do the reorganizing):

Make it so!

My build tool (A-line) currently requires a directory A-line.confd at  
project level (meaning alongside Nitrogen, Nucleus, et al).  Any  
objections to my checking one in?

Josh

[Nitric-dev] Nitrogen source rearrangement

[Marshall C. <mar...@id...>]

As time as moved on, and the Nitrogen project has evolved, I am 
finding that the arrangement of the sources in CVS is ... confused. 
Therefore, I suggest that the sources be reorganized in the following 
manner (and yes, I am volunteering to do the reorganizing):

top level:
	Nucleus		-- basic building blocks
	Nitrogen		-- a wrapper library for CarbonLib; 
uses Nucleus
	POSeven		-- a wrapper library for POSIX (six,seven, 
get it?) using Nucleus
	ClassicToolbox	-- a wrapper library for the Classic Mac OS 
toolbox ; uses Nucleus
	Libs		-- a set of projects for building static 
libraries for the above libraries
	Samples		-- example programs
	Extras		-- Extra useful routines
		[ Extras/Nitrogen replaces "Nitrogen Extras" ]

Note that the "Nucleus" and "Nitrogen" folders are the same as before.

I'd also like to see a folder named "Extras/Nitrogen/UtilityRoutines" 
(or a similar name), where  people
can put utility routines that they have developed, each as a 
self-contained unit. HICommandHandlers is probably the best-known one 
of these, although there are others.

Things that I'm not sure about will stay in "Nitrogen Extras" for a 
while, and gradually migrate into the correct places.
-- 
-- Marshall

Marshall Clow     Idio Software   <mailto:mar...@id...>

It is by caffeine alone I set my mind in motion.
It is by the beans of Java that thoughts acquire speed,
the hands acquire shaking, the shaking becomes a warning.
It is by caffeine alone I set my mind in motion.

[Nitric-dev] Plan to reduce OnlyOnce<> globals and calls to RegisterFooBarErrors()

[Joshua J. <wan...@me...>]

In a conversation with Marshall, he expressed his dislike of the  
repeated occurrences of OnlyOnce< RegisterXXXErrors >(), and with his  
assistance I came up with this plan.

First, a recap of current behavior:

Let us consider a hypothetical FooBar Manager.  The Nitrogen/FooBar.h  
header declares a routine to register OSStatus result codes:

	void RegisterFooBarErrors();

All routines calling ThrowOSStatus() must first register the OSStatus  
result codes that could be thrown:

	// Defined in Nitrogen/FooBar.h
	inline void Foo()
	{
		NN::OnlyOnce< RegisterFooBarErrors >();

		ThrowOSStatus( ::Foo() );
	}

	void Bar();

	// Defined in Nitrogen/FooBar.cp
	void Bar()
	{
		NN::OnlyOnce< RegisterFooBarErrors >();

		ThrowOSStatus( ::Bar() );
	}

This statically allocates a bool for each such function defined in  
FooBar.cp, and each usage of each inline function defined in the  
header.  As a lesser concern, RegisterFooBarErrors() may be called up  
to as many times as there are OnlyOnce variables.

I would have liked to use statically constructed objects to register  
the OSStatus result codes, but I was shocked and disappointed to  
learn that such objects are not guaranteed to have been constructed  
until a function within the same compilation unit has been called,  
and therefore cannot be relied upon to have run before an inline  
function is called.

I suggest a hybrid approach.  We proceed with a static object:

	// In FooBar.h
	class ThingWhatRegistersFooBarErrors
	{
		ThingWhatRegistersFooBarErrors();

		public:
			static void Nudge();
	};

	// In FooBar.cp
	static ThingWhatRegistersFooBarErrors gRegistrar;

	ThingWhatRegistersFooBarErrors::ThingWhatRegistersFooBarErrors()
	{
		RegisterFooBarErrors();
	}

	ThingWhatRegistersFooBarErrors::Nudge()
	{
		// does nothing, but calling it guarantees construction of gRegistrar
	}

The functions are now defined as so:

	// Defined in Nitrogen/FooBar.h
	inline void Foo()
	{
		ThingWhatRegistersFooBarErrors::Nudge();

		ThrowOSStatus( ::Foo() );
	}

	// Defined in Nitrogen/FooBar.cp
	void Bar()
	{
		ThrowOSStatus( ::Bar() );
	}

The use of OnlyOnce is eliminated.  At worst, a flag is maintained by  
the C++ runtime per compilation unit to track whether static  
initialization has occurred, and if all units are initialized prior  
to calling main(), no state at all is needed.  RegisterFooBarErrors()  
is called exactly once (or possibly not at all).

Inline functions must still take care to ensure the errors are  
registered, but this is simply a function call.  Non-inlines needn't  
do anything; they get guaranteed error registration for free.

"Oh, and there is one more thing..."

	// In FooBar.h

	inline void ThrowOSStatus_FooBar( ::OSStatus err )
	{
		if ( err != noErr )
		{
			ThingWhatRegistersFooBarErrors::Nudge();
	
			ThrowOSStatus( err );
		}
	}

	inline void Foo()
	{
		ThrowOSStatus_FooBar( ::Foo() );
	}

The non-error code path is completely inlined, with no function calls.

Josh

[Nitric-dev] Need a new Nitrogen metaphor

[Marshall C. <mar...@id...>]

I've been poking around with some Unicode stuff, and I'm looking at 
the UnicodeConverter
stuff I did in Nitrogen a while back, and I've about decided that 
it's just not right.

The basic problem is that of "partial completion".  The underlying 
Unicode converter calls
can process part of the input, and then return an error that says 
"Need more input", or
"yeah, it worked, but it could have been better"

Consider ConvertFromUnicodeToText :

Converts a Unicode text string to the destination encoding you specify.

OSStatus ConvertFromUnicodeToText (
    UnicodeToTextInfo iUnicodeToTextInfo,
    ByteCount iUnicodeLen,
    const UniChar iUnicodeStr[],
    OptionBits iControlFlags,
    ItemCount iOffsetCount,
    ByteOffset iOffsetArray[],
    ItemCount * oOffsetCount,
    ByteOffset oOffsetArray[],
    ByteCount iOutputBufLen,
    ByteCount * oInputRead,
    ByteCount * oOutputLen,
    LogicalAddress oOutputStr
);

It can return the following errors (among others)

kTECArrayFullErr
	The supplied TextEncodingRun, ScriptCodeRun, or 
UnicodeMapping array is too small, and the input was not completely 
converted. Call the function again with another output buffer-or with 
the same output buffer after copying its contents-to convert the 
remainder of the string

kTECPartialCharErr
	The input text ends in the middle of a multibyte character 
and conversion stopped. Append the unconverted input from this call 
to the beginning of the subsequent input text and call the function 
again.

kTECUsedFallbacksStatus
	The function has completely converted the input string to the 
specified target using one or more fallbacks. For the Unicode 
Converter, this status code can only occur if the 
kUnicodeUseFallbacksBit control flag is set.

kTECOutputBufferFullStatus
	The converter successfully converted part of the input text, 
but the output buffer was not large enough to accommodate the entire 
input text after conversion. Convert the remaining text beginning 
from the position where the conversion stopped.

If Nitrogen were to throw exceptions for these errors, then the 
return value for the function (how many input characters were 
converted and how many characters were output) could not be returned, 
and would be lost.

It seems to me that we need to return some kind of success/failure code,
as well as the result counts. However, we probably also want to throw
an exception in the cases where a complete failure has occurred.

Ideas?

[ The only comparable example I can think of is reading from a file - 
where the read may be cut short
by an EOF. However, that's just one specific error, not several. ]
-- 
-- Marshall

Marshall Clow     Idio Software   <mailto:mar...@id...>

It is by caffeine alone I set my mind in motion.
It is by the beans of Java that thoughts acquire speed,
the hands acquire shaking, the shaking becomes a warning.
It is by caffeine alone I set my mind in motion.

Re: [Nitric-dev] Attribution and public domain notice

[Joshua J. <wan...@me...>]

On May 23, 2006, at 8:38 AM, Lisa Lippincott wrote:

> I approve that wording.  I even checked the start date; I'm pretty
> sure I didn't start coding Nitrogen until 2002.

Excellent.  Thank you both for your quick responses.

Here's my proposed format.  The header includes the minimal  
information we use to maintain the project (file name and maintainer)  
followed by the legal notice of public domain status.

// Nitrogen/QuickDraw.cp
// ---------------------
//
// Maintained by Joshua Juran

// Part of the Nitrogen project.
//
// Written 2002-2006 by Lisa Lippincott, Marshall Clow, and Joshua  
Juran.
//
// This code was written entirely by the above contributors, who  
place it
// in the public domain.

Josh

Re: [Nitric-dev] Attribution and public domain notice

[Lisa L. <li...@ha...>]

>> Nitrogen/MacWindows.h
>> Part of the Nitrogen project <http://nitric.sourceforge.net/>
>>
>> Written 2002-2006 by Lisa Lippincott, Marshall Clow, and Joshua  
>> Juran.
>>
>> This code was written entirely by the above contributors, who  
>> place it in the public domain.
>
> Please let me know on-list that you approve of this notice, or what  
> changes you require.  I'll proceed once I have your approval.

I approve that wording.  I even checked the start date; I'm pretty
sure I didn't start coding Nitrogen until 2002.

                                                  --Lisa
  

Re: [Nitric-dev] Attribution and public domain notice

[Marshall C. <mar...@id...>]

At 12:24 AM -0700 5/20/06, Joshua Juran wrote:
>On Apr 26, 2006, at 10:02 AM, Lisa Lippincott wrote:
>
>>I think this is a good idea, too.  Thanks for taking the lead
>>on it.  And I agree that it's best to move any non-public-domain
>>code outside of Nucleus and Nitrogen, if possible.
>
>I appreciate your support on this.  Quoting myself:
>
>>I'm willing to do the work of adding such notice.  Barring any 
>>dissent, all we need to agree on are wording and a header format.
>
>Obviously, nobody disagreed with the plan to choose wording and 
>format, and it looks like I'm basically on my own with this.  
>However, I would feel very uncomfortable adding any notice of rights 
>reserved or waived without the explicit permission of those 
>involved.  Here again is the proposed text:
>
>>Nitrogen/MacWindows.h
>>Part of the Nitrogen project <http://nitric.sourceforge.net/>
>>
>>Written 2002-2006 by Lisa Lippincott, Marshall Clow, and Joshua Juran.
>>
>>This code was written entirely by the above contributors, who place 
>>it in the public domain.
>
>Please let me know on-list that you approve of this notice, or what 
>changes you require.  I'll proceed once I have your approval.

Works for me.
-- 
-- Marshall

Marshall Clow     Idio Software   <mailto:mar...@id...>

It is by caffeine alone I set my mind in motion.
It is by the beans of Java that thoughts acquire speed,
the hands acquire shaking, the shaking becomes a warning.
It is by caffeine alone I set my mind in motion.

Re: [Nitric-dev] Attribution and public domain notice

[Joshua J. <wan...@me...>]

On Apr 26, 2006, at 10:02 AM, Lisa Lippincott wrote:

> I think this is a good idea, too.  Thanks for taking the lead
> on it.  And I agree that it's best to move any non-public-domain
> code outside of Nucleus and Nitrogen, if possible.

I appreciate your support on this.  Quoting myself:

> I'm willing to do the work of adding such notice.  Barring any  
> dissent, all we need to agree on are wording and a header format.

Obviously, nobody disagreed with the plan to choose wording and  
format, and it looks like I'm basically on my own with this.   
However, I would feel very uncomfortable adding any notice of rights  
reserved or waived without the explicit permission of those  
involved.  Here again is the proposed text:

> Nitrogen/MacWindows.h
> Part of the Nitrogen project <http://nitric.sourceforge.net/>
>
> Written 2002-2006 by Lisa Lippincott, Marshall Clow, and Joshua Juran.
>
> This code was written entirely by the above contributors, who place  
> it in the public domain.

Please let me know on-list that you approve of this notice, or what  
changes you require.  I'll proceed once I have your approval.

Josh

Re: [Nitric-dev] Attribution and public domain notice

[Lisa L. <li...@ha...>]

I think this is a good idea, too.  Thanks for taking the lead
on it.  And I agree that it's best to move any non-public-domain
code outside of Nucleus and Nitrogen, if possible.

                                                  --Lisa

[Nitric-dev] Attribution and public domain notice

[Joshua J. <wan...@me...>]

I'm of the opinion that every file in Nitrogen should contain a  
notice crediting the authors and indicating its public domain status.

I'm willing to do the work of adding such notice.  Barring any  
dissent, all we need to agree on are wording and a header format.

Determining a file's contributors will be a trivial matter of  
recognizing their various indentation styles.  :-)

There are several rare but nonetheless occurring instances of code  
checked in to Nitrogen that was not written by Nitrogen developers.   
For each case, we need to either verify the code as in the public  
domain, mark it clearly as an exception and supply copyright  
information, or remove it.  One example of this is the code in  
MacTypes.cp to support UniString in gcc < 4; another is the non- 
Carbon implementation of CheckEventQueueForUserCancel() in Events.cp  
(which will get moved to Carbon Units).  My preference is that  
Nucleus and Nitrogen proper be entirely public domain.  Nitrogen  
Extras can't be entirely public domain as long as it contains (for  
example) Apple-supplied sample code.

Here's my first stab at wording:

> Nitrogen/MacWindows.h
> Part of the Nitrogen project <http://nitric.sourceforge.net/>
>
> Written 2002-2006 by Lisa Lippincott, Marshall Clow, and Joshua Juran.
>
> This code was written entirely by the above contributors, who place  
> it in the public domain.

I'd also like to indicate who the file's maintainer is (assuming that  
we assign a maintainer to each file).

Josh

Re: [Nitric-dev] Backward-incompatible merge in progress

[Joshua J. <wan...@me...>]

On Feb 7, 2006, at 10:00 AM, Joshua Juran wrote:

> I'm in the process of merging Lisa and Marshall's work splitting 
> Nitrogen into two parts, Nitrogen and Nucleus.  Nucleus is already 
> checked in, and soon I intend to commit the new Nitrogen files as 
> well.

This has now happened.

There were some issues specific to pre-Carbon.  Notably, 
Nitrogen::WindowRef is defined in Nitrogen/QuickDraw.h (as an alias to 
Nitrogen::WindowPtr, which gets special treatment under non-Carbon) and 
therefore definition in Nitrogen/MacWindows.h (as an alias of 
::WindowRef) is redundant and incorrect.  Other issues occurred in 
non-Carbon-only conditional blocks which naturally didn't show up in 
Carbon-targeted testing. These were all easily fixed.

I had to make some revisions regarding overloaded operators.  Operators 
accepting globally-scoped types as parameters (e.g. FSRef) must be 
defined in namespace Nucleus, not Nitrogen.  An operator accepting 
Nitrogen::Foo (where it's distinct from ::Foo) is of couse defined in 
namespace Nitrogen, but one that operates on Nucleus::Owned< 
Nitrogen::Foo > must be defined in namespace Nucleus.  Occasionally it 
may be necessary to write "using Nucleus::operator==;" or the like in 
Nitrogen code (and not within a function body or gcc will complain).

I've moved what I'll call the 'luxury' operators in 
Nitrogen/AEDataModel.h (e.g. operator<<( AEDescList&, const AEDesc& ), 
as opposed to essential operators like ==) into Operators/AEDataModel.h 
in Nitrogen Extras, and intend to follow suit with other modules, such 
as QuickDraw.

> This will break almost all client code, including Nitrogen Extras and 
> end user code.  I'll fix the parts that I developed (e.g. 
> ClassicToolbox).

The parts of Nitrogen Extras that I developed are mostly if not 
entirely working (they compile at least), but it's impossible to know 
without tests.  Take a look in Nitrogen/Sound.h (namespace Tests) for 
an example of what I'm talking about -- active sample code in the 
header ensures that the definitions are, well, sound.  (Not intended, I 
swear!)  :-)

Josh

[Nitric-dev] Backward-incompatible merge in progress

[Joshua J. <wan...@me...>]

I'm in the process of merging Lisa and Marshall's work splitting 
Nitrogen into two parts, Nitrogen and Nucleus.  Nucleus is already 
checked in, and soon I intend to commit the new Nitrogen files as well.

This will break almost all client code, including Nitrogen Extras and 
end user code.  I'll fix the parts that I developed (e.g. 
ClassicToolbox).

If there are any objections to this planned maneuver, now would be a 
really good time to voice them.

Josh

[Nitric-dev] Upcoming developments in Nitrogen

[Marshall C. <mar...@id...>]

Lisa and I worked a bunch on Nitrogen this last week at WWDC, and we have
accomplished two things:

1) Splitting of Nitrogen into two parts; the first, called "Nucleus", 
is full of
the building blocks of Nitrogen, and contains no Carbon-specific 
code. The second,
called "Nitrogen", uses Nucleus to provide the wrappers on the Carbon API.

2) This is now building using XCode 2.1.

3) The elimination of "Pending" ;-)

I will be updating the CVS with these new files this week.
-- 
-- Marshall

Marshall Clow     Idio Software   <mailto:mar...@id...>

It is by caffeine alone I set my mind in motion.
It is by the beans of Java that thoughts acquire speed,
the hands acquire shaking, the shaking becomes a warning.
It is by caffeine alone I set my mind in motion.

[Nitric-dev] Nitrogen Web site

[Joshua J. <wan...@me...>]

I've started following the Mac programming newgroups on Usenet.  
Nitrogen has been mentioned a few times, but one person noted that we 
haven't released any files and gave up.  I think it would be useful to 
have more information about Nitrogen on our Web site.

I'd like us to have a versatile index page that links to various 
resources.  (The current page is an overview that refers only to the 
Nitrogen Manifesto, not even to the SourceForge project page.)  Also, 
we should probably store the Web site in CVS, so as to avoid 
overwriting accidents.

Incidentally, I've just learned PHP.

Josh

-- 
Joshua Juran
Metamage Software Creations - Mac Software and Consulting
http://www.metamage.com/

                * Creation at the highest state of the art *