Re: [Crm114-discuss] CRM binary file format & (future?) portability

[Gerrit E.G. H. <Ger...@be...>]

Paolo wrote:
>> Brilliant idea! Hadn't thought about the 'head -x', but I _like_ it. I 
>> was thinking about maybe 1 or 2 Kbytes reserved for the header anyway, 
>>     
> these are ideas that floated in ML threads long ago. Note that OSBF makes
> room for 4k header.
>   
Yes, I saw there was some version checking and header code in there already.

BTW, 'man head' on my box doesn't give a -x option. Is that an option to 
read until the EOF (or NUL?) character in an ASCII file?

> ok, ok - no b2b ;)
>   
Sorry, recalled some 'cool hacking' sessions of long past that went pear 
shaped as nobody could'handle' it. With 20-20 hindsight it was an 
exercise in  complexity capability (how much nasty little details can 
you handle all at once).

> no, if you put the classes in 2 sets like
> !      classify (classA_1 classA_2 ... | classB_1 classB_2 ...)
> you get a scalar (success|fail, but still all pR values). If you insted
> say
> !      classify (classA_1 classA_2 ... classB_1 classB_2 ...)
> (note no '|') ie run in 'stats-only' - you get just the pR vector. I think
> you can use that for building your fuzzifier, either in CRM or your favourite
> prog.lang. A tricky point is that pR is normalized, so that it cannot be
> used as class-membership function as is; an artifice could be to add a 
> class 'AnythingElse', ie the complement to the set of your classes.
>   
I've copied this to my project notes. At the moment, the details of this 
are beyond my grasp, but that will change when I move away from the code 
cleanup into the actual algorithmic material of crm114.
Thank you for this tip for it gives me a direction to investigate.
> note that not all classifiers work well for N >2, nor those that are 
> *supposed* to work have been thoroughly tested.
>   
I already suspected that much. That's why I don't mind going through all 
the code: I expect I'll need this exercise later on.

> well, crm114 is a jit engine + classifiers plugged-in (bolted-in, at present).
> [...]
> I think that, if none of the (pR output from) current classifiers fits your
> task, it'd be relatively easy to hack one of them into a new one, which 
> would be named eg f-osb (Fuzzy-OSB) or even OSBG (OSB-Gerrit) ;).
>   
:-)  heh, OSBG, now that would be something.
Seriously though, I immediately recognized the plug/bolting in features 
when I first had a look at the crm114 code.
Of course, a bit less of a copy&paste approach would have been 'nice' 
from a certain design point of view, but given the research nature of 
this type of tool (as Bill put it so eloquently somewhere: 'spam is a 
moving target') copy&paste is a very good approach (you can always 
refactor the sections that have stabilized).
Besides, there are very nice tools out there to ease diff&merge-ing 
source files, so it's not much of a hassle to keep them in sync for now 
(like I did with my copy of SVM vs SKS: SKS seems to have started as an 
utterly stripped version of SVM, but the behaviour is _very_ similar so 
I merged the SVM code back in, just so I have lesser diffs to look at 
when cross-checking SKS vs SVM after a code change in either one of them.

Ger