rak-developers — Development efforts for Rakiura Modules Management System.

[rak-developers] Examples for rak testing

[Mariusz N. <ma...@ma...>]

It would be good to have sowe dummy jars to play with - and of course it 
would be good to do some testing before the rak release.

Look at:
   http://nzdis.otago.ac.nz/test/modules/

there is a bunch of empty rak compliant jars, with the following 
structure  (where "--" means "requires"):

A.jar  (A.1.0.0)

B.jar  (B.1.0.0)

B2.jar (B.2.0.1) -- A.1

C.jar  (C.1.0.0) -- B.1

D.jar  (D.1.0.0) -- C.1

D2.jar (D.2.0.0) -- C.1, B2

E.jar  (E.3.0.0) -- D.2



You can kickstart it by simply saying for example:

java -jar rakiura-rak.jar -i http://nzdis.otago.ac.nz/test/modules/E.jar

which will by default install E with all the dependencies.

All combinations and other possible scenarios for testing welcome.
I have noticed that there is currently (at least) one bug: if you try to 
install B.2.0.1 with existing B.1.0.0, rak will try to remove B.1.0.0 
first, but will fail if there is anything dependent on B.1.
I will fix it now.

There are three other bugs/problems with the annotator gui -
1. the dependencies cannot be removed (will fix it in a minute)
2. the dependencies download urls are not removable (do not know how to 
fix this)
3. and urls are not editable - will fix it now.

[rak-developers] Re: [jfern-users] Add a token to an executing petri net

[Mariusz N. <ma...@ma...>]

Hi Jason,

Thanks again for taking your time and formulating queries and 
sugestions. I will try to respond here to some of the key issues.

> After a bit more debugging it seems to do it for even the most
> simple nets. Two inputs and one output arcs result in 2 tokens going
> into the next transition. Most times though you never see the second
> token but it is in there. Maybe its a feature :) When should a cpn
> move two tokens from the input arcs to two tokens on the one output
> arc?



Ok. This seems to be a feature indeed ;o)

Lets consider a Petri Net with two input places A, B, one transition and 
one output place (your case). Assume now that there is a token 'a' in 
place A and token 'b' in place B. What should be the default behaviour 
of the transition? To propagate token 'a'? or 'b'? or both? or maybe 
place complately new token 'c' in the output place? Well, we do not know 
- it is really up to the particular design. But still, there must be 
some default behawior:
1) propagate token 'a'
2) propagate token 'b'
3) place new token 'c'
4) propagate both 'a' and 'b'

1 and 2 are symmetrical - there is now basis to favour one token over 
the other.
3 is awkward - there is no basis as to what kind of token should be created.

we are left with case 4.  In a more general case of n input arcs the 
default behaviour is to propagate all of the tokens.

How to control the behaviour? Easy - this is where the OutputArc API 
comes in. One simply need to change the default OutputArc.Expression 
from the default one to the requied custom one.

The default one is:

      Expression expr = new Expression(){
       public Multiset expression(Multisets multisets){
         java.util.Iterator i = multisets.iterator();
         Multiset resultset = new BasicMultiset();
         while(i.hasNext()){
           resultset.add((Multiset)i.next());
         }
         return resultset;
       }
     };



And to change it for example to behaviour 1 or 2 one would write:

oa = get the desired OutputArc reference
oa.setExpression(
      new Expression(){
       public Multiset expression(Multisets multisets){
         // just pick only the first token from the first
         // multiset and return it
         Multiset resultset = new BasicMultiset();
         Multiset input = multisets.toArray()[0];
         resultset.add(input.toArray()[0]);

         return resultset;
       }
     }
);



And to change it for example to behaviour 3 one would write:

oa = get the desired OutputArc reference
oa.setExpression(
      new Expression(){
       public Multiset expression(Multisets multisets){
         // ignore all current tokens
         // and return newly created token
         Multiset resultset = new BasicMultiset();
         resultset.add(new ObjectToken("c"));

         return resultset;
       }
     }
);


Hope you understand it now. I will patch the documentation to 
explicitely describe that case. Also, if you have better/nicer idea how 
to deal with the default behaviour please let me know.


 
> Also, I'd like the system we build to be "on top" of JFern. ie., make
> as few changes to JFern as possible. But there are a few things
> that might need to be changed in JFern for us to implement in J2EE.
> For example, building a workflow engine means extending the Simulator
> object. However, the Simulator is threaded and you can't have
> threaded objects in J2EE. At present the Simulator doesn't do anything
> with its threaded capabilities so its easy for us to remove
> the runnable interface in Simulator.java. But that means our
> version of JFern is different to your constantly improving one. Would
> you like to know about these things so you can produce, for example,
> a threaded and non-threaded version of JFern?


Very good point.  I will refactor the current Simulator API and I will 
clearly separate the non-threaded simulator (which is the one actually 
implemented) and the threaded simulator (which is not yet finished). The 
new non-threaded API will not use the Thread class nor the Runable 
interface.



> We've also been looking at hierarchic PNs where a transition at one
> level is just an abstraction on another PN at the next level
> down. It doesn't look like this is built into JFern (unless I missed
> it). Are you thinking about it?


there are two possible ways of looking at it
1) Hierarchies are just an abstraction, and the actual implementation is 
flat (the net where the transtion is just another subnet is equivalent 
to the net where the actual subnet is "inlined" into the main net).
2) Hierarchies are not just abstractions, they are in fact implemented 
in a hierarchical manner.

JFern is perfectly suited for 1), however 2) is not yet supported. We 
are thinking of supporting it because it would enhance the modularity 
and subnet reuse in our agent systems, but as I said, it is not 
implemented yet, and  I suspect setting up visualization is our first 
priority.
However, the point is noted and it may change our schedule a little (we 
may throw more resource to actually support it sooner, as we do in fact 
need that feature as well).



> Finally, we'll need some graphical viewer capabilties soon. Its says
> in the documentation that you might have something done by Spring 2002
> (I guess that means NZ spring right :). So, we'll have to knock
> something together to get us going but I'd like to make sure what we 

> put together is similar to what you are planning. Have you started on 
> the PN viewer and are you interested in collaborating on it?

We have sort of started and YES YES YES for collaboration! We have been 
evaluating the graph drawing packages which could be suitable to be used 
by JFern, and we have looked at JHotDraw which is the package Renew is 
using for its visualization.  We also have been thinking which API model 
for JFern would be best to make the visualization nice and modular (we 
are thinking of API which would allow different pluggable visualization 
modules). But it is all in thinking/design stage as of today.  Also, 
because JHotDraw is not anymore supported as such, and, because it was 
AWT and Java 1.1 only, we have played with it a little, and we have put 
together an LGPLed visualization package called JRio which is originally 
based on JHotDraw API. It is almost ready, but it is not released and 
not documented yet. This new, smaller version of JHotDraw was our first 
bet as to what kind of drawing package to use for JFern visualization. 
If you are interested, I may clean it a little and put a non-official 
release for you to look at it.
The voices were divided in our team though, because some people think a 
completely new from scratch small Swing based visualization 
implementation would be best for minimalistic embeded runtime 
visualisation.

We definitelly need visualization, and we will definitely push it 
forward as one of the main priorities for JFern.


best regards
Mariusz

[rak-developers] Things to be done

[Mariusz N. <ma...@ma...>]

Hi,

ModuleInfo:
we had agreement about the XML ModuleInfo file format so lets make it 
into "stable" mode again. I will adjust documentation and implementation 
to reflect the latest decisions.


Dual naming and versioning:
We all agree that the software does not need to use the RAK name and 
version for marketing purposes. The RAK name (which suppose to be long) 
and CRA versioning are just internal for dynamic dependency tracking 
etc. Right?
For some libraries and applications it may make sense just to use single 
RAK compliant name and CRA version, for some not.

RAK does not really care about the "marketing" names and version 
numbers. However, the RAK GUI may need to display this kind of 
"end-user" names and versions. Should we include them explicitelly into 
ModuleInfo? I think not. First, it is up to the application or library 
to decide if it is necessary or not to have dual naming/versioning. 
Second, it would be yet another name/version (Manifest 
specification-name/version, implementation-name/version, ModuleInfo 
name/version). But, in the wee paragraph about this possibility I have 
to write something about this dual naming/versioning. So, what should I 
write in the documentation, and, how RAK GUI will display these 
marketing names/versions?  Via summary? or via long name?

<longname>JUnit Testing Framework v.3.7.0</longname>



Comparison with other java module managers:
Etienne is right that we should write some kind of comparison to the 
existing module managers and dependency tracking for Java. What we 
should include? Why deb and RPM are not suitable for Java? Should we 
spend more time talking why Java specification/implementation versions 
are not a good solution? Should we talk about JNLP? Can someone give me 
hand with this bit of the documentation?



Bootstrapping:
Before the release we also need to sort out the bootstrapping problem: 
how to install the very first RAK without having RAK yet? Once the RAK 
is installed, new versions of RAK are installed simply via RAK itself (*).

I can think of two models: one based on the zipped rak jar together with 
  the configurator, and a wee script which starts up the configurator 
and sets the properties. The user needs to put a jar in a proper place 
then. This is the current model I am using here. Also, it relates to the 
(*), because every time I am installing new RAK via RAK i have to make 
by hand one symbolic link, to let the "rak" script to fetch the latest 
RAK jar. It works quite well on unix, but it is not nice, and it will 
not work on Windows.

So, the second model is to have a small booting rak implementation, 
which could be used to set up properties and fetch RAK from the net. 
That would be the initial bootstrapping install. This booting rak would 
also upon invokation check the module registry and fetch the latest RAK 
jar and run it.  This solution is nicer, for the user, and would work on 
any Java system (no symlink, even no scripts necessary, apart for this 
"rak" convenience script). However, this means a little more work for 
me, as this booting rak needs to know enough to look up the module 
registry (or maybe just lookup the jar directory?), but not too much, so 
it would be applicable without upgrades to all RAK versions. Quite 
tricky, huh? Because future RAK can have a different moduleInfo format, 
the booting rak my not know it... maybe RAK will have a different 
moduleRegistry implementation, booting rak should not mess up with it 
either.  So?   Simple filename based convention and lookup? Sounds like 
hack to me, but, well, this is the only possible safe booting rak... 
what do you think? Any suggestions?



best regards
Mariusz

Re: [rak-developers] ModuleInfo about to freeze

[Etienne M. G. <eti...@uq...>]

Mariusz Nowostawski wrote:

> 
> As I do not really like this year-month XMLish versioning, so what about 
>  simply:
> 
> <moduleinfo xmlns="http://rak.sf.net/1.0/rak.xsd"
>     group="Development/Tools"
>     ...


Looks fine to me.

>...

> <moduleinfo xmlns="http://rak.sf.net/1.0/rak.xsd"
>      name="org-junit-junit"


;o)

Cheers,


Etienne

-- 
Etienne M. Gagnon                    http://www.info.uqam.ca/~egagnon/
SableVM:                                       http://www.sablevm.org/
SableCC:                                       http://www.sablecc.org/

Re: [rak-developers] ModuleInfo about to freeze

[Mariusz N. <ma...@ma...>]

Thanks Etienne and Kyle for your suggestions. I think we will have to 
iterate one more time.


[ModuleInfo XML version]
Etienne wrote:

> My only suggestion is to make this LESS important. This way, it will be 
> easy to make even backward incompatible format changes. Later RAK 
> versions will simply need to understand older formats. Here's how: you 
> could start the MODULEINFO file with a "rak-version" tag, something like:
> 
> <rak-version 1.0>  <!-- MANDATORY; public version, NOT C.R.A -->
> <moduleinfo ...



What if we use the namespace as a version "keeper"? as Kyle suggested, 
for example:

<moduleinfo xmlns="http://rak.sf.net/2001/12/rak.xsd"
     group="Development/Tools" 

      ...

As I do not really like this year-month XMLish versioning, so what about 
  simply:

<moduleinfo xmlns="http://rak.sf.net/1.0/rak.xsd"
     group="Development/Tools" 

     ...


?


Ok, so, here it goes, a newer proposal, please give your feedback again. 
If set, I will "freeze" it "officially", and Kyle will prepare a Schema 
for it. Kyle, you have convinced me with the consistency argument, so I 
have changed the URLs and Download handling a little. Tell me all what 
you think.


<moduleinfo xmlns="http://rak.sf.net/1.0/rak.xsd"
      name="org-junit-junit"
      group="Development/Tools"
      version="3.7.0"
      on-install="install-script"
      on-uninstall="unistall-script" >

   <metadata>
      <vendor href="http://junit.com">JUnit Team</vendor>
      <longname>JUnit Testing Framework</longname>
      <summary>This is a great Java testing framework, design for bla
               bla bla</summary>
      <copyright>GNU Public License</copyright>
      <packager href="emailto:bla@com">whoever</packager>
   </metadata>

    <dependencies jvm-version="1.2+">
       <dependency name="log4j.2" classpath="yes" unless-jvm-version="1.4+">
           <download>
             <rak-url href="http://downloads.sf.net/log4j.rak" />
             <jar-url href="http://downloads.sf.net/log4j.jar" />
           </download>
       </dependency>
       <dependency name="log4j.2" classpath="yes" 
unless-available="java.util.logging.Logger">
           <download>
             <rak-url href="http://downloads.sf.net/log4j.rak" />
             <jar-url href="http://downloads.sf.net/log4j.jar" />
           </download>
       </dependency>
    </dependencies>

    <executables default="junit">
       <executable class="org.junit.Main">
           <alias name="junit" />
           <alias name="main" />
       </executable>
       <executable class="org.junit.Install">
           <alias name="install-script" />
       </executable>
       <executable class="org.junit.Uninstall">
           <alias name="uninstall-script" />
       </executable>
    </executables>

    <urls>
       <rak-url href="http://prdownloads.sourceforge.net/junit/junit.rak" />
       <jar-url 
href="http://prdownloads.sourceforge.net/junit/junit3.7.jar" />
       <jar-url href="http://alternativelocation.net/junit/junit.jar" />
    </urls>

</moduleinfo>


Once decided I will change the ModuleInfo.java to reflect the decided 
XML format.


best regards
Mariusz


ps. I was gone fishing for two days, unfortunatelly it ended up only in 
two little trouts. Which is not too bad considering that I have bought 
my first ever "fishing pass" three days ago ;o) I had a great time in 
the middle of nowhere alone waking up at 04:30 ;o) Hovewer, the best 
luck I had the previous day about 21:00, which was somehow surprising to me.

Re: [rak-developers] ModuleInfo about to freeze

[Etienne M. G. <eti...@uq...>]

Mariusz Nowostawski wrote:

> Hi,
> 
> I am about to freeze the ModuleInfo XML file format. Please please 
> please HAVE a look and tell me if you have any better names or some 
> suggestions. This is VERY important.



My only suggestion is to make this LESS important. This way, it will be easy to 
make even backward incompatible format changes. Later RAK versions will simply 
need to understand older formats. Here's how: you could start the MODULEINFO 
file with a "rak-version" tag, something like:

<rak-version 1.0>  <!-- MANDATORY; public version, NOT C.R.A -->
<moduleinfo ...


> ...
> <moduleinfo
>      name="junit"


I thought we were recommending multi-part names...
Why not use our recommended names in our examples, something like:

<rak-version 1.0>
<moduleinfo
      name="org-junit-junit"
      group="Development/Tools"
      version="3.7.0"
...

Etienne


-- 
Etienne M. Gagnon                    http://www.info.uqam.ca/~egagnon/
SableVM:                                       http://www.sablevm.org/
SableCC:                                       http://www.sablecc.org/

[rak-developers] ModuleInfo about to freeze

[Mariusz N. <ma...@ma...>]

Hi,

I am about to freeze the ModuleInfo XML file format. Please please 
please HAVE a look and tell me if you have any better names or some 
suggestions. This is VERY important.
Once the ModuleInfo format is public, it will be EXTREMELY difficult to 
have any changes in it (additions are fine, removals and changes are 
simply not possible! look at ANT project and their problems with 
removing anything from the XML build file!).

My own personal suggestion is to rename <requires> into <required>. It 
just sounds better to me (verb sounds quite not rights as an element 
name, it id not quite explicit who requires what).
Any other suggestions?

Also, can any native speaker tell me the difference between dependency 
and dependancy? My dictionary is not of much help, apart that the former 
seems to be more "normal".


best regards
Mariusz



This is how it looks right now:

moduleinfo:name,group,version are required, rest is all optional, and 
recursivelly optional (eg. packager:href is optional within packager tag 
itself).



<moduleinfo
      name="junit"
      group="Development/Tools"
      version="3.7.0"
      on-install="install-script"
      on-uninstall="unistall-script" >

    <vendor href="http://junit.com">JUnit Team</vendor>
    <longname>JUnit Testing Framework</longname>
    <summary>Hey, this is a great Java testing tool!</summary>
    <copyright>GNU Public License</copyright>
    <packager href="emailto:bla@com">whoever</packager>


    <dependencies>

       <requires name="log4j.2" classpath="yes" unless-jvm-version="1.4+">
           <download href="http://downloads.sf.net/log4j.rak" />
           <download href="http://downloads.sf.net/log4j.jar" />
       </requires>
       <requires name="log4j.2" classpath="yes" 
unless-available="java.util.logging.Logger">
           <download href="http://downloads.sf.net/log4j.rak" />
           <download href="http://downloads.sf.net/log4j.jar" />
       </requires>


    </dependencies>

    <executable default="junit">
       <class name="org.junit.Main">
           <alias name="junit" />
           <alias name="main" />
       </class>
       <class name="org.junit.Install">
           <alias name="install-script" />
       </class>
       <class name="org.junit.Uninstall">
           <alias name="uninstall-script" />
       </class>
    </executable>

    <jar-url href="http://prdownloads.sourceforge.net/junit/junit3.7.jar" />
    <rak-url href="http://prdownloads.sourceforge.net/junit/junit.rak" />
    <jar-url href="http://alternativelocation.net/junit/junit.jar" />

</moduleinfo>

Re: [rak-developers] Launcher and class loaders

[Etienne M. G. <eti...@uq...>]

Mariusz Nowostawski wrote:

> Hi Kyle.
> 
> 
>> Yes, esp. on UNIX systems, that may not even be possible due to file
>> permissions. However, I do think both JDK version and standard extensions
>> need to be in scope for Rak.
> 
> 
> 
> Agreed. As you said, direct manipulation of jre/lib/ext is not really an 
> option (even on my own machine I need to be root to mess around this 
> directory ;o)  Leaving it up to the user is not nice at all either (some 
> users may even do not now what CLASSPATH is, not to mention messing with 
> standard extensions).  There must be somehow a way for RAK to deal with it.
> 


Ugly trick....  Why not have a standard extention directory in the user's home, 
by adding this directory to the bootclasspath in the "rak" script. E.g.:


In /usr/bin/rak:
#!/usr/sh
java -bootclasspath ...:$HOME/.java/lib -jar .../rak.jar


As I said: ugly...


>> There's another issue along these lines, unfortunately. Custom URL
>> implementations will fail, because Java insists on loading them from the
>> system CLASSPATH only. I assume this is a security feature.
>>



Do you think a better solution to all these problems could be found by writing 
our own security manager(s), instead of trying to circumvent the JDK default 
behavior?

Etienne



-- 
Etienne M. Gagnon                    http://www.info.uqam.ca/~egagnon/
SableVM:                                       http://www.sablevm.org/
SableCC:                                       http://www.sablecc.org/

Re: [rak-developers] Launcher and class loaders

[Mariusz N. <ma...@ma...>]

Thanks Kyle,

[...]

> I think it's a good solution.
> 
> For the XML, maybe something like
> 
> <requires unlessAvailable="java.util.regex.Pattern">


Ok, I will do it as: unless-available="class name here"


> would be a little less verbose. But the bigger issue, in my opinion, is 
> that those class checks are touchy. For instance, JAXP can be in the 
> CLASSPATH either because
> 
> a) it's installed as a standard extension
> b) the user's running JDK 1.4
> 
> And which version? For instance, the newest JAXP comes with a particular 
> version of Xalan bundled. If I want that overridden, I need it in my 
> bootclasspath, right? So even that bootclasspath option may be 
> conditional on JDK version and what standard extensions are installed.


Ok. I think, we could extend the "unless" attribute to cover also JDK 
versions, so, for example one could do:

<dependencies jvm-version="1.2.2+">

this is to notify RAK that the minimal JVM version is 1.2.2, and all 
attempts to install a given module in lower JVM version should issue a 
warning/error message. (of course specifying jvm version lower than 
1.2.1 is pointless, as RAK itself requires 1.2+ ;o)


<requires name="log4j.2" bootclasspath="yes" unless-jvm-version="1.4+">
   <download href="http://downloads.sf.net/log4j.jar" />
</requires>

this is saying that, if the JVM is lower then 1.4, a given module needs 
to be installed and used inside bootclasspath.


<requires name="log4j.2" unless-available="java.util.logging.Logger">
  <download href="http://downloads.sf.net/log4j.jar" />
</requires>

same as above but this time instead of testing JVM version we are doing 
simple class check and we do not require the module to be inside 
bootclasspath.


I am not sure if the combination of the both above is necessary? 
Maybe... if so we may allow both "unless" condition being present at the 
same time in single requires. Also, maybe the test needs to consider 
more than single class? Maybe... we could allow comma-separated 
classnames then.

what you think?
If you can think of anything better, it is time to speak ;o)


Ah, also, because we will spawn new JVM, we need to allow somehow the 
user to pass the JVM args into the new JVM. Before the user could do it 
simply when invoking RAK itself, not it will not work ;o(
Any ideas here? We could add for example jvm-args="-Xmsblabla" attribute 
  to the class inside executable element. But this is half of the 
solution. The user invoking the given application should be able to set 
his own preferred JVM args.  We could do it like:

rak [--start] --jvm-args "-Xblabla -Xblabla" modulename [application 
[application arguments]]

but it looks ugly... any ideas?



best regards
Mariusz

Re: [rak-developers] Launcher and class loaders

[Mariusz N. <ma...@ma...>]

Hi Kyle.


> Yes, esp. on UNIX systems, that may not even be possible due to file
> permissions. However, I do think both JDK version and standard extensions
> need to be in scope for Rak.


Agreed. As you said, direct manipulation of jre/lib/ext is not really an 
option (even on my own machine I need to be root to mess around this 
directory ;o)  Leaving it up to the user is not nice at all either (some 
users may even do not now what CLASSPATH is, not to mention messing with 
standard extensions).  There must be somehow a way for RAK to deal with it.

[...] 
> There's another issue along these lines, unfortunately. Custom URL
> implementations will fail, because Java insists on loading them from the
> system CLASSPATH only. I assume this is a security feature.
> 


Ok. I have an idea: what about instead of instantiating a main class 
from within the current JVM, launcher will spawn a seperate JVM process, 
setting up all these "nasty" jars inside bootclasspath? This way we 
could solve some of the problems with security and static class binding, 
like the ORB example. We could also by doing that cope with Java 
standard extensions nicely: all jars which suppose to be in jre/lib/ext 
and are not present there, would be "subsituted" by this bootclasspath jars.

What do you think?

1. Additional optional minimal minimum-jdk argument - when tried to be 
installed on a smaller JDK version the warning/error may be issued.

2. Additional bootclasspath on/off (yes/no) argument to tag if the given 
module should be included into the bootclasspath.

3. Additional if-not-available element to check if a given standard 
extension is present, if not, install and load a given required module 
(in bootclasspath or as regular module)


<dependencies minimum-jdk="1.2">

  <requires name="nzdis-util.1">
   <download href="http://prdownloads.sf.net/nzdis/nzdis-util.jar" />
  </requires>

  <requires name="orbacus-OB.1" bootclasspath="yes">
   <download href="http://www.ooc.com/orbacus-OB.jar" />
  </requires>

  <if-not-available class="java.util.regex.Pattern">
     <requires name="gnu-regexp.1">
       <download href="http://www.gnu.org/gnu-regexp.jar" />
     </requires>
  </if-not-available>

</dependencies>


the first nzdis-util is a regular dependency. second, ORB, must be 
included into the bootclasspath. third, gnu-regexp is only installed if 
the JDK 1.4 regular expressions pattern matching is not present, and is 
treated later as regular module (it could be tagged as bootclasspath 
optionally).

Any better names for the attribute and element names?
what do you think? suggestions?

best regards
Mariusz

Re: [rak-developers] Launcher and class loaders

[Kyle F. D. <kd...@am...>]

> Anyway, there are some problems:
>
> First, as mentioned before, the JDK version: some of the classes, like
> XML parsing are standard in some JDKs and not in others. There needs to
> be a mechanism to do a conditional module installation, or, some
> mechanism for checking out Java extensions. I does not look nice either
> way, anybody with some ideas here? Should we even bother with Java
> extensions? It would be nice to check them, install stuff there, and
> remove stuff from there. But it looks quite not nice to me to mess up
> with the jre/lib/ext subdirectory.
>

Yes, esp. on UNIX systems, that may not even be possible due to file
permissions. However, I do think both JDK version and standard extensions
need to be in scope for Rak.

> Second, some of the classes are not working well with custom class
> loaders. Our example is Orbacus ORB.  Because ORBacus ORB is loaded in a
> seperate class loader, the org.omg.CORBA.ORB which is a standard Java
> class cannot see it, thus, fails to instatiate it correctly. The only
> solution is to have ORBacus ORB in the Java standard extensions or in a
> classpath. This sucks.  I am sure there must be a way to go around it,
> but I have not discovered it yet. Help?

There's another issue along these lines, unfortunately. Custom URL
implementations will fail, because Java insists on loading them from the
system CLASSPATH only. I assume this is a security feature.

In Web applications that use my enterprise class library, I have to create
a separate JAR file called classlib-protocols-1.3.2.jar, for instance, and
put it in Tomcat's system CLASSPATH, because all of my custom URL handlers
will not be found it they're part of the WAR file, which is loaded in its
own ClassLoader. I assume Rak will have the same problem.

--kd

[rak-developers] Launcher and class loaders

[Mariusz N. <ma...@ma...>]

Hi,

I have ported most of our stuff to use RAK module management. It is 
really nice to see the installation process fetching all the necessary 
dependencies without a single question to the user. It simply rocks! ;o)
Especially, when the initial .rak module is also fetched dynamically 
from the net! We definitelly need to do an upgrader soon - this will be 
cool to do the upgrades the same way. Ah, and I have put in my /usr/bin 
a wee rak script like Etienne suggested, and I like it. It would be good 
to give rak script for unix and windows

Anyway, there are some problems:

First, as mentioned before, the JDK version: some of the classes, like 
XML parsing are standard in some JDKs and not in others. There needs to 
be a mechanism to do a conditional module installation, or, some 
mechanism for checking out Java extensions. I does not look nice either 
way, anybody with some ideas here? Should we even bother with Java 
extensions? It would be nice to check them, install stuff there, and 
remove stuff from there. But it looks quite not nice to me to mess up 
with the jre/lib/ext subdirectory.

Second, some of the classes are not working well with custom class 
loaders. Our example is Orbacus ORB.  Because ORBacus ORB is loaded in a 
seperate class loader, the org.omg.CORBA.ORB which is a standard Java 
class cannot see it, thus, fails to instatiate it correctly. The only 
solution is to have ORBacus ORB in the Java standard extensions or in a 
classpath. This sucks.  I am sure there must be a way to go around it, 
but I have not discovered it yet. Help?

Third, the code which uses reflaction to instantiate classes works well 
in the standard mode. However, in the dynamic mode it simply fails, as 
some of the dependencies between different modules may not be explicit. 
Example: we have XML processing library which stores and instantiates 
object from/to XML on the fly. Lets call it A.  We have module B, which 
depends on A, to store read XML objects of type B, however, A does not 
know about B existence, thus A is not dependent on B. When launcher 
prepares the class loaders graph based on the dependency graph, A fails 
to dynamically instantiate some of the B classes.  In standard mode, all 
jars are handled by flat class loader, and it simply works. In the 
dynamic mode, class loaders form a tree-like structure, and are very 
strictly based on dependencies, and, because A does not depend on B, A 
classes do not see B classes, thus the relfaction based instantiation 
fails.  I am not sure we can solve this nicely though. The only solution 
would be a mixed, flat and graph based class loader, which would do a 
graph only for a conflicting subgraphs, and use flat everywhere else. 
But, so far we do not know in advance which packages are the conflicting 
ones. The ones with the same name and different versions?  Any 
suggestions or help here?


Once we have the above points sorted out, I will write a section about 
launcher different modes in the docs.


with best regards
Mariusz

Re: [rak-developers] Pre-release

[Mariusz N. <ma...@ma...>]

Thanks Etienne for helping me out here.

> 1- You should put the "domain-name based" uniqueness mechanism first, on 
> top of  suggestions. (simple reordering;-)


In section 7.3 page 7 point 1 says exactly that. Check the latest
manual.pdf 132858 bytes on the web site.

If you are refering the the other section please let me now and I will 
fix it.

 
> 2- You do not insist enough on the fact that there are 2 versions and 2 
> names for packages: a) user visible name + version (e.g. 
> sablecc.2.16.1.jar), and b) RAK name and version (e.g. 
> org.sablecc.sablecc.jar.1.0.3).


Ok, I will add a paragraph about it.


> 3- The document does not really explains why RAK is better than 
> alternatives.


Please, please, give me a hand here. Where it should go? And how we 
should say this? ;o)


> 4- You don't explain how RAK is good at eliminating dependency conflicts 
> [the dynamic class loading trick for incompatible library versions].


Ok, I will do a section about it, however, I have to solve some of the 
technicalities first. (See next post).


> 5- You don't explain why RAK's internal CRA versioning is good, and that 
> it's simple to use as it is (almost) automatically assigned.


It should go into the general 7.3 section, or more detailed 8.3? Or 
somewhere else? Can you please post a wee paragraph about it?

 > I feel that without these explanations, RAK will not look as
 > attractive as it should.  Just an opinion.

Agreed!

I really need your hand here, guys, please help! ;o)
Raif, Kyle?!?   Please, please, give me some text to put in! ;o)

Ah, and one more thing, it would be PERFECT if someone writes one or two 
paragraphs briefly describing what RAK is and what it does, so we could 
use it as a template to send to different projects agitating them to use 
ModuleInfo descriptions for their JAR files.




> Finally, I dislike the fact that you have to explicitly give an option 
> to RAK in case of semantic changes; I think RAK sould automatically ask 
> the question when appropriate (e.g. when there are no incompatible 
> changes).  The "non-interactive" argument does not hold here: the 
> programmer has to be present at the time of "version assignment".

Ok, this one is a tricky one. The main motivation for not having a 
text-based version interactive is, that it will be used from within the 
developers IDE or building environment (like ANT for example where I 
will use rak-tools not as a seperate called by hand command, but as a 
target in ANT.) Imagine rak-tools being used in some GUI based IDE, and 
suddenly some weird question shows up in some strange place in some 
text-mode console, blocking the build process?

There is a trade-off, as always, and I was balancing the benefits for 
the majority of users.

I agree that this question should be asked in text-mode, however, I am 
not sure if the rak-tool itself should ask that question.  I thought, it 
should be left to the IDE/build environment. Let's take ANT as an 
example: I think, the question should be asked via ANT own mechanisms, 
and then, appropriate command (with or without --incompatibile) would be 
executed.

Of course, if the user is calling rak-tools herself, it is quite risky 
to let her "remember" to put --incompatibile or not. So, my reasoning 
was as follows:

* having rak-tools text-mode interactive may cause confusions with 
ANT/IDEs building environments
* most of the developers will not call rak-tools by hand, but will use 
IDEs/build environments (like ANT for example). The question then is 
asked in the most IDEs compliant way: dialog box, text-mode question, etc.
* pure semantic changes are not frequent (I have did a brief survey in 
our team, and we had none of such pure semantic change so far. We 
usually rename the method if we need a semantic-like change).
* most of the semantic changes will have other incompatibile changes as 
well, so, rak-tools retrospector will discover them anyway, even without 
the --incompatibile flag. This should minimize the problems with 
developers who forgot to put --incompatibile flag and use rak-tools 
directly.

Well, I think what we could do is: add a flag --interactive, and go with 
"if in interactive mode, then show up the question" model. What about that?


mariusz

Re: [rak-developers] Pre-release

[Etienne M. G. <eti...@uq...>]

Hi Mariusz.

I do not have much time.  Just a few comments about the docs:

1- You should put the "domain-name based" uniqueness mechanism first, on top of 
  suggestions. (simple reordering;-)

2- You do not insist enough on the fact that there are 2 versions and 2 names 
for packages: a) user visible name + version (e.g. sablecc.2.16.1.jar), and b) 
RAK name and version (e.g. org.sablecc.sablecc.jar.1.0.3).

3- The document does not really explains why RAK is better than alternatives.

4- You don't explain how RAK is good at eliminating dependency conflicts [the 
dynamic class loading trick for incompatible library versions].

5- You don't explain why RAK's internal CRA versioning is good, and that it's 
simple to use as it is (almost) automatically assigned.

Finally, I dislike the fact that you have to explicitly give an option to RAK in 
case of semantic changes; I think RAK sould automatically ask the question when 
appropriate (e.g. when there are no incompatible changes).  The 
"non-interactive" argument does not hold here: the programmer has to be present 
at the time of "version assignment".

I feel that without these explanations, RAK will not look as attractive as it 
should.  Just an opinion.

Etienne

Mariusz Nowostawski wrote:

> Hi,
> 
> So, I think we have reached a point of consensus in most of the issues. 
> I think we are ready to finish the first phase. Please, do check the 
> documentation, and the current CVS implementation, and let me know what 
> you think. You can apply all corrections to the original documents in 
> CVS (of course) or submit the correction to me so I will do that.
> 
> I will not have time to work on rak-gui (definitely not before January) 
> so if someone is happy to work on it please go ahead.
> (But please consult with me the creation of any new directories in CVS 
> first).
> 
> I will move our local modules to RAK, and it will be a good test case 
> for the current implementation. I will see how it works with relatively 
> simple deps, and how it behaves on Linux and Windows.
> 
> I am planning to make an official release in about two weeks, what about 
> that? After that, we can formulate a workplan for the next phase.
> 
> have fun
> Mariusz
> 
> 
> _______________________________________________
> rak-developers mailing list
> rak...@li...
> https://lists.sourceforge.net/lists/listinfo/rak-developers
> 
> .
> 


-- 
Etienne M. Gagnon                    http://www.info.uqam.ca/~egagnon/
SableVM:                                       http://www.sablevm.org/
SableCC:                                       http://www.sablecc.org/

[rak-developers] Java standard extensions

[Mariusz N. <ma...@ma...>]

Hi

I am preparing dependency tracking mechanism based on RAK and I have one 
general question:

How we should deal with Java standard extensions mechanism? (And related 
to that how about required JDK version?)

This applies to things like XML parsing, Sun tools.jar etc. If I will 
put direct dependency on some XML modules, or tools.jar it will be 
redundant for JDK 1.4 or if someone already has XML parsers as Java 
extensions. If I will not place dependencies, it will work with JDK 1.4 
but not with earlier ones.

The only reasonable solution I can think of, is, to add to RAK a check 
for existance of a given package/class, and instruct the user to put a 
particular JAR file inside Java extensions when needed. What do you think?

something like ANTish:

<dependencies>
    <available class="org.w3c.dom.Node">Please install XML parsing 
libraries as Java standard extensions. See 
http://java.sun.com/xml</available>
</dependecies>



best regards
Mariusz

[rak-developers] Pre-release

[Mariusz N. <ma...@ma...>]

Hi,

So, I think we have reached a point of consensus in most of the issues. 
I think we are ready to finish the first phase. Please, do check the 
documentation, and the current CVS implementation, and let me know what 
you think. You can apply all corrections to the original documents in 
CVS (of course) or submit the correction to me so I will do that.

I will not have time to work on rak-gui (definitely not before January) 
so if someone is happy to work on it please go ahead.
(But please consult with me the creation of any new directories in CVS 
first).

I will move our local modules to RAK, and it will be a good test case 
for the current implementation. I will see how it works with relatively 
simple deps, and how it behaves on Linux and Windows.

I am planning to make an official release in about two weeks, what about 
that? After that, we can formulate a workplan for the next phase.

have fun
Mariusz

Re: [rak-developers] Circular deps

[Etienne M. G. <eti...@uq...>]

On Wed, Nov 28, 2001 at 10:14:10PM -0500, Mariusz Nowostawski wrote:
> Naming:
> 
> I (almost) agree with all your points, Etienne! ;o) really ;o) 

I have to admit that I also agree with your points. ;-)  Super!
 
> In summary we have:
> 
> [1] Module names must be unique, and multi-part names (based on domain 
> names, following the Java convention) are required.
> 
> [2] RAK does not enforce a multi-part name depth, but encourages depth of 2 
> or more; RAK does not "strictly enforce" [1,2], but discourages developers 
> from not following the convention. 
> 
> [3] There is simply no short names, no additional aliasing.  But, 
> interactive tools [GUI/command-line] are a little smarter, allowing a user 
> to "search" for matching packages on a given substring of a module name.
> Configuration by default allows RAK to automatically fire single matches.

Agreed.

I would limit the "search" capability to a "tail search" (i.e. a
package matches if packageName.endsWith(searchedName)).  So, for
example, one could easily fire the different rak tools as:

 $ rak rak
 $ rak rak-gui
 $ rak rak-tools

[of course, the leading "rak" is a script of the form:
#!/bin/sh
java -jar /usr/java/rakiura-rak.jar $*
]

> So, there are only two more things to agree on. First is the multi-part 
> names separators. Do we need to enforce dots as separators? If so, why?

No, we do not need to enforce the dot.  The only thing is, if we want
to propose domain based names, we could use a separator which cannot
appear in a domain name part.  "-" is an example of a valid character
within a domain part.  "_" would probably be a better choice (illegal
within a domain name).  I really have no preference.  [The probability
of conflicts due to using "-" as separator is probably very low...]

> So the options are:
> 
> 1. enforce single separator in multi-part names (a). dot, b). minus, c). 
> underscore, etc)
> 2. suggest some separators but allow a choice

2. sounds fine to me. :-)


> The second "problem" is now just pure fun ;o)  RAK itself should be an 
> example of proper naming ;o)  Ups, so, we have to decide what the 
> actual, proper, mutli-part name for RAK and rak-gui and rak-tools should 
> look like.
> 
> It will be probably no surprise to you, if I will suggest respectivelly:
> 
> rakiura-rak
> rakiura-rak-gui
> rakiura-rak-tools
> 
> if you really really really want the "org" in front of it, and if the 
> majority thinks it would be better to have this "org" in front, well, I will 
> agree (after weeks of debating that is, hehehe, ;o)

I'll let Raif make his suggestion here.  My only argument for the
"org" is for rak to be an example of "good citizen".  On the other
hand, I fully agree that the probability of conflict on "rakiura-rak"
seems pretty low to me.  [I do not know Polish, so I cannot assert of
how common "rakiura" is..:-).]


> As for the - vs. . I do not really mind, but it seems to me that there 
> is no much of a benefit to constrain a choice here. And if there is, I 
> would rather go with - not . (to clearly distinguish Java packages from 
> module names).
> 
> What do you think?

Agreed.

> ps.
> 
> > Let me ask you: Why are you using the "nzdis" simple prefix, instead
> > of the fully qualified "nz.otago.nzdis" one?
> 
> The actual fully qualified would have been: nz.ac.otago.nzdis.*
> ;o) Why we have picked the "nzdis" prefix? Bryce McKinley picked it at 
> the first place (know him, Etienne? he has mention you some time ago to 
> me but I cannot remember the context) and it was later approved by all 
> of the team as safe and unique enough. If the probability of conflict is
> 1/(x^100) for "nzdis", we do not need to lower that probability to 
> 1/(x^1000000) by the use of "nz.ac.otago.nzdis" (for the arbitrary 
> constant x > 1). I fully agree, longer names do indeed lower the 
> conflict probability. But, as always, there is a trade-off to consider.
> 
> Note also, that the Java-like, long package naming based on DNS is not 
> the only human readable unique namespace model. Take a look at 
> Smalltalk: they have two-level package prefixes, followed by the actual 
> class name. That's all. Extremely simple, yet powerful. So, for example 
> you have Collections-Arrayed-Array, Collections-Unordered-Set, or 
> Kernel-Objects-Object. For our nzdis we would have enough space to have 
> exactly what we have now: Nzdis.Utils.XMLUtils, 
> Nzdis.Agent.AbstractAgent, etc.
> 
> And Smalltalk designers and developers (since 1970) think, it is all you 
> need. Smalltalk DOES not different classloaders, and, for most of the 
> cases all ppl are using ALL other ppl work ;p - thus the name conflict 
> is far easier here than in Java.
> Not that I fully agree with them, because we often need more than this 
> two-levels model to nicely package our stuff (for encapsulation and 
> engineering reasons). But the fact is, that it works well for them. (I 
> am unaware if Smalltalk has naming conflicts, and if so, how they deal 
> with it).

So, mainly, we agree.  I am not religious about domain based names; I
simply wanted the system to encourage the use of "long-enough" names,
as to avoid conflicts as much as possible.  On the other hand, I
didn't want those longer names to cause usability problems.

I am starting to like our latest proposal (yours + tail-search): I
could type:
 $ rak cc
or
 $ rak ecc
to fire up org-sablecc-sablecc ;-)

and maybe
 $ rak s
for firing up rakiura-rak-tools ;-)))

[I'm already finding ways to abuse the system; it must be a good one;-))]

Have fun!  And thanks for your incredible patience with me!

Etienne
-- 
Etienne M. Gagnon                    http://www.info.uqam.ca/~egagnon/
SableVM:                                       http://www.sablevm.org/
SableCC:                                       http://www.sablecc.org/

Re: [rak-developers] Circular deps

[Mariusz N. <ma...@ma...>]

Hi,

Scripting: we basically agree. Two optional Java executables invoked onInstall and onRemoval events are available for module packagers, and later XML-based Preferences-driven application configuration will be provided.


Naming:

I (almost) agree with all your points, Etienne! ;o) really ;o)  And I 
have to say it is a pleasure to work and talk with you - you _are_ 
persistant ;o)

The idea of smart aliases is simply brilliant!  I will put it into the 
text-based implementation. In case one module name matches "an alias" 
this module will be used. In case of conflict, all matching names will 
be printed out. GUI would give an option of picking one from the list I 
would guess.

In summary we have:


[1] Module names must be unique, and multi-part names (based on domain 
 names, following the Java convention) are required.


[2] RAK does not enforce a multi-part name depth, but encourages depth of 2 

or more; RAK does not "strictly enforce" [1,2], but discourages developers 

from not following the convention. 


[3] There is simply no short names, no additional aliasing.  But, 
interactive tools [GUI/command-line] are a little smarter, allowing a user to 
"search" for matching packages on a given substring of a module name.

Configuration by default allows RAK to automatically fire single matches.



So, there are only two more things to agree on. First is the multi-part 
names separators. Do we need to enforce dots as separators? If so, why?

I do not think we need to enforce any particular separator here, dot, 
underscore, minus etc All seem quite good to me. The dot, minus and 
underscore would be the default separators recognized by RAK GUI (for 
the sorting thingy).  I actually thing the use of dots would be 
confusing with the actual Java packages, which is not necessary too 
good. Also, we agree that the canonical name should be for a user 
convenience a file name as well, so for org-sablecc-sablecc.jar sounds 
quite nicer than org.sablecc.sablecc.jar. I am not so sure how different 
OSes handle mutliple dots in file names. Maybe it will be also easier 
for developers of existing packages to adopt their naming conventions to 
the one "required" by RAK.


So the options are:

1. enforce single separator in multi-part names (a). dot, b). minus, c). 
underscore, etc)
2. suggest some separators but allow a choice


The second "problem" is now just pure fun ;o)  RAK itself should be an 
example of proper naming ;o)  Ups, so, we have to decide what the 
actual, proper, mutli-part name for RAK and rak-gui and rak-tools should 
look like.

It will be probably no surprise to you, if I will suggest respectivelly:

rakiura-rak
rakiura-rak-gui
rakiura-rak-tools


if you really really really want the "org" in front of it, and if the majority thinks it would be better to have this "org" in front, well, I will agree (after weeks of debating that is, hehehe, ;o)


Note, this "org" in front will NOT in fact decrease the naming conflict 
probability anyway though.

As for the - vs. . I do not really mind, but it seems to me that there 
is no much of a benefit to constrain a choice here. And if there is, I 
would rather go with - not . (to clearly distinguish Java packages from 
module names).

What do you think?

best regards
Mariusz



ps.

 > Let me ask you: Why are you using the "nzdis" simple prefix, instead
 > of the fully qualified "nz.otago.nzdis" one?

The actual fully qualified would have been: nz.ac.otago.nzdis.*
;o) Why we have picked the "nzdis" prefix? Bryce McKinley picked it at 
the first place (know him, Etienne? he has mention you some time ago to 
me but I cannot remember the context) and it was later approved by all 
of the team as safe and unique enough. If the probability of conflict is
1/(x^100) for "nzdis", we do not need to lower that probability to 
1/(x^1000000) by the use of "nz.ac.otago.nzdis" (for the arbitrary 
constant x > 1). I fully agree, longer names do indeed lower the 
conflict probability. But, as always, there is a trade-off to consider.

Note also, that the Java-like, long package naming based on DNS is not 
the only human readable unique namespace model. Take a look at 
Smalltalk: they have two-level package prefixes, followed by the actual 
class name. That's all. Extremely simple, yet powerful. So, for example 
you have Collections-Arrayed-Array, Collections-Unordered-Set, or 
Kernel-Objects-Object. For our nzdis we would have enough space to have 
exactly what we have now: Nzdis.Utils.XMLUtils, 
Nzdis.Agent.AbstractAgent, etc.

And Smalltalk designers and developers (since 1970) think, it is all you 
need. Smalltalk DOES not different classloaders, and, for most of the 
cases all ppl are using ALL other ppl work ;p - thus the name conflict 
is far easier here than in Java.
Not that I fully agree with them, because we often need more than this 
two-levels model to nicely package our stuff (for encapsulation and 
engineering reasons). But the fact is, that it works well for them. (I 
am unaware if Smalltalk has naming conflicts, and if so, how they deal 
with it).

Re: [rak-developers] Circular deps

[Etienne M. G. <eti...@uq...>]

Mariusz Nowostawski wrote:

> Scripting.
> 
> Ok, I can see you point now. I will read the docs there, but my first 
> conclusion is, that because Java 1.4 already have a robust scalable and 
> flexible way of handling multi-library and multi-application 
> configurations (i.e. Preferences), we probably should stick to that. ...


In fact, scripting, as you were suggesting, might be required, if developers 
want to do more than setting up properties.  (Your example of application 
specific caches, and so on, come to mind).  In which case, learning more about 
the .deb policy might be a good idea: .deb had 2 levels of uninstallation: 1- 
remove, which removes the software but leaves configuration information as is, 
and 2- purge, which cleans up everything realted to the application.

> We 
> should probably leave those two event handlers as they are but clearly 
> formulate the "policies", and give the module developers some XML-based 
> way of setting up Preferences. But, because of JDK 1.4 not being 
> released yet can wait I think.


Agreed.



> Naming.
> 
> 
> I think we agree on most of the points.


Agreed. ;-)

> And I will summarize it here:
> 
> 1. modules must have unique names


OK.

> 2. users should "handle" modules via somehow nice, human readable names 
> (not necessary equivalent to "short")


We disagree on the "(not necessary equivalent to "short")".

> 3. the design should be simple, yet robust and flexible


OK.

> 4. we do not enforce any "depth" on the module names


OK.

> 5. adopting ideas from Java package names is good enough (for me and 
> Etienne)


OK.

> 6. adopting prefix-name seems to be good as well for both of us


OK.

> Ok, there are some indirect conclusions which can be derived from the 
> above: because we do not enforce any depth (like in Java packages), 
> single name is still a valid multi-part name in respect to RAK.


Yep, but their usage is highly discourage.  (We disagree on this, but I'll try 
to convince you otherwise...).

> the only point we have not agreed is:
> 
> 7. additional aliasing scheme is being used, which maps a short name to 
> the module name (alias --> multi-part name).



In fact, I would agree no to have this additional aliasing, as long as we do not 
implicitly encourage the use of "single" names.  So, if we can find some magical 
way to achieve the goal of encouraging the usage of "multi-part" package names 
(org.apache.ant), whithout complicating programmer and user experiences, then we 
should have a solution that satisfies both of our objectives.

You might agree or not on whether multi-part names are good discriminators, but 
if you think of RAK as a widely used packaging system, in all kinds of 
environments (open-source, proprietary, public, private, etc.), on a very large 
scale basis, then the domain name prefix, even thought not perfect, is a really 
good scheme (even thought you think it is a somehow cheap trick).


Let me ask you: Why are you using the "nzdis" simple prefix, instead of the 
fully qualified "nz.otago.nzdis" one?  I am sure your answer is something like: 
because we got tired of writing "package nz.otago.nzdis" and creating all these 
"useless" subdirectories "nz/otago/".  If it hadn't been such a pain to use the 
fully qualified name, you probably  wouldn't have even considered infringing the 
Java package naming convention.

We both know that generalized use of the fully qualified name would lead, in the 
longer term, to a much lower probability of name conflicts.  And, if using the 
fully qualified name is really easy, then there will be a very easy way to 
resolve a conflict if two package providers decided not to follow the convention 
and used a similar name.

I think that the Java designers mistake was to make it such a pain to use the 
fully qualified names.  The pain argument is the sole one I have heard, up to 
now, for not following the convention.  I do not want us to repeat that mistake.

Your solution is to say, "well, the hell with fully qualified names; most of the 
time, there's no conflict anyway".  My solution is to say: "well, some conflicts 
will eventually happen, if people do not use fully qualified names, and when 
they do, they'll mess up the end user's system.  So, let's encourage the use of 
fully qualified names by making them as easy to use a single-names.  This way, 
most people (but maybe some unreductible people like you;-), will use them, and 
thus the probability of screwing up a user's system will be much reduced."

I do really think that requiring somebody to type:

$ java nz.otago.nzdis.tools.rak ca.mcgill.sable.soot

is a pain in the %$/@.  We want people to type:

$ rak soot

instead.

Now, in your approach, this indirectly (or directly) encourages the usage of 
single names.  (Much like browsers encourage the registration of ".com" domain 
names by letting you type "xyz" and automatically looking up "www.xyz.com").

You do not like the usage of an additional aliasing level, to simplify a user's 
life, while encouraging the use of "longer" multi-part fully qualified names. 
Fine with me; I do understand the need for reducing the number of aliases in the 
system.

So, I see a combination of 2 tricks to accomodate both our objectives [(me) 
fully qualified + (you) no additional aliases]:

1- Use a GUI for lauching applications.  The GUI *allows* the user to "sort" 
applications (in lists) by the latest component of the fully qualified name 
(e.g. sort on "ant, sablecc, rak" instead of "com, org, ca, ...").  This way, 
fully qualified names do not make life anymore difficult for anybody (but the 
RAK GUI developer;-).

2- For the command line tool.  Instead of providing aliases, the tool, when 
called with the "--interactive" argument, will try to be a little smarter, and 
will search for matching packages, e.g.:

   $ rak ca.mcgill.sable.soot  => execute it.


   $ rak --interactive soot
               => no "soot" found => single name => search on the last component
                  of qualified names for "soot" and propose found ones to
                  end-user:
     Do you mean "ca.mcgill.sable.soot" (Y/N)? y
     [execute it].

(In fact, in case of a single possibility, we should let the user configure RAK 
to automatically launch the found package, e.g.:

   $ rak --interactive soot  => execute ca.mcgill.sable.soot
)

As we know that "developers" using RAK will probably be inclined to use the 
command line tool.  Making their life simple, when using fully qualified names, 
is critical to their wide adoption (which is my goal).

This proposal eliminates the additional aliasing.  We are simply making the 
interactive tools a little smarter.  We leave non-interactive tools as simple as 
they currently are, e.g. "$ rak soot => error!".

What do you think?  Does this satisfies you?

> Ok. The current resolution is:
> 
> [1]. module names must be unique, and multi-part names are suggested (we 
> may say even "are required")


OK.  Multi-part names follow the Java package name convention.  e.g. 
"org.apache.ant".

> [2]. no depth requirement (which makes point 1. a little weaker)


OK.  But, highly discouraged; maybe we could even get the RAK packager to 
display a warning when a developer tries to use one (and why not ask the 
developer for the fully qualified name and automatically update the MODULEINFO 
file ;-)).

> [3]. and if developers think they should use domain names it is fine 
> with RAK (full compliance with Etienne model)


I would put this in [1].


> [4]. there is simply no short names, no additional aliasing. If one 
> calls his module: a.b.c.d.MyPackage it is fine with RAK, the user will 
> need to know the full module name then.


Yes and no.  Yes for non-interactive tools.  In the GUI, lists contain the fully 
qualified names, but can be sorted either on the whole name, or on its latest 
component (for convenience).

Interactive command line tools will be a little smarter, and search for matching 
packages when single-name arguments fail.

> What you say Etienne? Close enough?
> I think we agreed on [1]-[3], the only conflict left is in [4].


Here's my proposal:


[1] Module names must be unique, and multi-part names (based on domain names,
     following the Java convention) are required.
[2] RAK does not "strictly enforce" [1], but discourages developers from not
     following the convention.
[3] There is simply no short names, no additional aliasing.  But, interactive
     tools [GUI/command-line] are a little smarter, allowing a user to "search"
     for matching packages.  Configuration optionally allows RAK to automatically
     fire single matches.

> ps. Ah, the reasons that I do not want additional aliasing, I gave them
> all in the text above, but to summerize:
> 
> * increased complexity as in point 3 high above
> * we already have one aliasing scheme: application names into the actual 
> executable classes; additional aliasing would be confusing
> * handling module removal and launching with aliases, names, and 
> canonical names (and even conflicting aliases) would be far too 
> complicated for me, and an average user. It is better and far more 
> simpler to enforce the use of names and canonical names (assuming they 
> are not ridiculously long that is ;o)


I am proposing to allow names to grow "ridiculously long" without making a 
user's experience difficult...

> * the aliases promote the user not to know the acutal canonical names, 
> which will cause problems understanding the storage areas - I feel that 
> canonical names and filenames should be same, and user should be aware 
> of them, i.e. know them. Once the user knows canonical names, there is 
> no need for aliases.


Maybe, but you'll agree that asking the user to remember 
"nz.otago.nzdis.tools.rak" and type it everytime is not pragmatic.  All I am 
saying is that the user should recognize "nz.otago.nzdis.tools.rak", but it 
should be sufficient to actually remember "rak" for everyday usage.  RAK should 
be smart enough to find matching packages.

You know, when you use a book, you don't have to remember that the documentation 
for a specific question is on page 423; you can go in the index and lookup the 
function name, and find that there's information about it on pages 19, 125, and 
423.  I am simply proposing a similar scheme, where the user has only to 
remember the "important" information, and the system can help the user find the 
more precise definition.

I agree that my alias proposal was adding some level of complexity in the cache. 
  On the other hand, my current proposal requires no change to the 
cache/repository.  It only requires (little) smarter interactive tools.


> * developers and module packagers must be aware of the canonical names 
> anyway, no need for aliasses for them


My current proposal does not eliminate the need to know the canonical name, for 
either developers and users.  It simply saves a little typing, and allows for 
weaker "memories" like mine.  For example, I do have to use cut/paste to type 
"nz.otago.nzdis.tools.rak".  There's no way I'll ever remember something like 
that, unless I am forced to type it 50 time a day....


> * I believe aliases would cause the canonical names go ridiculously 
> long, which is something I would like to avoid. Library and application 
> canonical names should be short enough for the user and developer to be 
> aware of them, and know them


We disagree here.  Why not allow them to grow long?  To save you some typing?

Longer names = lower conflict probability.  So, from a RAK point of view, they 
are good:-)  All we have to care about is not to make users/developers life 
difficult when using them; thus my "search/sorted" interactive tools.

> * I believe aliases would tend to be ridiculously short, thus useless 
> due to frequent aliases conflicts


My search proposal will display all matching packages.  No more conflicts to 
deal with:-)

So, how about it?  Have I finally convinced you?


Etienne

-- 
Etienne M. Gagnon                    http://www.info.uqam.ca/~egagnon/
SableVM:                                       http://www.sablevm.org/
SableCC:                                       http://www.sablecc.org/

[rak-developers] RAK manual (draft)

[Mariusz N. <ma...@ma...>]

Please have a look at the current RAK manual (draft)

    http://www.rakiura.org/rak/manual.pdf


All feedback welcome: rewording, reorganizing section, improved 
paragraphs, spotted mistakes, etc etc. If you do not want to fiddle with 
LaTeX just send me a raw text and I will put it in.

I have just printed it out and will go through it as well.

best regards
Mariusz

Re: [rak-developers] Circular deps

[Mariusz N. <ma...@ma...>]

Hi,


> I'll be having a heated argument below.  I'm simply writing my raw 
> thoughts quickly, so I do not have time to wrap them in a diplomatic 
> envelope;-)  Please do not take anything personally:)))


No worries, it is just a piece of software we are talking about, isn't 
it? Not a big deal ;o)  Btw, I am very often just writing my raw 
thoughts or just thinking aloud in the emails, so please do excuse me as 
well for that.
I think it is ok to discuss different points of view here, and have an 
open mind for different views - somethink like a brainstorming sessions, 
where every idea counts. It would be pointless to "discuss" anything if 
everybody agrees with everone else.



Scripting.


> I didn't say that .deb didn't support [install/removal]-time scripts; in 
> fact, it supported them way before .rpm did!  I said that Debian had a 
> policy for .deb packages.  This policy contains rules like: 
> configuration files go into the /etc directory tree, etc.  See: 
> http://www.debian.org/doc/debian-policy/


Ok, I can see you point now. I will read the docs there, but my first 
conclusion is, that because Java 1.4 already have a robust scalable and 
flexible way of handling multi-library and multi-application 
configurations (i.e. Preferences), we probably should stick to that. We 
should probably leave those two event handlers as they are but clearly 
formulate the "policies", and give the module developers some XML-based 
way of setting up Preferences. But, because of JDK 1.4 not being 
released yet can wait I think.



Naming.


I think we agree on most of the points. And I will summarize it here:

1. modules must have unique names
2. users should "handle" modules via somehow nice, human readable names 
(not necessary equivalent to "short")
3. the design should be simple, yet robust and flexible
4. we do not enforce any "depth" on the module names
5. adopting ideas from Java package names is good enough (for me and 
Etienne)
6. adopting prefix-name seems to be good as well for both of us

Ok, there are some indirect conclusions which can be derived from the 
above: because we do not enforce any depth (like in Java packages), 
single name is still a valid multi-part name in respect to RAK.


the only point we have not agreed is:

7. additional aliasing scheme is being used, which maps a short name to 
the module name (alias --> multi-part name).

Etienne thinks 7. is good, I think it adds unnecessary level of 
complexity. In other words Etienne does not think 1. and 2. could be 
done via a single name, I believe it can. If it can be done via a single 
name, the point 3. will be happier.


Use case: junit, bcel, sablecc, jakarta-ant, nzdis-tools ....

Etienne:
multi-part names are superior and we should have:

short names:
      junit, bcel, sablecc, ant, tools
names:
      org.junit, org.bcel or jakarta.apache.bcel, org.sablecc.sablecc
      org.jakarta.ant, nz.otago.nzdis.tools
canonical names:
      org.junit.3.7.0, org.bcel.4.4.0 or jakarta.apache.bcel.4.4.0,
      org.sablecc.sablecc.2.17.0, org.jakarta.ant.1.4.0,
      nz.otago.nzdis.tools.1.0.0

Mariusz:
multi-part names are good, but the essence of the multi-part name is its 
superior discrimination in the "name space" when compared with the 
single name. However, not all the parts of the multi-part name are (by 
any means) good discriminators just by definition. For example "org." or 
"com." are useless discriminator in general. In summary, I believe it is 
enough in most of the cases to use two-part multi-part names - the very 
issue is to use a part which guarantees a uniqueness of the second part, 
yet itself is highly unique. for example: apache, jakarta, ibm, sun, sf 
(sourceForge), sablecc, rakiura, nzdis, etc. are good first level 
discriminators. Good first level discriminators are: domain names, brand 
names, big projects names. However, in my proposal RAK does not limit 
the depth of the multi-part name either. (RAK treats in both cases the 
module name as a single word, regardless its multi-part nature).

So, we could have for the above use case:

names:
      junit, bcel or jakarta-bcel, sablecc
      jakarta-ant, nzdis-tools
canonical names:
      junit.3.7.0, bcel.4.4.0 or jakarta-bcel.4.4.0,
      sablecc.2.17.0, jakarta-ant.1.4.0,
      nzdis-tools.1.0.0

As names are short enough, there is no need for the additional aliasing.



Risks.

In both models there is a risk of misuse:
short names:
      ant, tools
names:
      ant, tools
canonical names:
      ant.1.4.0, tools.1.0.0

which will cause RAK system misfunction once more than a single module 
uses such a name. However, there is NO implicit risk in either scheme - 
when used according to the guidelines, both guarantee correct and unique 
naming schemes.

I agree, the second, Mariusz's scheme is a little more easier to be 
missused. The trade-off is between the higher risk and increased 
simplicity.  Assuming the "misuse" to be an exceptional and marginal 
thing, I am for going with the second scheme. As said before, they are 
equivalent, with the second one being reduced version of the first one.


I am not saying the name should have only two parts. It can have as many 
as you want, you can even call it:
com.bla.foo.a.b.c.MyName

The only constrain is, that you should maximize the uniqness, but 
keeping the human readability aspect high. Two-parts names are good 
enough in most of the cases.


In Etienne model, multi-part names may go ridiculously long, even though 
the discriminating aspect may stay low, while the short names may go 
ridiculously short, rendering them useless (like calling a module "ant").




> You are suggesting to allw package managers to corrupt a user's system.  
> I am resolutely opposed to this. :)


Package manager in your scheme can also corrupt user's system. Your 
scheme does not exclude that possibility either.

 
> What you suggest has been tried before, and has proven not to work.


I do not think so. It is being tried in the Java packaging scheme for 
quite a long time and it seems to work quite well. ;o)  For example we 
prefix all our packages with nzdis (even though we do not have such a 
domain) since 1996 and have never noticed any conflicts here ;o)
I bet if you have picked to use sablecc prefix from the start, instead 
of org.sablecc and previous ca.something.something.sablecc you would not 
have any troubles either.


> What do you think is causing the .dll maintenance nightmare on Windows 
> machines?  Microsoft hoped evey product would give their own names to 
> ".dll"s, and that the possibility of name conflicts was low, and that it 
> would be self regulating, much like your vision.  Today, system 
> administrators have a very difficult job solving problems for users that 
> install their own packages, as sometimes packages overwrite the .dll of 
> other packages...


Well, you were very limited with 8.3 scheme at the begining, and I 
suspect there was no strict guidlines about DLL naming conventions as 
such either. With a strict naming convetions, like companyname-name the 
scheme would work as good as com.companyname.name model.


> Note: ** You cannot build a distributed conflict-free system without 
> uniqueness **


Fully agreed.

 
> Now, a hierarchical model imposes uniqueness through central control.  
> You want to avoid that.  Fine.  But, as RAK is managing Java packages, 
> you might as well take advantage of some prerequisites already imposed 
> by Java on package developers: each package developer has to put his 
> code in a Java package of the form inversed.domain.name.*.


Which is not far from what I am saying. I am saying use multi-part names 
the way you think you can maximize uniqueness, keeping the simplicity 
high at the same time.

> My proposal provides a unique canonical name.  


Agreed. I am saying the same. My proposal suggest the use of unique, 
multi-part names.

> It then lets the user use a friendly shorter non-canonical name, 

> as long as no conflict arises.


As I said, having shorter names under the condition that there is no 
conflict is self contradicting. Naming something "ant" or "tools" or 
"parser" makes no sense, because users will not be able to use these 
"short and nice and human readable names" anyway. So what is the point?
And if we say, short names should be somehow short, but somehow unique, 
we are not far from what I am saying.

The point should be to have unique, yet human readable and relatively 
short names. Is it possible? I believe so. Based on the essense of java 
package naming scheme.


> Now, if I have to state everything, I'll do.  When a conflict happens, 


It should never happen. If it does happen, RAK 2.0 (once there is 
zillion RAK packages everywhere in the world ;o)  will have modules 
multi-storage areas model, which will allow user to install different 
modules with the same name in seperate storage areas. This will allow a 
module dependencies subtrees work well even if the names are screwed up.



> Now, why all this trouble?  You seem not to see the importance of the 
> problem caused by a name conflict.  A name conflict has the potential of 
> severely damaging RAK's package repository/cache.


It does, I know that. It renders the whole subtree of all installed 
modules unusable. I simply think it should never happen at the first 
place. It is like RuntimeException - RAK 2.0 as a side effect of 
multi-storage area will help to cope with that, but, this is something 
which should never happen.

 

> Using domain names approach is not a hierarchical approach: it is in 
> fact an elegant solution to giving a UNIQUE private name space to every 
> package provider, without the need for a "new" central authority.  


This is not in conflict with what I was saying. There are some issues 
with raw domain names: not all module publisher have domain names, not 
all domain names are in fact good discriminators, we are not enforcing 
any depth so nothing stops module publishers calling the module with the 
brilliant multi-part name "ant", and finally, there is no way we can 
(without central authority) make sure name conflict will never happen.

> In fact, every Java programmer has already to deal with the 

> "domain name" central (in fact relatively distributed) authority.


This is correct. This is however a cheap yet robust trick ;o) (cheap 
because first it does not address the developers without the domains and 
second because it relies on underlying DNS model to solve the uniqueness 
problem for us.
It is not necessary in most of the cases to use full domain name, just 
the most discriminating bits of it.  I am just proposing a little more 
elaborate variation of that scheme, that's all.



> Within that name space, every package provider can organize things as he 
> wants.  No need to register a package anywhere when releasing it, ...    
> Obtaining a private name space is a ONE-TIME operation (preceedes even 
> the work on writing the package!).  This is, in fact, the key for a 
> flexible yet conflict free decentralized system.


I fully agree. Still, I think the unique private name space like: 
sablecc, jakarta, sf, ibm, apache, nzdis, rakiura, etc are good enough 
discriminators to be used as a root, as a private name space. The 
uniqueness of subprojects is guaranteed then by the "root" project.

Also note, projects names UNLIKE Java package hierarchies are relativelly flat structures, they usually consist of a main project name, like Jakarta or SourceForge (or company name), and a flat list of all modules within that project.

> Lets agree on the terms:
> 
> Canonical name       == domain name based package name, e.g.: 
> org.apache.ant


My terms are:
   name == unique module name
   canonical name == (unique module) name + CRA == name.CRA
   dependency link == (unique module) name + C == name.C

> Short name/Nick name == user friendly name, e.g. ant, sablecc, rak


I do not have short names.


> I am proposing this:
> 
> [Dependency encoding]
> (some package) depends on (canonical name + CRA version)


My term is:

Canonical name of some package depends on dependency link



> User gets to write "rak ant"  and "rak sablecc".


In my model user has to use name (unique module names) at the first 
place. There is no point using "ant" as we have "ant the build tool" and 
"ant the screensaver" are in conflict.

 
> If a conflict happens, RAK can do any of the following:


the conflict will not happen, the names are unique.

 >
 > Small detail, while I'm thinking of it:
 >
 > When you write:
 >   $ rak sablecc
 >
 > This means: run the highest installed SableCC package, right?  How do
 > you run an older version?  Maybe:
 >
 >   $ rak sablecc.2  [which happens to be: SableCC 2-> SableCC 4.0.2]
 >
 > Is this what you have implemented?


Yes. I mean close enough, currently if the user say: sablecc, and there 
is more than one, the highest one will be picked (for start) and all 
will be removed (for remove). If the user wants to specify a concrete 
version user needs to give canonical name (in my vocabulary: name.CRA).


 > I do not know if we should continue the name argument for long.
 > I think I have stated my mind pretty clearly.

Ok, one more iteration and we are done ;o) I think I have stated my 
point also quite clear this time.

Raif? Kyle? Any feedback here?


 > I simply feel that your reasons for not adopting the
 > domain name approach are based on a confusion of hierarchical vs.
 > distributed approaches.

Not really.
I think I have rehashed and clarified my points in this respect here. 
Domain-names-like approach is best we can get from both worlds in my 
opinion: the world of Java packages, with deep tree-like organization, 
and module names, with mostly two-level organization. Also, note that 
the Java package "prefix" is in fact the main construct for 
discrimination; the rest is the organization of class references and 
encalpsulation. This is something out of the module naming world, this 
is object oriented encapsulation and "java packages" world - which has 
nothing to do with modules naming. Personally I do not see any need for 
a.b.c.d.MyPackage module naming, but I can see many reasons to use it 
for packaging my classes.


>  > Well, I do not know what to do. Should we resolve it before 1.0 is out?
> 
> I do think so.


Ok. The current resolution is:

[1]. module names must be unique, and multi-part names are suggested (we 
may say even "are required")
[2]. no depth requirement (which makes point 1. a little weaker)
[3]. and if developers think they should use domain names it is fine 
with RAK (full compliance with Etienne model)
[4]. there is simply no short names, no additional aliasing. If one 
calls his module: a.b.c.d.MyPackage it is fine with RAK, the user will 
need to know the full module name then.


What you say Etienne? Close enough?
I think we agreed on [1]-[3], the only conflict left is in [4].


thanks a lot for your comments and patience  ;o)

best regards
Mariusz


ps. Ah, the reasons that I do not want additional aliasing, I gave them 
all in the text above, but to summerize:

* increased complexity as in point 3 high above
* we already have one aliasing scheme: application names into the actual 
executable classes; additional aliasing would be confusing
* handling module removal and launching with aliases, names, and 
canonical names (and even conflicting aliases) would be far too 
complicated for me, and an average user. It is better and far more 
simpler to enforce the use of names and canonical names (assuming they 
are not ridiculously long that is ;o)
* the aliases promote the user not to know the acutal canonical names, 
which will cause problems understanding the storage areas - I feel that 
canonical names and filenames should be same, and user should be aware 
of them, i.e. know them. Once the user knows canonical names, there is 
no need for aliases.
* developers and module packagers must be aware of the canonical names 
anyway, no need for aliasses for them
* I believe aliases would cause the canonical names go ridiculously 
long, which is something I would like to avoid. Library and application 
canonical names should be short enough for the user and developer to be 
aware of them, and know them
* I believe aliases would tend to be ridiculously short, thus useless 
due to frequent aliases conflicts

Re: [rak-developers] Circular deps

[Etienne M. G. <eti...@uq...>]

OOps...

Etienne M. Gagnon wrote:

> [Dependency encoding]
> (some package) depends on (canonical name + CRA version)


Should be:

(some package) depends on (canonical name + C version)
where C is the C in CRA :-)

Etienne
-- 
Etienne M. Gagnon                    http://www.info.uqam.ca/~egagnon/
SableVM:                                       http://www.sablevm.org/
SableCC:                                       http://www.sablecc.org/

Re: [rak-developers] Circular deps

[Etienne M. G. <eti...@uq...>]

Hi Mariusz.

I'll be having a heated argument below.  I'm simply writing my raw thoughts 
quickly, so I do not have time to wrap them in a diplomatic envelope;-)  Please 
do not take anything personally:)))

Etienne

Mariusz Nowostawski wrote:

> 
> So how Debian handles modules removal, e.g. if my app is creating some 
> temporary cache files which are not really installed, but created when 
> running,and which are completely useless without the app? How Deb will 
> remove those files?  RPM simply removes those file via custom script.


I didn't say that .deb didn't support [install/removal]-time scripts; in fact, 
it supported them way before .rpm did!  I said that Debian had a policy for .deb 
packages.  This policy contains rules like: configuration files go into the /etc 
directory tree, etc.  See: http://www.debian.org/doc/debian-policy/

To put things in perspective, I do not really care if you adopt the config 
policy approach or not, and if you provide scripting or not.  I care much more 
about the naming issue.

I think it is important that canonical names make it into RAK 1.0, as it will be 
too late later to introduce them (backward compatibility would leave open the 
possibility of system corruption).

> 
> True, but I still think, even if Debian had distributed model, and the 
> problem occured in one of the nodes, the fix would eventually made its 
> way to all the nodes. The name change happens not in the node, it 
> happens in the original development team. The important thing is a 
> feedback loop - if I noticed two independent modules using the same 
> name, I am reporting it to either development team, and they are fixing 
> it. Feedback loop, self regulation, no central repository needed. No 
> single convention needed.


You are suggesting to allw package managers to corrupt a user's system.  I am 
resolutely opposed to this. :)

What you suggest has been tried before, and has proven not to work.  What do you 
think is causing the .dll maintenance nightmare on Windows machines?  Microsoft 
hoped evey product would give their own names to ".dll"s, and that the 
possibility of name conflicts was low, and that it would be self regulating, 
much like your vision.  Today, system administrators have a very difficult job 
solving problems for users that install their own packages, as sometimes 
packages overwrite the .dll of other packages...

OK.  I read your motivation in your second message.  Personally, I think your 
are confusing separate things.  You are confusing a hierarchical system, where 
you always need to publish everything through a central channel, with a robust 
decentralized system, which insures the absence of conflict by "pre"-allocating 
private name spaces.

First, even if you want a  non-hierarchical distributed model, you still need 
some way of uniquely identifying a package.  This can be done in different ways:
1- Assign a GUID to a package.  The GUID is based on the unique ID of your 
network card + the creation time of the GUID.
2- Add a mandatory "author" field to the package, with some kind of unique way 
to identify the author, e.g. : name + birth date + birth place + social security 
number (or some other government ID number).
3- The Java way: use a domain that you control, then do whatever you want within 
that name space.
4- Variations on the same theme.

Personally, I do not like 1-, because:
a) It discloses private information about my network card.  Also, it is 
"probably" unique (not "necessarily").
b) The ID is not really human readable, e.g. af03-2903-fe92-289273839-...

I do not like 2- either, as it discloses personal information.

This leaves 3-. (Or 4-)

Note: ** You cannot build a distributed conflict-free system without uniqueness **

Now, a hierarchical model imposes uniqueness through central control.  You want 
to avoid that.  Fine.  But, as RAK is managing Java packages, you might as well 
take advantage of some prerequisites already imposed by Java on package 
developers: each package developer has to put his code in a Java package of the 
form inversed.domain.name.*.

[continued below]

>> I think my proposal, with appropriate tool support (by letting the 
>> tools use short non-canonical names) is the right solution.
> 
> well, it assumes that there is no conflict, which is self contradicting.


No,  I disagree.  My proposal provides a unique canonical name.  It then lets 
the user use a friendly shorter non-canonical name, as long as no conflict arises.

Now, if I have to state everything, I'll do.  When a conflict happens, RAK can 
let the user assign a "system specific" shorter name to conflicting packages. 
So, from a user's perspective, the problem is easily solved without having to 
wait for both package providers to agree on who will change the name of his 
package [I see long and heated conflicts there, specially if both own the 
tradmark in their respective countries!  Yep, this can happen, like it or not.].

Now, nothing prevents the user to ask them to do something about it, but 
meanwhile, the user can have a working system.

Now, why all this trouble?  You seem not to see the importance of the problem 
caused by a name conflict.  A name conflict has the potential of severely 
damaging RAK's package repository/cache.

Imagine the following situation.

User installs Prog 1.0.0 which depends on Foo 1 (installs Foo 1.0.0 by same 
vendor).  All goes well.

A couple of months later, user Downloads SuperHotProg 4.5.2 (by another 
unrelated vendor), which happens to depend on a different (name conflict) Foo 1 
(Foo 1.0.0 written by vendor of SuperHotProg).

RAK installs SuperHotProg 4.5.2, and finds that Foo 1 is already resolved by the 
old Foo 1.0.0 library.

User lauches:
$ rak superhotprog
=> link time exceptions

Now, imagine vendor of SuperHotProg had Foo 2.0.1 instead.  So, running SPH will 
go well.  But a few weeks later, user needs to use Prog => everything is 
broken!!  The user is left to himself trying to solve the problem.  Just imagine 
if Foo 1 was a fifth level dependency of Prog...

Now, the ugly part is that, even if users knows about the conflict, he can do 
nothing about it until vendors resolve their conflict.  In my opinion, this is 
completely unacceptable, specially for a Java packaging system, as the Java 
designers had already solved the problem for "Java  packages".

---

Using domain names approach is not a hierarchical approach: it is in fact an 
elegant solution to giving a UNIQUE private name space to every package 
provider, without the need for a "new" central authority.  In fact, every Java 
programmer has already to deal with the "domain name" central (in fact 
relatively distributed) authority.

Within that name space, every package provider can organize things as he wants. 
  No need to register a package anywhere when releasing it, ...    Obtaining a 
private name space is a ONE-TIME operation (preceedes even the work on writing 
the package!).  This is, in fact, the key for a flexible yet conflict free 
decentralized system.

---

The goal of UNIQUE canonical package names is to let RAK maintain a consistent 
repository on a user system.  Each package encodes its dependence using the 
canonical name.  This is not a problem at all for enc users, as they do not have 
to use these canonical names.

Once system consistency is assured, you provide nice nicknames to the user for 
managin his system.  These nickname will be proposed by package managers, and we 
hope that a minimum of conflicts will happen.  Now, the thing is that, if a 
conflict actually happens, then the user is simply asked to provide a different 
  system specific nickname for one of the two conflicting packages.  No harm 
done to the RAK repository, no broken application, no endless wait for providers 
to agree.

This, in my opinion, IS a truely non-hierarchical, yet "robust" system.  No need 
for a central "dispute resolution" authority (courts, or whatever to resolve 
heated disputes).  And, remember, once you have a private name space, you can 
orgnize it AS YOU WISH.  You can as many sub-levels as you want, e.g. 
com.mycompany.poducts.pojects.yooteeseeroooo.libs.whatever.the.package  [No, the 
RAK user won't have to type it! Unlike the Java programmer...]


> In fact, I think my "short unique names" model is a subset of your one, 
> a special case, where the long name is in fact a single word ;o)


I do not agree.  Your "short unique names" model allows for corrupt RAK 
repositories, mine does not.

> So, the only thing I am arguing against is having the additional level 
> of aliasing.

> 
> At the moment we have:
> name -> canonical name (name + version) -> long name



Lets agree on the terms:

Canonical name       == domain name based package name, e.g.: org.apache.ant
Short name/Nick name == user friendly name, e.g. ant, sablecc, rak


I am proposing this:

[Dependency encoding]
(some package) depends on (canonical name + CRA version)

[User view]
(short name) -> (anonical name + CRA version)

[Package encoding]
(some package) proposes short name: (short name)

In the cache, you store ".jar"s into their canonical form, avoiding name 
conflicts, e.g.:

org.sablecc.sablecc.jar.2.4.1
org.apache.ant.jar.4.0.0
...

User gets to write "rak ant"  and "rak sablecc".

If a conflict happens, RAK can do any of the following:
1- ask the user for giving a different nickname to either of the conflicting 
packages
2- let the amgiguity in the system, and simply ask question in ambiguous 
situations e.g.,
    $rak ant
     ** Do you want to run (a)org.apache.ant or (b)com.company.ant (a/b)?
    This is reasonible, as conflicts will be more frequent for support libraries
    which will never directly be called, and therefore won't need disambiguation.
3- force the user to use canonical names in case of conflicts...


Personally I am in favor of a combination of 1 and 2.  Asking for nicknames is 
mostly useful for packages that are executable.  2 is probably sufficient for 
libraries, as users will not directly manage them often.

 > Well, I do not know what to do. Should we resolve it before 1.0 is out?

I do think so.

----

Small detail, while I'm thinking of it:

When you write:
   $ rak sablecc

This means: run the highest installed SableCC package, right?  How do you run an 
older version?  Maybe:

   $ rak sablecc.2  [which happens to be: SableCC 2-> SableCC 4.0.2]

Is this what you have implemented?

----

I do not know if we should continue the name argument for long.  I think I have 
stated my mind pretty clearly.  You do as you wish; you are writing the code.  I 
simply feel that your reasons for not adopting the domain name approach are 
based on a confusion of hierarchical vs. distributed approaches.  All the 
distributed systems I have used had some form of unique ID generators.  Most of 
them use GUID, which are effective but unnatural for humans.  Domain names are 
much more elegant, and fit so well within the aready existing Java framework.

Have fun!

Etienne


-- 
Etienne M. Gagnon                    http://www.info.uqam.ca/~egagnon/
SableVM:                                       http://www.sablevm.org/
SableCC:                                       http://www.sablecc.org/

Re: [rak-developers] Circular deps

[Mariusz N. <ma...@ma...>]

Hi,

To make a brief summary:

Scripting: we have to discuss it further. If I will know how the Debian 
model works exactly, and I will know how flexible it is, we may consider 
using some predefined "policy" way (like Etienne is suggesting) of 
setting up the application. I will not make a 1.0 release without the 
decision here.


Naming: well, because the name is not a "parsable" item, and RAK treats 
it atomically, there is no problem or conflict between discussed ideas.
We may consider different "policies" of how this name should be 
generated by the module publishers. What I will put into the manual will 
be the "current" policy, we may change, adopt it once we reach the 
compromise. The 1.0 release is not waiting for the resolution on that.


Running conflicting modules at the same time: this is a big feature of 
RAK, and I will push this feature to go into RAK 1.0.  This is related 
to the hot-deployment feature which I do not think I will have time to 
work on now. Hot deployment to me means, that, if I am running 
application A from jar A.jar, and, on-the-fly, I will upgrade A.jar to 
the new version, the running application will use, for all new instances 
of its classes, the new A.jar.  Eventually this is a very important to 
us feature as well (upgrade of the system without the shutdown), but 
this feature is a little "hairy" and need big deal of thinking (the 
actual implementation will probably take 2 days I reckon ;o)
I may try to play with the prototype, but, I do not think we should wait 
for resolution on that before RAK 1.0


To excuse a little myself, I am working on emergent self-organizing 
systems, thus, I am biased against centralized and top-down systems 
design. So, please, do understand my problems with top-down designs ;o)
I am for letting users do as much and how they want it: no fixed 
policies. From the other hand i know it is not always the optimal way of 
doing things.   Of course, during the process I have agreed on several 
top-down things: versioning (no regular expression pattern matching as 
it was originally proposed, but fixed CRA model), moduleinfo file (no 
Manifest reuse).  Not that I have problem with that: I do not.  I know 
that top-down sometimes is the best we can do, like CRA versioning: it 
is simply brilliant ;o)
I like the current design a lot. But, it takes some time before I can 
"fit" my way of thinking into top-down policies, that's all.  So be 
patient, please ;o)


best regards
Mariusz

Re: [rak-developers] Circular deps

[Mariusz N. <ma...@ma...>]

Hi,

>> Well, that would work for all new applications designed this way.
>> But what about existing apps? They may have some custom ways of
>> handling configurations.
> 
> 
> This is the point of the Debian policy: to eliminate these "application 
> specific" differences.  For example, RAK could support "JDK properties" 
> based configuration, by providing XML tags for specifying them.  RAK 
> would not support installation-time scripting.  This would somehow 
> enforce the policy to to all configuration through XML tags & JDK 
> properties.
> 


So how Debian handles modules removal, e.g. if my app is creating some 
temporary cache files which are not really installed, but created when 
running,and which are completely useless without the app? How Deb will 
remove those files?  RPM simply removes those file via custom script.

How Deb discovers the deployment OS to set up preferences correctly to 
the default values?

How Deb is creating all necessary directory structures? Data files? How 
it checks for running deamons, net connections, etc?

Well it all can be done by the application itself upon first invocation, 
  but this is what I think the installing/removing process should handle.


> So, to "RAKify a package", one would need to:
> (1) add a CRA version (using RAK tools)
> (2) use JDK properties (and XML for installation time configuration).

> The XML tags could provide:
> (1) Cues for automatically proposing reasonible system specific defaults,
> (2) The option of asking questions to the user, when interactive tools 
> are used to install the package (e.g. GUI and [command-line + 
> --interactive]).


Well, assuming that all Java world will go with uniform Preferences 
model, this is quite tempting, regardles scripting. It would be nice to 
specify which and how the Preferences should be set and reset in 
install/remove. But, until JDK is oficially released we have to either 
wait or go with the Properties (which I do not like that much).


> For "older" packages that have their own configuration mechanism, we 
> could let RAK display a message to the end-user providing the necessary 
> directions to manually complete the application setup.  This is a 
> convenient way for us to eventually force people into adopting the 
> "policy" standard.


This is not too nice though. However, most of the old modules will not 
really do anything with scripting/Preferences anyway, so this is a quite 
reasonable.

----------------


> As I said before: I am in favor of:
> (1) Requiring canonical "unique" package names (from package managers).
> (2) Letting RAK users (and developers) use the short non-canonical name 
> of packages on the command-line (and even graphically) as long as no 
> conflict exists.


I agree that we should require UNIQUE names. But, whatever the model is, 
  we have no way of enforcing uniqueness. The canonical naming scheme is 
one way of increasing probability of uniqueness. Simple prefixes are 
another. Full domain name like names yet another.

 
> FYI: to answer your "Debian/ant" comment about "self regulation": Fixing 
> the problem was made possible only because Debian uses the "unique 
> central repository" approach; so all that was required was for a few 
> Debian developers to argue a name change.  


True, but I still think, even if Debian had distributed model, and the 
problem occured in one of the nodes, the fix would eventually made its 
way to all the nodes. The name change happens not in the node, it 
happens in the original development team. The important thing is a 
feedback loop - if I noticed two independent modules using the same 
name, I am reporting it to either development team, and they are fixing 
it. Feedback loop, self regulation, no central repository needed. No 
single convention needed.

Btw, the "ant build tool" module is in fact often refered as jakarta-ant 
(it is true that the jar is called ant.jar, but the project official 
name is jakarta-ant I believe).

> We do really need to address this problem before it actually happens.  


I fully agree. This is what I mean by asking for unique names. If 
someone calls something ANT it is like asking for trouble. It is known 
that common names are "common names". Calling it jakarta-ant or 
whatever-ant is all is needed for example. From the other hand if I call 
my module a really unique name, like "kawarau" or "waitaki" the 
probability of this name being unique is very high anyway.

> I think my proposal, with appropriate tool 
> support (by letting the tools use short non-canonical names) is the 
> right solution.


well, it assumes that there is no conflict, which is self contradicting.
If I have "ant" the build tool and "ant" the screensaver in my machine, 
I cannot use short names. I have to use the unique name. Your solution 
says that the package has two names, short, not that much unique, and 
longer, more probable to be unique.

I am saying, module name should be as much unique as developers think it 
should be. The "short" name should be unique. It does not contradict 
your proposal, it just makes it more flat. In case two unique short 
names are not unique RAK has problem. In case two short non-unique names 
are not unique RAK falls to use long name. In case two long names are 
not unique RAK has a problem.

 

In fact, I think my "short unique names" model is a subset of your one, 
a special case, where the long name is in fact a single word ;o)

So, the only thing I am arguing against is having the additional level 
of aliasing.

At the moment we have:
name -> canonical name (name + version) -> long name

the unique property is on the "name" level. The files are stored in 
canonical form, together with the version. Long name is a descriptive 
name for the GUI.
User interacts either using name, or the canonical name (if multiple 
versions are present at the same time). If the name is not unique, the 
module will overwrite existing one in the repository upon installation 
(which is called "a name conflict problem").

name can be:  ant,  jakarta-ant, org.jakarta.ant,  etc.  I do not care, 
it should be unique (as much as necessary), single word, no spaces, and 
easy to remember and type. Like a brand name. If there are two brand 
names, well, THEY have a problem. The problem must be reported back to 
the team and fixed by the team. RAK has nothing to do with branding.
(and I know that brand names work because of central registry - but this 
is only because "teams" are not cooperating, and "brands" are a valuable 
property.  If we assume brands do not have value, and it is in team 
interest to have a unique brand, then we do not have problem  ;o)


Well, I do not know what to do. Should we resolve it before 1.0 is out?

best regards
Mariusz