q-lang-users — general discussion, questions, suggestions, etc.

Re: [q-lang-users] Trouble compiling Q

[Albert G. <Dr....@t-...>]

Sean E. Russell wrote:
> I'm merely reporting another failure to compile on AMD64; I know this is a 
> known problem[1].  [...]

Yeah, Q doesn't work on 64 bit processors right now. I'll see what I can 
do about this when the semester is over (11 days to go...).

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

[q-lang-users] Trouble compiling Q

[Sean E. R. <se...@ge...>]

Hiya,

I'm merely reporting another failure to compile on AMD64; I know this is a=
=20
known problem[1].  I wanted to add that it fails to compile under Gentoo in=
=20
32-bit compatibility mode (CFLAGS=3D-m32), because -- while it ignores most=
 of=20
the 64-bit compiled libraries -- it is picking up and trying to link agains=
t=20
libgmp, which is also in incompatible 64-bit format.  I'll probably poke=20
around with this some more, but if anybody else on 64-bit architecture gets=
=20
any further, please let me know.


The rest of this email is just information about how and when Q fails to=20
compile under 64-bit, so you can ignore it.  I'm putting the information in=
to=20
the ML archives for posterity.

When trying to compile in 64-bit mode the compilation gets all the way thro=
ugh=20
all of the core modules, and down to where it is trying to run the src/q=20
wrapper to evaluate qcwrap.q:

	...
	make[2]: Leaving directory `/home/ser/Desktop/q-6.2-1/modules'
	-- Making all in src
	make[2]: Entering directory `/home/ser/Desktop/q-6.2/src'
PATH=3D.:/usr/local/bin:/usr/bin:/bin:/opt/bin:/usr/x86_64-pc-linux-gnu/gcc=
=2Dbin/3.4.3:/usr/i686-pc-linux-gnu/gcc-bin/3.3:/opt/ati/bin:/opt/sun-jdk-1=
=2E5.0.01/bin:/opt/sun-jdk-1.5.0.01/jre/bin:/usr/qt/3/bin:/usr/kde/3.4/bin:=
/usr/kde/3.3/bin:/usr/games/bin:/usr/share/karamba/bin:/opt/limewire:/sbin:=
/usr/sbin=20
QPATH=3D../stdlib:../modules/clib:../modules/clib q ./qcwrap.q ./qcwrap.q
	def: error loading module
	Warning: 248 unresolved external symbols
	! File def, line 250: Value mismatch in definition
	make[2]: *** [qcwrap.c] Error 2
	make[2]: Leaving directory `/home/ser/Desktop/q-6.2/src'
	make[1]: *** [all-recursive] Error 1
	make[1]: Leaving directory `/home/ser/Desktop/q-6.2'
	make: *** [all] Error 2

I've never done anything with Q before, and don't know how to interpret the=
se=20
errors; neither q nor qcwrap.q have more than 212 lines, and I haven't dug=
=20
into the src/q shell script to track down where the problem could be=20
occuring.  I'm rather surprised that it gets this far.  It doesn't look lik=
e=20
a GCC compilation error, so I'm assuming gcwrap.q (or dependency) has some=
=20
32-bit specific code in it, and this is what is failing.

	Linux ender 2.6.11-gentoo-r11 #1 Fri Jun 24 08:26:09 EDT 2005 x86_64 AMD=20
Athlon(tm) 64 Processor 3400+ AuthenticAMD GNU/Linux

	Thread model: posix
	gcc version 3.4.3 20041125 (Gentoo Linux 3.4.3-r1, ssp-3.4.3-0, pie-8.7.7)

	model name      : AMD Athlon(tm) 64 Processor 3400+

	And this is q-6.2

Just for kicks, the libraries look OK:

	55 % LD_LIBRARY_PATH=3D./libq/.libs ldd src/.libs/lt-q
	        libq.so.6 =3D> ./libq/.libs/libq.so.6 (0x00002aaaaabc1000)
	        libgmp.so.3 =3D> /usr/lib/libgmp.so.3 (0x00002aaaaacf4000)
	        libdl.so.2 =3D> /lib/libdl.so.2 (0x00002aaaaae29000)
	        libpthread.so.0 =3D> /lib/libpthread.so.0 (0x00002aaaaaf2c000)
	        libreadline.so.5 =3D> /lib/libreadline.so.5 (0x00002aaaab0c1000)
	        libncurses.so.5 =3D> /lib/libncurses.so.5 (0x00002aaaab1fc000)
	        libcrypt.so.1 =3D> /lib/libcrypt.so.1 (0x00002aaaab356000)
	        libutil.so.1 =3D> /lib/libutil.so.1 (0x00002aaaab48a000)
	        libnsl.so.1 =3D> /lib/libnsl.so.1 (0x00002aaaab58e000)
	        libm.so.6 =3D> /lib/libm.so.6 (0x00002aaaab6a4000)
	        libc.so.6 =3D> /lib/libc.so.6 (0x00002aaaab82a000)
	        /lib64/ld-linux-x86-64.so.2 (0x00002aaaaaaab000)
	        libgpm.so.1 =3D> /lib/libgpm.so.1 (0x00002aaaaba51000)
	56 % ldd libq/.libs/libq.so.6
	        libcrypt.so.1 =3D> /lib/libcrypt.so.1 (0x00002aaaaabdf000)
	        libutil.so.1 =3D> /lib/libutil.so.1 (0x00002aaaaad13000)
	        libnsl.so.1 =3D> /lib/libnsl.so.1 (0x00002aaaaae17000)
	        libm.so.6 =3D> /lib/libm.so.6 (0x00002aaaaaf2d000)
	        libc.so.6 =3D> /lib/libc.so.6 (0x00002aaaab0b3000)
	        /lib64/ld-linux-x86-64.so.2 (0x0000555555554000)


[1]http://sourceforge.net/mailarchive/forum.php?thread_id=3D7131101&forum_i=
d=3D36790

=2D-- SER

"As democracy is perfected, the office of president represents,=20
more and more closely, the inner soul of the people.  On some=20
great and glorious day the plain folks of the land will reach=20
their heart's desire at last and the White House will be adorned=20
by a downright moron."        -  H.L. Mencken (1880 - 1956)

[q-lang-users] Q-OpenAL in cvs

[Albert G. <Dr....@t-...>]

Hi all,

a preliminary version of Q's upcoming OpenAL interface is now available 
in cvs (module q-openal) at http://q-lang.sf.net. OpenAL is a companion 
to OpenGL which allows you to do spatialized audio applications (games 
anyone? ;-) in a fashion analogous to OpenGL (you position the sound 
sources and a listener, and OpenAL renders the multichannel audio output 
for you). Two simple examples (which also require q-opengl) are included.

Enjoy! :)
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

Re: [q-lang-users] Smash version 0.2

[Peter M. <pet...@wa...>]

Albert Graef wrote:
> Hi, Peter,
> 
> thanks a lot for updating your package, works very well indeed. Do you 
> have plans to update json-q for Q 6.2 as well?

I just did, was just a matter of replacing % with // and the whole tuple 
thing.

> I think I will put the smash and bqd modules into a single module and 
> add that to the core package. For the time being, I added a pointer to 
> http://q-lang.sf.net/examples.html. Do you also have a personal homepage 
> that I could link to?

Nope.

Greetings,

Peter Minten

Re: [q-lang-users] Smash version 0.2

[Albert G. <Dr....@t-...>]

Peter Minten wrote:
> I've uploaded a new version of the serialization lib Smash. The 
> converter from and to a binary representation was rewritten in C. This 
> gives a pretty extreme speed improvement for large expressions (like a 
> relatively simple test expression put in a list 100 times). On the other 
> hand for smaller expressions the old implementation is pretty fast too.

Hi, Peter,

thanks a lot for updating your package, works very well indeed. Do you 
have plans to update json-q for Q 6.2 as well?

I think I will put the smash and bqd modules into a single module and 
add that to the core package. For the time being, I added a pointer to 
http://q-lang.sf.net/examples.html. Do you also have a personal homepage 
that I could link to?

Tim, so there you have your binary marshalling module. Now you just need 
to write the pvm interface. ;-)

> A word of warning: I'm not sure if it works well with 64 bits systems 
> (it could be that something serialized on a 64 bits system can't be 
> deserialized on a 32 system). But as Q doesn't have 64 bits support yet 
> that's not likely to cause major problems at the moment :-).

Well, we just got a new 64 bit machine, so I'm probably going to get 
serious about the 64 bit port real soon now(TM) ...

Cheers,
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

[q-lang-users] Smash version 0.2

[Peter M. <pet...@wa...>]

Hi,

I've uploaded a new version of the serialization lib Smash. The 
converter from and to a binary representation was rewritten in C. This 
gives a pretty extreme speed improvement for large expressions (like a 
relatively simple test expression put in a list 100 times). On the other 
hand for smaller expressions the old implementation is pretty fast too.

Anyway the whole lot is available at 
'http://www.il.fontys.nl/~silvernerd/q/'.

A word of warning: I'm not sure if it works well with 64 bits systems 
(it could be that something serialized on a 64 bits system can't be 
deserialized on a 32 system). But as Q doesn't have 64 bits support yet 
that's not likely to cause major problems at the moment :-).

Greetings,

Peter Minten

Re: [q-lang-users] Q 6.2 is out

[Tim H. <q...@st...>]

Albert Graef <Dr....@t-...> writes:

> Tim Haynes wrote:
> > Coolness. I guess I'll have to reorganize all my Q stuff for the
> > websites soon too - I need to switch to using libxml, unify versions of
> > cgi.q and other suchlike libraries into a central library .. and fix
> > the tuples as well. Beginning to add up to a decent hacking session ;)
> 
> cgi.q should be ok (at least the version included with the Q core package
> is), I checked that.

There'll be a better version - it's evolved POST and PATH_INFO handling in
the meantime :)

Cheers,

~Tim
-- 
<http://spodzone.org.uk/>

Re: [q-lang-users] Q 6.2 is out

[Albert G. <Dr....@t-...>]

/me wrote:
> today I've released Q 6.2.

Meanwhile, I've also updated Q-Graph and Q-Xine. Q-Faust will follow in 
due course, probably today. These required only minimal changes, due to 
the new tuple syntax. All the other packages should work just fine with 
Q 6.2.

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

Re: [q-lang-users] Q 6.2 is out

[Albert G. <Dr....@t-...>]

Tim Haynes wrote:
> Coolness. I guess I'll have to reorganize all my Q stuff for the websites
> soon too - I need to switch to using libxml, unify versions of cgi.q and
> other suchlike libraries into a central library .. and fix the tuples as
> well. Beginning to add up to a decent hacking session ;)

cgi.q should be ok (at least the version included with the Q core 
package is), I checked that.

Thanks to the migration path suggested by John, fixing the 1-tuples is a 
trivial business. Simply install Q 6.1 first, feed your scripts to it, 
and fix all places where the compiler warns you about the deprecated 
1-tuple syntax (adding a comma at the end of the tuple is enough). Your 
scripts will then run with Q 6.2 just fine.

BTW, despite all testing one error has crept in: The except.q script in 
the examples of the core package won't compile. The fix is trivial, and 
an updated version of the script is already in cvs.

>>public (xor) X Y @ 3; // 3 = precedence level
>>X xor Y = (X or Y) and not (X and Y);
> 
> Oh, jolly good. I was wondering how one did that, earlier :)

You couldn't. ;-) Even the (in) operator, which is used in the standard 
library to construct list and stream comprehensions, had to be 
hard-coded into the parser. I'm glad that this is fixed now.

Ok, the two remaining big issues now are unicode and 64 bit support. I 
plan to do this for Q 6.3. Back to coding...

Cheers,
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

Re: [q-lang-users] Q 6.2 is out

[Tim H. <q...@st...>]

Albert Graef <Dr....@t-...> writes:

> today I've released Q 6.2. As promised, this release finishes off the
> revision of the tuple syntax. 

Coolness. I guess I'll have to reorganize all my Q stuff for the websites
soon too - I need to switch to using libxml, unify versions of cgi.q and
other suchlike libraries into a central library .. and fix the tuples as
well. Beginning to add up to a decent hacking session ;)

> As a bonus, I've also implemented user-defined operators (this feature
> has been requested numerous times, and has actually been on my TODO list
> for several years ;-). E.g., to define yourself an infix exclusive-or
> operator you can now write:
> 
> public (xor) X Y @ 3; // 3 = precedence level
> X xor Y = (X or Y) and not (X and Y);

Oh, jolly good. I was wondering how one did that, earlier :)

~Tim
-- 
<http://spodzone.org.uk/>

[q-lang-users] Q 6.2 is out

[Albert G. <Dr....@t-...>]

Hi all,

today I've released Q 6.2. As promised, this release finishes off the 
revision of the tuple syntax. As a bonus, I've also implemented 
user-defined operators (this feature has been requested numerous times, 
and has actually been on my TODO list for several years ;-). E.g., to 
define yourself an infix exclusive-or operator you can now write:

public (xor) X Y @ 3; // 3 = precedence level
X xor Y = (X or Y) and not (X and Y);

More details here: http://q-lang.sourceforge.net/qdoc/qdoc_6.html#SEC23

Enjoy! :)
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

Re: [q-lang-users] ANN: Q 6.1 released

[Albert G. <Dr....@t-...>]

Albert Graef wrote:
> Work on Q 6.2 will start immediately, so if you find any bugs in this 
> release please let me know asap.

Ok, Q 6.2 is in cvs now and I'm ready to make a release. Did anyone test 
Q 6.1 yet? I'm holding back the new release for a few hours, so that you 
have an opportunity to post bug reports to this thread.

The only change in Q 6.2 is that the deprecated (X) syntax for 1-tuples 
has been disabled, so that (X) always denotes a parenthesized expression 
now. This finishes off the revision of the tuple syntax as suggested by 
John Cowan. (I decided against the other library changes I proposed 
earlier, since they would break too much other stuff.)

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

[q-lang-users] ANN: Q 6.1 released

[Albert G. <Dr....@t-...>]

Hi all,

Q 6.1 has just been released. This release sports some bug fixes, a new 
Haskell'ish infix application operator (as suggested by Tim Haynes) and 
a revision of the list/tuple syntax (as suggested by John Cowan).

NOTE: This is a transitional release. We're currently in the process of 
replacing Q's somewhat confusing 1-tuple syntax (X). The new syntax for 
1-tuples is (X,) -- note the trailing comma, just as in Python. Both the 
old (X) and the new (X,) syntax are supported in this release, but the 
former generates a warning message. Beginning with the next release, the 
new syntax will be _mandatory_, and the notation (X) will _always_ 
denote a parenthesized expression, so you should now start fixing your 
scripts accordingly.

Work on Q 6.2 will start immediately, so if you find any bugs in this 
release please let me know asap.

Enjoy! :)
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

Re: Building from cvs on MacOS X (was Re: [q-lang-users] [Fwd: [ q-lang-Bugs-1165977 ] batch behaviour of mismatched defs in imported modules])

[Albert G. <Dr....@t-...>]

Tim Haynes wrote:
> It appears AF_LOCAL is defined in sys/socket.h over here, so somewhere
> along the line the logic in clib.c's #idefs is failing, I suspect.

Hi Tim, looks like this is new, I never got this error on Jaguar and 
Panther. We've recently upgraded our G4 to Tiger, but I don't have the 
devkit installed yet, so it'll probably be a while until I can test on 
OSX again. :( If you find a fix in the meantime please let me know...

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

Re: [q-lang-users] [Fwd: [ q-lang-Bugs-1165977 ] batch behaviour of mismatched defs in imported modules]

[Tim H. <q...@st...>]

Albert Graef <Dr....@t-...> writes:

> Hi, Tim,
> 
> I finally fixed this bug, can you have a look at the latest cvs please and
> check whether it works for you?

Yup, happiness:

 | ==> import mod
 | ! File mod.q, line 3: Value mismatch in definition

Similarly accurate in batch-mode. 

Ta,

~Tim
-- 
<http://spodzone.org.uk/>

[q-lang-users] [Fwd: [ q-lang-Bugs-1165977 ] batch behaviour of mismatched defs in imported modules]

[Albert G. <Dr....@t-...>]

Hi, Tim,

I finally fixed this bug, can you have a look at the latest cvs please 
and check whether it works for you?

Albert

-------- Original Message --------
Subject: [ q-lang-Bugs-1165977 ] batch behaviour of mismatched defs in 
imported modules
Date: Sat, 02 Jul 2005 05:36:37 -0700
From: SourceForge.net <no...@so...>
To: no...@so...

Bugs item #1165977, was opened at 2005-03-18 15:53
Message generated for change (Comment added) made by agraef
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=616240&aid=1165977&group_id=96881

Please note that this message will contain a full copy of the comment 
thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Tim (spodzone)
Assigned to: Albert Graef (agraef)
Summary: batch behaviour of mismatched defs in imported modules

Initial Comment:
As per 423...@t-..., if you've got some
incorrect syntax in an imported module, such as

def pi=4*atan 1;

then you get your deserved "Value mismatch in
definition" error, but execution continues unabated. In
case of such an error, it should probably stop to
facilitate debugging in interactive mode, or abort
completely in batch mode.

----------------------------------------------------------------------

>Comment By: Albert Graef (agraef)
Date: 2005-07-02 14:36

Message:
Logged In: YES
user_id=873905

Fixed (kind of). The interpreter now recovers from such
errors (continuing execution of remaining initialization
code) and gives a proper error message with filename and
line number for each of them. In batch mode the interpreter
then exits with an exit code of 2.

Is that good enough?

Note that there is no way to report such errors in the
debugger, as no rule is executing and hence there's no stack
frame the debugger could be invoked upon. Thus I decided
that in interactive mode it would be more friendly to
continue to the command prompt, instead of bailing out.
After all these are only runtime errors, so when the
interpreter is running interactively, the user should be
given an opportunity to determine what went wrong and fix
things manually.


----------------------------------------------------------------------

Comment By: Tim (spodzone)
Date: 2005-03-18 15:54

Message:
Logged In: YES
user_id=546782

Also this is missing a file:line# so if I weren't aware of
which edits I was testing, it could be *anywhere* in the
source. Eek!

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=616240&aid=1165977&group_id=96881


-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

Re: [q-lang-users] RFC: Q 6.1/7.0 roadmap

[Tim H. <q...@st...>]

Albert Graef <Dr....@t-...> writes:

[snip]
> Will anyone here be upset by these additional changes? Anyone who's making
> heavy use of streams, cons or complex numbers?
> 
> 
> Comments appreciated.

Always feel a little bit weird about calling something cons, but then again
it sounds like prepend is actually a better name for it. So no problems
with the proposed changes, here.

~Tim
-- 
<http://spodzone.org.uk/>

[q-lang-users] RFC: Q 6.1/7.0 roadmap

[Albert G. <Dr....@t-...>]

Hi all,

those of you who are subscribed to q-lang-cvs have probably noticed that 
I have started working on Q 6.1. For this release I have the following 
user-visible changes on my TODO list:

- bug fixes (in particular, #1165977: better error diagnostics in init code)

- language: ($) operator as suggested by Tim Haynes; revised list and 
tuple syntax as suggested by John Cowan (see comments below)

- interpreter: collect local variables in an equation as soon as they're 
overridden with local variables with the same name

- library: some new functions which I found to be useful in many 
scripts, so I want to add them to the standard library:

   * cond.q: when and unless special forms:

     when P X = X if P; = () otherwise;
     unless P X = X if not P; = () otherwise;

   * new search.q script with binary and sequential search operations on 
lists

(Anything to be added to this list?)

As John suggested, Q 6.1 will feature an optional trailing comma in 
lists and tuples. The (X) syntax for denoting 1-tuples will be 
deprecated (although my heart bleeds over this ;-) and the compiler will 
warn about the use of this construct. This will give you (and me) the 
opportunity to fix your scripts by replacing (X) with the new 
recommended syntax for 1-tuples, (X,).

My plan is to release Q 7.0 shortly afterwards, where the new 1-tuple 
syntax will be _mandatory_, and the construct (X) will _always_ be a 
parenthesized expression. Since this breaks backward compatibility 
anyway, I'd like to take the opportunity to mess things up a little 
more, by making two changes to the library which have been on my TODO 
list for a while:

- Rename the Stream::bin constructor to cons. The name bin is really 
misleading as it suggests a kind of tree data structure, and cons is the 
traditional name for this kind of constructor. Consequently, I will have 
to rename the stdlib::cons function to something else, probably prepend. 
(Or could we just agree on using flip push instead?) This will affect 
all scripts which use either the Stream data structure or the former 
cons function.

- Make complex numbers an algebraic data type, type Complex = const 
complex X Y. Currently, they're represented as pairs of numbers (X,Y). 
While this notation is more convenient, it precludes other potential 
uses of tuples as vectors, matrices etc. I'd also like to add a new Rat 
type for (arbitrary precision) rational numbers which will be 
implemented in an analogous fashion: type Rat = const rat X Y.

Will anyone here be upset by these additional changes? Anyone who's 
making heavy use of streams, cons or complex numbers?

Comments appreciated.

Cheers,
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

Re: [q-lang-users] nutty idea?

[Tim H. <q...@st...>]

Albert Graef <Dr....@t-...> writes:

> Tim Haynes wrote:
> > Ah, enlightenment dawns, I think: I'm comparatively used to PVM being
> > implemented a layer under what I'm trying to do
> 
> I'd vote for wrapping the entire PVM. That'd give you the full power of the
> interface, and it's always possible to add an abstraction layer on top of
> that.

Yup, agreed now. Next stop, seeing how complex the pvm API is ...

> > As long as we're talking ability to take Q expressions, prefix them
> > with a function call citing a node on which to be evaluated and off it
> > goes, this is cool. Something like, perhaps:
> 
> Well, as Q normally evaluates all expressions sequentially, in order to
> achieve the desired parallelism the remote call should rather kick off a
> computation in another thread, and then you collect the results, like so:

Funnily enough I thought of this last night shortly after that mail, too.
OK, now we're cooking. :)
[snip]
> > Now what have I got into? If it were down to me, it'd be yeeeeears :/
> > Help! :)
> 
> Learn Q-SWIG first. With that, creating the necessary modules should be a
> matter of hours, not years. ;-)

Eek. OK, but I still think I'll come in last ;)

~Tim
-- 
<http://spodzone.org.uk/>

Re: [q-lang-users] nutty idea?

[Albert G. <Dr....@t-...>]

Tim Haynes wrote:
> Ah, enlightenment dawns, I think: I'm comparatively used to PVM being
> implemented a layer under what I'm trying to do

I'd vote for wrapping the entire PVM. That'd give you the full power of 
the interface, and it's always possible to add an abstraction layer on 
top of that.

> As long as we're talking ability to take Q expressions, prefix them with a
> function call citing a node on which to be evaluated and off it goes, this
> is cool. Something like, perhaps:

Well, as Q normally evaluates all expressions sequentially, in order to 
achieve the desired parallelism the remote call should rather kick off a 
computation in another thread, and then you collect the results, like so:

     takfp (X, Y, Z) = Z if Y>=X;
                 = takfp (result H1, result H2, result H3)
		    where H1 = thread (remotetakfp (X-1.0, Y, Z)),
                       H2 = thread (remotetakfp (Y-1.0, Z, X)),
                       H3 = thread (remotetakfp (Z-1.0, X, Y))
                     otherwise;

That way all remote computations in the child nodes go off immediately, 
and the local threads H1..H3 just sit there waiting for the results to 
arrive. You then apply the result function from clib to extract the 
results from the threads (this will block until all three results have 
arrived).

Of course you could also wrap up the parallelism machinery in the 
preceding example in a higher-order function, say:

parallel F Xs:Tuple
		= tuple (map result Hs)
		    where Hs = map (thread.F) (list Xs);

Then you could write something like:

     takfp (X, Y, Z) = Z if Y>=X;
                 = takfp (parallel remotetakfp
                   ((X-1.0, Y, Z),(Y-1.0, Z, X),(Z-1.0, X, Y)));

That's just one way to go about this, but you get the idea...

> Sounds like it'll be a bit of work to abstract it up to the level above :)

I don't think that it would be all that hard. You'd have to add some 
boilerplate code for managing the parent and child nodes, maybe a simple 
load balancing algorithm. That's probably not more than a few lines of 
Q. The "remote eval" function itself would then essentially be a one 
liner (to be wrapped up in a thread of its own, as pointed out above): 
pick a child node, serialize the input expression, send it over to the 
child node, receive the response, de-serialize it and return the 
resulting expression.

> Now what have I got into? If it were down to me, it'd be yeeeeears :/ 
> Help! :)

Learn Q-SWIG first. With that, creating the necessary modules should be 
a matter of hours, not years. ;-)

Gotta run...
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

Re: [q-lang-users] nutty idea?

[Tim H. <q...@st...>]

Albert Graef <Dr....@t-...> writes:

> Tim Haynes wrote:
>
> > Been pondering. Over in the land of erlang, I gather they have ability
> > to distribute computation all over a cluster/network. Seems quite nice:
> 
> Hi Tim,
> 
> actually I've been tossing around the idea of an MPI module for a while
> now. So I don't think it's a nutty idea at all. Having taken a brief look
> at PVM, it seems even better suited for this kind of thing.

Ah, excellent :) 
 
> As Q already has all the stuff needed for communicating over sockets, you
> can even do something like this right now. See dgram.q in the core
> distribution for a (very simple) example. This uses Q's builtin
> text-based serialization (str/val) for communicating Q values over a
> socket. Of course this is not ideal; 

Yes, I was wondering about evolving something usable from a network-repl
bit of code upwards.

> it's ok for simple applications, but for passing around big chunks of Q
> data we certainly need a binary serialization protocol which uses network
> byte order for the numbers. Something along the lines of Peter Minten's
> binary marshalling library (written in Q), but implemented in C for
> maximum efficiency.

Oh, that's a good idea. Slipped the mind, oops..

> And then, as you point out below, we need a decent infrastructure for
> distributing the tasks. PVM might be the way to go there.

Ah, enlightenment dawns, I think: I'm comparatively used to PVM being
implemented a layer under what I'm trying to do - my experience is greatest
with POV-ray, where it uses PVM to say "you, go render the top-left block
of the image", and so on - but my interface to pvm-povray is just the same
as to povray itself, apart from setting up all the boxes to run the
utility. Hence I was thinking of PVM as taking typical chunks of Q
interpreter code - `run the parser over here' versus `evaluate it over
there'. Obviously, it doesn't need to be so :)

> > * a connectivity daemon, * network protocol for conveying requests &
> > results back & forth, * a system for identifying a node
> > (`tim@somebox'), * a shared-secret file (a la .Xauthority) for
> > security, * node-availability management, and * an easy syntax for
> > remoteifying expression-evaluation.
> 
> > Guess what: I'm about to ask how hard it would be to slam all this into
> > a world-dominating future release of Q. :)
> 
> Well, as I pointed out above, IMHO nothing needs to be changed inside the
> interpreter to implement all this (minus the special syntax, but I don't
> think that this is really needed). 

Well, over in erlang it looks like almost any other function-call, to me -
it just takes a node, a lambda and some parameters as its args and makes
the whole evaluation happen "over there". So it's not "special" in that
regard at all; FFI-ing in PVM with SWIG would be a good start.

> What we need is (1) a serialization module and (2) a wrapper for PVM.
> With SWIG (2) is probably quite easy. (1) might be a bit tricky, since we
> need a secure way to pass around function symbols between different
> scripts running on different machines.

Yup.

> Given these two things, the rest of the required functionality which is
> not provided by PVM (authentication, maybe?) could then also be
> programmed in Q.

PVM operates over rsh/ssh, so I doubt authentication will be a problem :)

> > I'm also wondering if PVM could help out, but unless the nature of Q
> > lends itself to chunks of interpretation being carried-out on different
> > machines at an internal level, I'd doubt it was that much use.
> 
> Why would you need that kind of special support from the interpreter?
> We're not talking about fine-grained parallelism here. 

As long as we're talking ability to take Q expressions, prefix them with a
function call citing a node on which to be evaluated and off it goes, this
is cool. Something like, perhaps:

    randnode = Ns ! (random mod #Ns) 
                 where Ns=remote_nodes;

    remotetakfp = spawn_link randnode takfp ;

    takfp (X, Y, Z) = Z if Y>=X;
                = takfp (
                   remotetakfp (X-1.0, Y, Z),
                   remotetakfp (Y-1.0, Z, X),
                   remotetakfp (Z-1.0, X, Y)) 
                     otherwise;

    takstart N = takfp (N*3.0, N*2.0, N*1.0);

I should probably consider it a mini-project to implement something like
the above in terms of dgram.q, I think. Data is simple enough, at least.

> As I imagine it, in Q you'd just call the PVM functions, like you'd do in
> a C program. You'd pass around serialized Q data in PVM buffers to send
> the jobs (e.g., unevaluated Q expressions) and receive the results (e.g.,
> the evaluated expressions). Pretty much like in the dgrams.q example.
> Using these primitives, it should be quite easy to implement any kind of
> remote_eval function that you can think of.

Sounds like it'll be a bit of work to abstract it up to the level above :)

> Any volunteers? ;-)

Now what have I got into? If it were down to me, it'd be yeeeeears :/ 
Help! :)

~Tim
-- 
<http://spodzone.org.uk/>

Re: [q-lang-users] nutty idea?

[Albert G. <Dr....@t-...>]

Tim Haynes wrote:
> Been pondering. Over in the land of erlang, I gather they have ability to
> distribute computation all over a cluster/network. Seems quite nice:

Hi Tim,

actually I've been tossing around the idea of an MPI module for a while 
now. So I don't think it's a nutty idea at all. Having taken a brief 
look at PVM, it seems even better suited for this kind of thing.

As Q already has all the stuff needed for communicating over sockets, 
you can even do something like this right now. See dgram.q in the core 
distribution for a (very simple) example. This uses Q's builtin 
text-based serialization (str/val) for communicating Q values over a 
socket. Of course this is not ideal; it's ok for simple applications, 
but for passing around big chunks of Q data we certainly need a binary 
serialization protocol which uses network byte order for the numbers. 
Something along the lines of Peter Minten's binary marshalling library 
(written in Q), but implemented in C for maximum efficiency. And then, 
as you point out below, we need a decent infrastructure for distributing 
the tasks. PVM might be the way to go there.

>   * a connectivity daemon, 
>   * network protocol for conveying requests & results back & forth, 
>   * a system for identifying a node (`tim@somebox'), 
>   * a shared-secret file (a la .Xauthority) for security, 
>   * node-availability management, and 
>   * an easy syntax for remoteifying expression-evaluation.
> 
> Guess what: I'm about to ask how hard it would be to slam all this into a
> world-dominating future release of Q. :)

Well, as I pointed out above, IMHO nothing needs to be changed inside 
the interpreter to implement all this (minus the special syntax, but I 
don't think that this is really needed). What we need is (1) a 
serialization module and (2) a wrapper for PVM. With SWIG (2) is 
probably quite easy. (1) might be a bit tricky, since we need a secure 
way to pass around function symbols between different scripts running on 
different machines.

Given these two things, the rest of the required functionality which is 
not provided by PVM (authentication, maybe?) could then also be 
programmed in Q.

> I'm also wondering if PVM could help out, but unless the nature of Q lends
> itself to chunks of interpretation being carried-out on different machines
> at an internal level, I'd doubt it was that much use. 

Why would you need that kind of special support from the interpreter? 
We're not talking about fine-grained parallelism here. As I imagine it, 
in Q you'd just call the PVM functions, like you'd do in a C program. 
You'd pass around serialized Q data in PVM buffers to send the jobs 
(e.g., unevaluated Q expressions) and receive the results (e.g., the 
evaluated expressions). Pretty much like in the dgrams.q example. Using 
these primitives, it should be quite easy to implement any kind of 
remote_eval function that you can think of.

Any volunteers? ;-)

Cheers,
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

[q-lang-users] nutty idea?

[Tim H. <q...@st...>]

Hi folks,

Been pondering. Over in the land of erlang, I gather they have ability to
distribute computation all over a cluster/network. Seems quite nice:

  * a connectivity daemon, 
  * network protocol for conveying requests & results back & forth, 
  * a system for identifying a node (`tim@somebox'), 
  * a shared-secret file (a la .Xauthority) for security, 
  * node-availability management, and 
  * an easy syntax for remoteifying expression-evaluation.

Guess what: I'm about to ask how hard it would be to slam all this into a
world-dominating future release of Q. :)

I'm also wondering if PVM could help out, but unless the nature of Q lends
itself to chunks of interpretation being carried-out on different machines
at an internal level, I'd doubt it was that much use. 

Any thoughts?

~Tim
-- 
<http://spodzone.org.uk/>

Re: [q-lang-users] Types and multiple inheritance

[Albert G. <Dr....@t-...>]

Hi, John,

John.Cowan wrote:
> As I understand it, variable references with a type guard T match only
> expressions whose operators are declared in the type declaration for T
> or inherited from T's supertype, if any.

s/operators/constructors/

Nope, it's just the other way round. A variable of type T matches the 
constructor patterns of T itself, as well as those of all its 
_subtypes_. So that effectively makes the supertype the union of all of 
its subtypes.

Conversely, with multiple inheritance you'd have that a certain 
constructor pattern would match different supertypes (i.e., the common 
subtype belongs to the intersection of its supertypes). That does make 
perfect sense. (It won't buy you much, though, unless you design all 
involved datatypes as ADTs, but I take this for granted when you're 
programming the OOP way.)

At least the pattern matching automaton generator algorithm in the 
bytecode compiler and the runtime symbol table would have to be changed 
to accommodate for this. That should be doable, but, as it won't be a 
walk in the park to implement this in an efficient manner, please 
understand that it's not very high on my priority list, until someone 
comes up with a good usage scenario. ;-)

Cheers,
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

Re: [q-lang-users] A proposal to adopt Python tuple syntax

[Albert G. <Dr....@t-...>]

John.Cowan wrote:
> It's a matter of how you conceptualize the optional trailing comma in tuples
> other than 1-tuples.  If the rule is "a comma is permitted after the last
> element", then "(,)" is illegal.  If the rule is "a comma is permitted before
> the right parenthesis", then "(,)" is legal and means the same as "()".
> (And the same reasoning for lists.)

I vote for the former (no (,) allowed).

> As I say, I don't care how this one is decided.

So it's decided then? Anyone else?

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag