q-lang-users — general discussion, questions, suggestions, etc.

Re: [q-lang-users] CSV update

[Albert G. <Dr....@t-...>]

> A new source tarball (q-csv-0.2) is now available from the usual
> locations and I've updated the Windows MSI as well as the Debian eeePC
> package. I've also notified Toni Graffy, so a new q-addon SUSE rpm will
> hopefully be available soon.

Toni has updated the q-addons package, so the latest Q-CSV should now be
included. Thanks Toni!

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

Re: [q-lang-users] CSV update

[Albert G. <Dr....@t-...>]

Eddie Rucker wrote:
> A new version of CSV is in CVS. Several subtle bugs have been 
> eradicated. When the new version of the csv library is imported or 
> included, '\n' is the default line terminator for UNIX and MAC OS X 
> instead of '\r\n' as specified in RFC4180.

A new source tarball (q-csv-0.2) is now available from the usual
locations and I've updated the Windows MSI as well as the Debian eeePC
package. I've also notified Toni Graffy, so a new q-addon SUSE rpm will
hopefully be available soon.

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

[q-lang-users] CSV update

[Eddie R. <ed...@ri...>]

A new version of CSV is in CVS. Several subtle bugs have been 
eradicated. When the new version of the csv library is imported or 
included, '\n' is the default line terminator for UNIX and MAC OS X 
instead of '\r\n' as specified in RFC4180.

Eddie

Re: [q-lang-users] Q 7.11 on Mac OS X

[Albert G. <Dr....@t-...>]

John Cowan wrote:
> I've attached a trivial implementation of clock_gettime that I found in
> OpenBSM; it may be useful.

Thanks a bunch. clock_nanosleep shouldn't be much harder. I'll add those
emulations in the next release. I'll have to think what to do about
clock_getres, though, just assuming 1000 there probably won't be that
useful, do you maybe have some code for that, too?

For the time being, the following Q definitions should also do the trick
(minus the error checking, and it also adds some latency so
nanosleep_until won't be very accurate):

import system;
nanotime _ = round (time*1e9);
nanosleep _ T = sleep (T*1e-9);
nanosleep_until _ T = sleep (T*1e-9-time);

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

Re: [q-lang-users] Q 7.11 on Mac OS X

[John C. <co...@cc...>]

Albert Graef scripsit:

> Ryan Schmidt wrote:
> > "No manual entry for clock_gettime"
> 
> Ok, so that probably means that the highres timers aren't available.
> Anyway, I've corrected the checks for the availability of these and
> various related features, so that they should now properly detect their
> absence on OS X, as well as on older Cygwin systems.

I've attached a trivial implementation of clock_gettime that I found in
OpenBSM; it may be useful.

-- 
We call nothing profound                        co...@cc...
that is not wittily expressed.                  John Cowan
        --Northrop Frye (improved)

Re: [q-lang-users] Order of codes in the symbol table

[Albert G. <Dr....@t-...>]

Jir(í Spitz wrote:
> I would like to ask, whether is it safe to suppose, that a consequent 
> declarations of enumeration types will always result in a continuous 
> series of codes in the symbol table.
> 
> E.g. whether the declarations
> 
> public type X = const x1, x2;
> public type Y = const Y1, y2;
> (nothing else between that two declarations)
> 
> will always result in a continuous series
> 
> ....., exprtype(X), exprsym(x1), exprsym(x2), exprtype(Y), exprsym(y1), 
> exprsym(y2), .....

Yes, this should hold in the current implementation, since the symbol
table only grows and symbols never get "reused". But that might change
if I ever get around implementing dynamic loading (and unloading) of Q
modules at runtime.

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

Re: [q-lang-users] Q 7.11 on Mac OS X

[Albert G. <Dr....@t-...>]

Ryan Schmidt wrote:
> "No manual entry for clock_gettime"

Ok, so that probably means that the highres timers aren't available.
Anyway, I've corrected the checks for the availability of these and
various related features, so that they should now properly detect their
absence on OS X, as well as on older Cygwin systems.

Patch is attached. Ryan, could you please check whether this builds ok
now? Sorry for the hassle, I should maybe have pointed you to RC3 so
that we could have checked this before the final release. But I was
pretty sure that OS X would have these, as they've been in POSIX for a
while and AFAIK FreeBSD has them, too.

NOTE to OS X users: What this means for Q programmers on OS X is that
the new highres timers in the system module (Q functions nanotime,
nanosleep etc.) won't be available for that platform. Of course, you can
still use the built-in time and sleep functions, which will
automagically use the best timers available on a system (on Unix systems
without clock_gettime et al this will usually be the C functions
gettimeofday and nanosleep). Alas, these will only work with wall clock
time, whereas the new functions also offer a monotonic clock and
per-process/per-thread cpu clocks. But maybe someone feels inclined to
bug the Darwin developers about implementing clock_gettime and friends,
so that they will work in a future OS X version? ;-)

Cheers,
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

[q-lang-users] Order of codes in the symbol table

[Jiří S. <jir...@bl...>]

Hello Albert,

I would like to ask, whether is it safe to suppose, that a consequent 
declarations of enumeration types will always result in a continuous 
series of codes in the symbol table.

E.g. whether the declarations

public type X = const x1, x2;
public type Y = const Y1, y2;
(nothing else between that two declarations)

will always result in a continuous series

....., exprtype(X), exprsym(x1), exprsym(x2), exprtype(Y), exprsym(y1), 
exprsym(y2), .....

Thanks,

Jiri

Re: [q-lang-users] Q 7.11 on Mac OS X

[Ryan S. <rya...@us...>]

On Mar 5, 2008, at 17:31, Albert Graef wrote:

> Ryan Schmidt wrote:
>
>> system.c:1685: error: 'CLOCK_REALTIME' undeclared (first use in this
>> function)
>> system.c:1687: error: 'CLOCK_MONOTONIC' undeclared (first use in this
>> function)
>> system.c:1692: error: 'CLOCK_PROCESS_CPUTIME_ID' undeclared (first
>> use in this function)
>> system.c:1697: error: 'CLOCK_THREAD_CPUTIME_ID' undeclared (first use
>> in this function)
>> system.c: In function '__F__system_process_cpu_clockid':
>> system.c:5710: error: 'clockid_t' undeclared (first use in this
>> function)
>
> Yes, this stuff is new, but it should only be included if the
> appropriate interfaces are detected during configure and some special
> symbols (_POSIX_TIMERS et al) are defined in unicode.h. Maybe this  
> again
> needs some special defines on OSX to be enabled? What does 'man
> clock_gettime' say?

"No manual entry for clock_gettime"

Re: [q-lang-users] Q 7.11 on Mac OS X

[Albert G. <Dr....@t-...>]

Hi Ryan,

sorry for the long delay, I just returned home from a conference...

Ryan Schmidt wrote:
> system.c:1685: error: 'CLOCK_REALTIME' undeclared (first use in this  
> function)
> system.c:1687: error: 'CLOCK_MONOTONIC' undeclared (first use in this  
> function)
> system.c:1692: error: 'CLOCK_PROCESS_CPUTIME_ID' undeclared (first  
> use in this function)
> system.c:1697: error: 'CLOCK_THREAD_CPUTIME_ID' undeclared (first use  
> in this function)
> system.c: In function '__F__system_process_cpu_clockid':
> system.c:5710: error: 'clockid_t' undeclared (first use in this  
> function)

Yes, this stuff is new, but it should only be included if the
appropriate interfaces are detected during configure and some special
symbols (_POSIX_TIMERS et al) are defined in unicode.h. Maybe this again
needs some special defines on OSX to be enabled? What does 'man
clock_gettime' say?

Cheers,
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

[q-lang-users] Q 7.11 on Mac OS X

[Ryan S. <rya...@us...>]

Hello. Q 7.10 compiled just fine on Mac OS X 10.4.11 Intel, but 7.11  
doesn't. Here's what it does:


$ sudo port install q
Password:
--->  Fetching q
--->  Verifying checksum(s) for q
--->  Extracting q
--->  Configuring q
--->  Building q with target all
Error: Target org.macports.build returned: shell command " cd "/opt/ 
local/var/macports/build/_Users_rschmidt_macports_dports_lang_q/work/ 
q-7.11" && make all " returned error 2
Command output: creating clib.la
(cd .libs && rm -f clib.la && ln -s ../clib.la clib.la)
/bin/sh ../../libtool --tag=CC   --mode=compile /usr/bin/gcc-4.0 - 
DHAVE_CONFIG_H -I. -I../.. -I../../libq  -I/opt/local/include  -O2 - 
MT system.lo -MD -MP -MF .deps/system.Tpo -c -o system.lo system.c
  /usr/bin/gcc-4.0 -DHAVE_CONFIG_H -I. -I../.. -I../../libq -I/opt/ 
local/include -O2 -MT system.lo -MD -MP -MF .deps/system.Tpo -c  
system.c  -fno-common -DPIC -o .libs/system.o
system.c: In function 'ictowcs':
system.c:766: warning: passing argument 2 of 'libiconv' from  
incompatible pointer type
system.c: In function 'icfromwcs':
system.c:787: warning: passing argument 2 of 'libiconv' from  
incompatible pointer type
system.c: In function '__F__system_sys_vars':
system.c:1685: error: 'CLOCK_REALTIME' undeclared (first use in this  
function)
system.c:1685: error: (Each undeclared identifier is reported only once
system.c:1685: error: for each function it appears in.)
system.c:1687: error: 'CLOCK_MONOTONIC' undeclared (first use in this  
function)
system.c:1692: error: 'CLOCK_PROCESS_CPUTIME_ID' undeclared (first  
use in this function)
system.c:1697: error: 'CLOCK_THREAD_CPUTIME_ID' undeclared (first use  
in this function)
system.c: In function '__F__system_process_cpu_clockid':
system.c:5710: error: 'clockid_t' undeclared (first use in this  
function)
system.c:5710: error: parse error before 'id'
system.c:5711: error: 'id' undeclared (first use in this function)
system.c: In function '__F__system_thread_cpu_clockid':
system.c:5727: error: 'clockid_t' undeclared (first use in this  
function)
system.c:5727: error: parse error before 'id'
system.c:5728: error: 'id' undeclared (first use in this function)
system.c: In function '__F__system_iconv':
system.c:6208: warning: passing argument 2 of 'libiconv' from  
incompatible pointer type
make[4]: *** [system.lo] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Error: Status 1 encountered during processing.
$

Re: [q-lang-users] New Wiki article on AVL trees implementation

[Albert G. <Dr....@t-...>]

Jiri Spitz wrote:
> I have posted a new article on the Q wiki page trying to explain the 
> magic behind the improved AVL trees implementation.

Hi Jiri, thanks a lot for the lucid explanation of this fairly tricky
algorithm. BTW, here's the direct link to Jiri's article:
http://q-lang.wiki.sourceforge.net/Faster+implementation+of+AVL+trees

It's nice to see that pattern-matching rules can actually be employed
here to speed up the algorithm. The effect may be over-emphasized with Q
since Q uses bigint arithmetic, but still... Anyway, the elegance of the
Bird/Wadler implementation which was used previously is one thing, but
I'm sure happy that my dictionaries and sets run more than twice as fast
now. :)

Again, thanks for your work on this, it's much appreciated!

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

[q-lang-users] New Wiki article on AVL trees implementation

[Jiri S. <jir...@bl...>]

Hello Q users,

I have posted a new article on the Q wiki page trying to explain the 
magic behind the improved AVL trees implementation.

Jiri

[q-lang-users] Q 7.11 released

[Albert G. <Dr....@t-...>]

Hi all,

I've just uploaded the final Q 7.11 release. You can get it here:
http://q-lang.sourceforge.net/download.html

Here's the obligatory blurb from the NEWS file:

- As suggested by Rob Hubbard and John Cowan, special Unicode characters
in strings can now be specified using an escape of the form '\&name;',
where name is any of the XML entity names specified here:
http://www.w3.org/TR/2007/WD-xml-entity-names-20071214/

- clib now provides a set of additional int/float vector operations, as
suggested by John Cowan on the mailing list a while ago. These allow you
to treat byte strings as mutable C vectors of signed/unsigned 8/16/32
bit integers or float/double values. Operations to convert between such
C vectors and corresponding lists of integers/floating point values are
provided as well.

- The glob and regex functions were moved back from system into clib.
Also, a bug in the await function function was fixed (now it properly
accepts timeouts specified in seconds, not millisecs), and support for
mutex and semaphore timeouts was added to the try function (this
requires a system with pthread_mutex_timedlock/sem_timedwait).

- Added support for highres timers (clock_gettime et al) on systems
where the POSIX timer extension is available. The system module provides
a number of new functions (nanotime, nanosleep, etc.) to deal with
these. Moreover, the builtin time function will now also make use of
clock_gettime/gettimeofday if they are available.

- Worked around some quirks with the MS Access ODBC driver which caused
it to reject NULL and empty string parameter values in SQL insert
statements. (Bug reported by Jiri Spitz.) The odbc_examp.q script now
also works with MS Access on Windows. Moreover, as suggested by Tim
Haynes, there are a couple of new functions to retrieve additional
metadata about a data source.

- Jiri Spitz also contributed a much improved AVL tree implementation
for the container types (bag.q, dict.q, hdict.q, set.q). The improved
algorithm has its origin in the SWI-Prolog implementation of association
lists by R.A.O'Keefe and Jan Wielemaker (see assoc.pl at
http://www.swi-prolog.org). Jiri rewrote the algorithm in Q and also
added the deletion operations which were missing in the Prolog version.
Tests indicate that updating the new data structures is at least twice
as fast for large trees, quite a notable improvement!

Thanks to all who contributed code and participated in the discussions
leading up to this release!

Enjoy. :)
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

Re: [q-lang-users] CVS build broken on my system

[John C. <co...@cc...>]

Albert Graef scripsit:

> That looks like the libltdl devel stuff is missing in your installed
> libtool. Maybe Debian/Ubuntu has a separate package for that?

Bingo.  It's "libltdl3-dev", so after I had installed that with apt-get
all was well.

Thanks.

-- 
Verbogeny is one of the pleasurettes    John Cowan <co...@cc...>
of a creatific thinkerizer.             http://www.ccil.org/~cowan
   -- Peter da Silva

Re: [q-lang-users] CVS build broken on my system

[Albert G. <Dr....@t-...>]

John Cowan wrote:
> + libtoolize --automake --force --ltdl --copy
> ls: libltdl/*: No such file or directory
> libtoolize: cannot list files in `/usr/share/libtool/libltdl'

That looks like the libltdl devel stuff is missing in your installed
libtool. Maybe Debian/Ubuntu has a separate package for that?

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

Re: [q-lang-users] Association lists

[Albert G. <Dr....@t-...>]

Jir(í Spitz wrote:
> I thought that you put there my versions because they are bytewise 
> identical.

Coincidence. :)

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

[q-lang-users] CVS build broken on my system

[John C. <co...@cc...>]

When I try to build the CVS head on Gutsy Ubuntu from a clean directory,
I get the following from "sh autogen.sh":

+ aclocal
+ autoheader
+ libtoolize --automake --force --ltdl --copy
ls: libltdl/*: No such file or directory
libtoolize: cannot list files in `/usr/share/libtool/libltdl'
+ automake --gnu --add-missing --copy
configure.in:66: required file `./ltmain.sh' not found
+ autoconf
+ cd libltdl
+ autoconf
autoconf: no input file


Am I missing some dependency?  My automake version is 1.10. (Autotools builds, *sigh*)

-- 
Winter:  MIT,                                   John Cowan
Keio, INRIA,                                    co...@cc...
Issue lots of Drafts.                           http://www.ccil.org/~cowan
So much more to understand!
Might simplicity return?                        (A "tanka", or extended haiku)

Re: [q-lang-users] Association lists

[Jiří S. <jir...@bl...>]

Albert Graef wrote:
> Hmm, I already have that in cvs. Would you mind going over my version to
> see if it's ok with you? You can find it at
> http://q-lang.cvs.sourceforge.net/q-lang/q/stdlib/

I thought that you put there my versions because they are bytewise 
identical.

 > I also corrected a missing arg in hdict::deletea.

Thanks.

Jiri

Re: [q-lang-users] Association lists

[Albert G. <Dr....@t-...>]

Jiri Spitz wrote:
> I have corrected the issue with key comparisons in dict, set and bag.
> You will find them enclosed.

Hmm, I already have that in cvs. Would you mind going over my version to
see if it's ok with you? You can find it at
http://q-lang.cvs.sourceforge.net/q-lang/q/stdlib/

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

[q-lang-users] Q 7.11 RC3 available

[Albert G. <Dr....@t-...>]

I'm currently uploading Q 7.11 RC3, it should soon be available in
testing (the source tarball is already there):
http://sourceforge.net/project/showfiles.php?group_id=96881&package_id=188958

What's new in RC3: Jiri Spitz' much improved AVL tree implementation for
the container types (bag.q, dict.q, hdict.q, set.q). The improved
algorithm has its origin in the SWI-Prolog implementation of association
lists by R.A.O'Keefe and Jan Wielemaker (see assoc.pl at
http://www.swi-prolog.org). Jiri rewrote the algorithm in Q and also
added the deletion operations which were missing in the Prolog version.
Tests indicate that updating the new data structures is at least twice
as fast for large trees, quite a notable improvement!

Everyone who has programs making use of the bag, set, dict and hdict
data types, please take the new version for a test drive and report bugs
asap. Thanks. :)

Cheers,
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

Re: [q-lang-users] Association lists

[Albert G. <Dr....@t-...>]

Albert Graef wrote:
> I'll try to fix this (shouldn't be too difficult) and give it a first
> test drive with a couple of my programs. If that works ok I'll upload a
> new release candidate later today.

Ok, the key comparisons are fixed now, and I also corrected a missing
arg in hdict::deletea. Looks good so far! RC3 is on its way, stay tuned...

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

Re: [q-lang-users] Association lists

[Jiri S. <jir...@bl...>]

Albert Graef wrote:
> Nope, update takes three args, see stddecl.q. Citing the manual:
> 
>  update D X Y
> 
>     same as insert D (X,Y)
> 
> I.e., for dictionaries update is just a completely curried form of
> insert. Has always been that way.

You convinced me, thanks for your patience.

I have corrected the issue with key comparisons in dict, set and bag. 
You will find them enclosed.

Jiri

Re: [q-lang-users] Association lists

[Albert G. <Dr....@t-...>]

Jir(í Spitz wrote:
> Nonetheless, I discovered another small bug. In dict and similarly hdict 
> should be
> 
> update D:Dict (X, Y) = insert D (X,Y);
> or
> update D:Dict XY = insert D XY;
> 
> instead of
> 
> update D:Dict X Y = insert D (X,Y);

Nope, update takes three args, see stddecl.q. Citing the manual:

 update D X Y

    same as insert D (X,Y)

I.e., for dictionaries update is just a completely curried form of
insert. Has always been that way.

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

Re: [q-lang-users] Association lists

[Jiří S. <jir...@bl...>]

Albert Graef wrote:
> That was one of the bugs I fixed. (You forgot the type guards on the
> insert/delete interface functions, so the wrong equation would be
> invoked.) Get the latest versions from cvs, they should be ok.
.....
 >
 > Moreover, to facilitate testing and debugging, in the interpreter it
 > is possible to gain access to all public and private symbols of the
 > program (also in modules not directly imported in the main script)
 > using qualified identifiers.

I didn't read the docs with appropriate attention... :-(

Nonetheless, I discovered another small bug. In dict and similarly hdict 
should be

update D:Dict (X, Y) = insert D (X,Y);
or
update D:Dict XY = insert D XY;

instead of

update D:Dict X Y = insert D (X,Y);

Regards,

Jiri