q-lang-users

[q-lang-users] newbie question on io, etc

[<do...@ya...>]

Hi Q experts (and thank you Dr Graf for your creation)

I'm trying to learn Q. I know some Haskell and C,etc.

I realise that the following is the kind of problem
that imperative languages are better suited to, but
one can learn a lot from seeing how these icky cases
are dealt with in non-imperative languages (and the
inverse).

-----------------------------------------------------
Lets say I have a file containing 2 columns like this:

john    12
mary    56
peter   ab
vincent 24
lisa    34
.
.
.
etc.

I would like to read the file and output something
like:

min score == 12 (John)
max score == 56 (Mary)
avg score == 31.5

Illegal values:
ab in column 2 on line 3.

How would I do this in Q? I would especially like to
see an imperative solution if it exists.
-----------------------------------------------------

I thank you in advance for your assistance.

Dorothy K.


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com

Re: [q-lang-users] newbie question on io, etc

[<mip...@ya...>]

 --- s swami <do...@ya...> wrote:
> Lets say I have a file containing 2 columns like
> this:
> 
> john    12
> mary    56
> peter   ab
> vincent 24
> lisa    34
> ...
> etc.
> 
> I would like to read the file and output something
like:
> 
> min score == 12 (John)
> max score == 56 (Mary)
> avg score == 31.5
> 
> Illegal values:
> ab in column 2 on line 3.
> 
> How would I do this in Q? I would especially like to
> see an imperative solution if it exists.

I have one solution.


reviewf Path = review (split "\n" S)
                 where S:String = fget (fopen Path
"r");

// replace 'out of band' values to something
appropriate -- or maybe
// just write a 'firstdata'
review Lines = nextdata (Min,Max,Avg,Bad) Lines 1
                 where Min = (99999,""),
                   Max = (-99999,""),
                   Avg = (0.0,0),
                   Bad = "";

nextdata (Min,Max,Avg,Bad) [] _
  = printf "%s\n%s\n%s\n%s\n" (N,X,V,B)
      where N = sprintf "min score == %d (%s)" Min,
        X = sprintf "max score == %d (%s)" Max,
        V = sprintf "avg score == %.1f (from %d)" Avg,
        B = "\nIllegal values:\n" ++ Bad;
nextdata (Min,Max,Avg,Bad) [L|Ls] Ln = nextdata
(N,X,V,B) Ls (Ln + 1)
                                         where D =
split_data L,
                                           N = nextmin
Min D,
                                           X = nextmax
Max D,
                                           V = nextavg
Avg D,
                                           B = nextbad
Bad D Ln;

nextmin (N1,_) [S2,N2:Int] = (N2,S2) if N1 > N2;
nextmin Min _ = Min;
nextmax (N1,_) [S2,N2:Int] = (N2,S2) if N1 < N2;
nextmax Max _ = Max;
nextavg (V,N) [_,X:Int] = ((V*N+X)/N2,N2) where N2 = N
+ 1;
nextavg Avg _ = Avg otherwise;
nextbad B [_,N:Int] L = B;
nextbad B [S,number N] L = sprintf "%s%s on line %d\n"
(B,N,L);
nextbad B [_,malformed S] L
  = sprintf "%smalformed line on line %d: %s\n"
(B,L,S);

split_data S:String = [Name, number Data]  // 'number'
can fail: see 'nextbad'
                        where [Name, Data] = filter
not_empty (split " \t" S);
split_data S:String = ["", `malformed S];

number S = val S if alldigits S;

alldigits S = true if [true] = regex "" "^[0-9]+$" S
true;
            = false otherwise;

not_empty "" = false;
not_empty X = true;


________________________________________________________________________
Yahoo! Messenger - Communicate instantly..."Ping" 
your friends today! Download Messenger Now 
http://uk.messenger.yahoo.com/download/index.html

Re: [q-lang-users] newbie question on io, etc

[<Dr....@t-...>]

Hi, Dorothy, and welcome to this list. :)

s swami wrote:
> Lets say I have a file containing 2 columns like this:
> 
> john    12
> mary    56
> peter   ab
> vincent 24
> lisa    34
> .
> .
> .
> etc.
> 
> I would like to read the file and output something
> like:
> 
> min score == 12 (John)
> max score == 56 (Mary)
> avg score == 31.5
> 
> Illegal values:
> ab in column 2 on line 3.

First, with this simple input format, I'd say that you could simply use 
sscanf to do the parsing. E.g.:

parse S	= (NAME,VAL) where (NAME,VAL) = sscanf S "%s %d"; // good value
	= (NAME,VAL) where (NAME,VAL) = sscanf S "%s %s"; // bad value
	= (NAME,"") where NAME:String = sscanf S "%s"; // missing value
	= () otherwise; // empty line

This will return a data line (given as a string S) in the form of a pair 
(NAME,VAL) with VAL an integer if possible. Empty lines will be returned 
as an empty tuple.

Now it is easy to check for the "good" data:

good (_,VAL)
	= isint VAL;

With that out of the way, a straightforward solution employing the usual 
list voodoo would be:

foo	= report MIN MAX SUM (#GOOD) ||
	  writes "\nIllegal values:\n" ||
	  do (fprintf ERROR "bad value '%s' at line %d\n") BAD
	    where
	      // get the contents of the data file as a list of strings
	      DATA = split "\n" (fget (fopen "data.txt" "r")),
	      // parse the data, add line numbers in front of each tuple
	      DATA = zipwith push (map parse DATA) [1..#DATA],
	      // get rid of empty lines
	      DATA = filter (compose (neq ()) pop) DATA,
	      // filter the good lines, keep only the data
	      GOOD = filter good (map pop DATA),
	      // filter the bad lines, keep only value and line number
	      BAD = filter (compose (neg good) pop) DATA,
	      BAD = zip (map (!2) BAD) (map (!0) BAD),
	      // compute statistics on the good lines
	      MIN = foldl update_min () GOOD,
	      MAX = foldl update_max () GOOD,
	      SUM = sum (map (!1) GOOD);

(If you're running a Q version < 5.1, replace the [1..#DATA] with (nums 
1 (#DATA)) above.)

Here, I use the following helper functions to collect the min and max 
statistics:

update_min () DATA
	= DATA;
update_min (NAME,VAL) (NAME1,VAL1)
	= (NAME,VAL) if VAL1>=VAL;
	= (NAME1,VAL1) otherwise;
update_max () DATA
	= DATA;
update_max (NAME,VAL) (NAME1,VAL1)
	= (NAME,VAL) if VAL1<=VAL;
	= (NAME1,VAL1) otherwise;

Finally, here's a routine to report the results:

report _ _ _ 0
	= fwrites ERROR "no data\n";
report (MIN_NAME,MIN_VAL) (MAX_NAME,MAX_VAL) SUM COUNT
	= printf "min score == %d (%s)\nmax score == %d (%s)\n"
	  (MIN_VAL,MIN_NAME,MAX_VAL,MAX_NAME) ||
	  printf "avg score == %g\n" (SUM/COUNT);

I actually ran this on your input, and it seems to work. :) I get the 
following output:

==> foo
min score == 12 (john)
max score == 56 (mary)
avg score == 31.5

Illegal values:
bad value 'ab' at line 3

But you also asked for an imperative solution. Here's a fully 
tail-recursive implementation which does the same job, but only reads 
one line at at time (this one prints out any error messages in advance, 
fixing that is left as an exercise ;-). It uses the same helper 
functions from above.

bar	= process F 1 () () 0 0 where F:File = fopen "data.txt" "r";

process F I MIN MAX SUM COUNT
	= report MIN MAX SUM COUNT if feof F;
	= update F I MIN MAX SUM COUNT (parse (freads F)) otherwise;

// skip empty lines
update F I MIN MAX SUM COUNT ()
	= process F (I+1) MIN MAX SUM COUNT;
// collect statistics on good lines
update F I MIN MAX SUM COUNT DATA
	= process F (I+1) (update_min MIN DATA) (update_max MAX DATA)
	  (SUM+VAL) (COUNT+1) where (_,VAL) = DATA
	    if good DATA;
// report bad values
	= fprintf ERROR "bad value '%s' at line %d\n" (VAL,I) ||
	  process F (I+1) MIN MAX SUM COUNT where (_,VAL) = DATA
	    otherwise;

This just mimics an imperative procedure with local state variables for 
the current line number and the statistics info, executing a loop until 
feof F becomes true. Note that the Q interpreter automatically optimizes 
tail recursion, so the loop is in fact executed in constant stack space.

Hope this helps,
Albert

-- 
Dr. Albert Gr"af
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

Re: [q-lang-users] newbie question on io, etc

[<Dr....@t-...>]

Following up to my earlier mail...

> But you also asked for an imperative solution. [...]

There also is a "real" imperative solution, with variable updates and 
all that nasty stuff, using Q's expression references (a.k.a. mutable 
values, cf. 12.13 in the manual). Here goes:

// additional control constructs

special when ~P X, loop P X;

when P X
	= X if P;
	= () otherwise;

loop P X
	= X || loop P X if P;
	= () otherwise;

// split data lines into (NAME,VAL) pairs

parse S	= (NAME,VAL) where (NAME,VAL) = sscanf S "%s %d"; // good value
	= (NAME,VAL) where (NAME,VAL) = sscanf S "%s %s"; // bad value
	= (NAME,"") where NAME:String = sscanf S "%s"; // missing value
	= () otherwise; // empty line

// the analysis program, imperative style

baz	= loop (not feof F)
	  (put LINENO (get LINENO+1) ||
	   match (parse (freads F))
	   (case () (), // skip empty lines
	    case (NAME,VAL)
	    (ifelse (isint VAL)
	     // update statistics
	     (when (eq () (get MIN_VAL) or else (get MIN_VAL>VAL))
	      (put MIN_VAL VAL || put MIN_NAME NAME) ||
	      when (eq () (get MAX_VAL) or else (get MAX_VAL<VAL))
	      (put MAX_VAL VAL || put MAX_NAME NAME) ||
	      put SUM (get SUM+VAL) || put COUNT (get COUNT+1))
	     // report bad value
	     (fprintf ERROR "bad value '%s' at line %d\n" (VAL,get LINENO))
	    ))) ||
	  // report results
	  ifelse (get COUNT>0)
	  (printf "min score == %d (%s)\nmax score == %d (%s)\n"
	   (get MIN_VAL,get MIN_NAME,get MAX_VAL,get MAX_NAME) ||
	   printf "avg score == %g\n" (get SUM/get COUNT))
	  (fwrites ERROR "no data\n")
	    where F:File = fopen "data.txt" "r",
	      LINENO = ref 0, SUM = ref 0, COUNT = ref 0,
	      MIN_NAME = ref (), MIN_VAL = ref (),
	      MAX_NAME = ref (), MAX_VAL = ref ();

Plain ugly, but as you can see it's possible. ;-) By adding some more 
control structures, one could easily embed a complete imperative 
sublanguage in Q, very much like those found in ML and Haskell (without 
the syntactic sugar). But I see no real point in doing so. Mutable 
values can occasionally be useful to speed things up, though. (Note that 
these aren't even in the core language, rather they are provided by the 
clib module.)

Albert

-- 
Dr. Albert Gr"af
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag

Re: [q-lang-users] newbie question on io, etc

[<do...@ya...>]

 --- Albert Graef <Dr....@t-...> wrote: > Following
> up to my earlier mail...
> But you also asked for an imperative solution. [...]
> ...

Wow! Thanks to Albert and Julian.



Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com