Re: [q-lang-users] newbie question on io, etc

[<Dr....@t-...>]

Hi, Dorothy, and welcome to this list. :)

s swami wrote:
> Lets say I have a file containing 2 columns like this:
> 
> john    12
> mary    56
> peter   ab
> vincent 24
> lisa    34
> .
> .
> .
> etc.
> 
> I would like to read the file and output something
> like:
> 
> min score == 12 (John)
> max score == 56 (Mary)
> avg score == 31.5
> 
> Illegal values:
> ab in column 2 on line 3.

First, with this simple input format, I'd say that you could simply use 
sscanf to do the parsing. E.g.:

parse S	= (NAME,VAL) where (NAME,VAL) = sscanf S "%s %d"; // good value
	= (NAME,VAL) where (NAME,VAL) = sscanf S "%s %s"; // bad value
	= (NAME,"") where NAME:String = sscanf S "%s"; // missing value
	= () otherwise; // empty line

This will return a data line (given as a string S) in the form of a pair 
(NAME,VAL) with VAL an integer if possible. Empty lines will be returned 
as an empty tuple.

Now it is easy to check for the "good" data:

good (_,VAL)
	= isint VAL;

With that out of the way, a straightforward solution employing the usual 
list voodoo would be:

foo	= report MIN MAX SUM (#GOOD) ||
	  writes "\nIllegal values:\n" ||
	  do (fprintf ERROR "bad value '%s' at line %d\n") BAD
	    where
	      // get the contents of the data file as a list of strings
	      DATA = split "\n" (fget (fopen "data.txt" "r")),
	      // parse the data, add line numbers in front of each tuple
	      DATA = zipwith push (map parse DATA) [1..#DATA],
	      // get rid of empty lines
	      DATA = filter (compose (neq ()) pop) DATA,
	      // filter the good lines, keep only the data
	      GOOD = filter good (map pop DATA),
	      // filter the bad lines, keep only value and line number
	      BAD = filter (compose (neg good) pop) DATA,
	      BAD = zip (map (!2) BAD) (map (!0) BAD),
	      // compute statistics on the good lines
	      MIN = foldl update_min () GOOD,
	      MAX = foldl update_max () GOOD,
	      SUM = sum (map (!1) GOOD);

(If you're running a Q version < 5.1, replace the [1..#DATA] with (nums 
1 (#DATA)) above.)

Here, I use the following helper functions to collect the min and max 
statistics:

update_min () DATA
	= DATA;
update_min (NAME,VAL) (NAME1,VAL1)
	= (NAME,VAL) if VAL1>=VAL;
	= (NAME1,VAL1) otherwise;
update_max () DATA
	= DATA;
update_max (NAME,VAL) (NAME1,VAL1)
	= (NAME,VAL) if VAL1<=VAL;
	= (NAME1,VAL1) otherwise;

Finally, here's a routine to report the results:

report _ _ _ 0
	= fwrites ERROR "no data\n";
report (MIN_NAME,MIN_VAL) (MAX_NAME,MAX_VAL) SUM COUNT
	= printf "min score == %d (%s)\nmax score == %d (%s)\n"
	  (MIN_VAL,MIN_NAME,MAX_VAL,MAX_NAME) ||
	  printf "avg score == %g\n" (SUM/COUNT);

I actually ran this on your input, and it seems to work. :) I get the 
following output:

==> foo
min score == 12 (john)
max score == 56 (mary)
avg score == 31.5

Illegal values:
bad value 'ab' at line 3

But you also asked for an imperative solution. Here's a fully 
tail-recursive implementation which does the same job, but only reads 
one line at at time (this one prints out any error messages in advance, 
fixing that is left as an exercise ;-). It uses the same helper 
functions from above.

bar	= process F 1 () () 0 0 where F:File = fopen "data.txt" "r";

process F I MIN MAX SUM COUNT
	= report MIN MAX SUM COUNT if feof F;
	= update F I MIN MAX SUM COUNT (parse (freads F)) otherwise;

// skip empty lines
update F I MIN MAX SUM COUNT ()
	= process F (I+1) MIN MAX SUM COUNT;
// collect statistics on good lines
update F I MIN MAX SUM COUNT DATA
	= process F (I+1) (update_min MIN DATA) (update_max MAX DATA)
	  (SUM+VAL) (COUNT+1) where (_,VAL) = DATA
	    if good DATA;
// report bad values
	= fprintf ERROR "bad value '%s' at line %d\n" (VAL,I) ||
	  process F (I+1) MIN MAX SUM COUNT where (_,VAL) = DATA
	    otherwise;

This just mimics an imperative procedure with local state variables for 
the current line number and the statistics info, executing a loop until 
feof F becomes true. Note that the Q interpreter automatically optimizes 
tail recursion, so the loop is in fact executed in constant stack space.

Hope this helps,
Albert

-- 
Dr. Albert Gr"af
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikwissenschaft.uni-mainz.de/~ag