Re: [Clus-devel] Clus sparse ARFF file implementation

[Bernard Z. <ber...@ij...>]

Pascal,

many thanks for the workaround instructions, they are sure useful until 
we get this fixed.

The output to the console during learning was meant for debugging 
purposes. You can influence the amount of output a bit with the 
"[General] Verbose" setting in the *.s file, but I'm afraid that not all 
learning algorithms use this setting in a uniform way (if at all). In 
tree learning, the values of test evaluation heuristic are being output 
by default.

Cheers, Bernard

On 27.2.13 13:35, Pascal Brandt wrote:
> Hi Bernard,
>
> I wrote the script below [code segment 1] to convert my ARFF file from
> 0-based to 1-based, but I was still having problems because my label
> attributes were not being read properly. I have 11 categorical labels
> with the possible values {0, 1}. Weka generates a sparse ARFF file where
> only the 1 values are actually written to file.
>
> The only way I could get Clus to work properly was to convert my dataset
> to non-sparse before exporting my ARFF file [code segment 2].
>
> I hope this helps someone else out in the future.
>
> Ciao,
> Pascal
>
> p.s. Is there any documentation regarding the information that gets
> dumped to the console when generating trees? It seems only the contents
> of the output file are (somewhat) documented?
>
> [code segment 1]
> #!/bin/bash
>
> cat $1 | gawk '
> BEGIN { FS = ","; found_data="FALSE" }; {
> if(found_data == "FALSE") {
> print $0
> if($1 == "@data")
> found_data="TRUE"
> } else {
> for (i = 1; i <= NF; i++) {
> matched_attr_index_str = gensub(/([0-9]+)/, "\\1", 1, $i)
> matched_attr_index = strtonum(matched_attr_index_str)
> matched_attr_index++
> new_str = gensub(/([0-9]+)/, matched_attr_index, 1, $i)
> printf new_str
> if(i == NF) {
> print ""
> } else {
> printf ","
> }
>          }
> }
> }'
>
> [code segment 2]
> Instances newData = null;
>
> try {
>          SparseToNonSparse stns = new SparseToNonSparse();   // new
> instance of filter
>          stns.setInputFormat(trainingData);                  // inform
> filter about dataset
>          newData = Filter.useFilter(trainingData, stns);   // apply filter
> } catch (Exception e) {
> logger.info <http://logger.info>("Error converting from sparse to
> non-sparse: " + e.getMessage());
> }
>
>
> On 27 February 2013 14:25, Bernard Zenko <ber...@ij...
> <mailto:ber...@ij...>> wrote:
>
>     Dear Pascal,
>
>     many thanks for this bug report! At the moment, we're not using any
>     bug tracking system, so clus-devel mailing list is the right address
>     report bugs.
>
>     Regards, Bernard
>
>
>
>     On 26.2.13 12:12, Pascal Brandt wrote:
>
>         Hi,
>
>         Firstly, my apologies if I'm directing this email to the wrong
>         audience.
>         I've just tried to use Clus with a sparse ARFF file and have
>         seen that
>         it uses a 1-based indexing system for the attributes as opposed
>         to the
>         0-based system defined here
>         <http://weka.wikispaces.com/__ARFF+%28book+version%29
>         <http://weka.wikispaces.com/ARFF+%28book+version%29>>. If there's a
>
>         issue/bug tracking system used to manage development of this
>         project I'd
>         be happy to log a bug for this.
>
>         Regards,
>         Pascal
>
>
>         ------------------------------__------------------------------__------------------
>         Everyone hates slow websites. So do we.
>         Make your web apps faster with AppDynamics
>         Download AppDynamics Lite for free today:
>         http://p.sf.net/sfu/appdyn___d2d_feb
>         <http://p.sf.net/sfu/appdyn_d2d_feb>
>
>
>
>         _________________________________________________
>         Clus-devel mailing list
>         Clu...@li....__net
>         <mailto:Clu...@li...>
>         https://lists.sourceforge.net/__lists/listinfo/clus-devel
>         <https://lists.sourceforge.net/lists/listinfo/clus-devel>
>
>