Re: [Jboost-cvs] jboost/src/jboost/tokenizer LTStreamTokenizer.java, 1.1.1.1, 1.2

[Yoav F. <yoa...@gm...>]

Hi William,

I would like to use comments to trace back examples with small  
margins. I see that you fixed a bug in that regard,
but I am not sure what is the specification for comments, it is not  
documented in

http://jboost.sourceforge.net/doc.html#input

Is a comment anything that appears after the example terminator symbol  
in the line? Does it need to have // in front of it (lines that start  
with // are ignored). Does jboost store the comment somewhere or can  
you retrieve the comment based on the example index number in the  
jboost log file?

Yoav

On May 16, 2008, at 2:50 PM, william beaver wrote:

> Update of /cvsroot/jboost/jboost/src/jboost/tokenizer
> In directory sc8-pr-cvs17.sourceforge.net:/tmp/cvs-serv10384/src/ 
> jboost/tokenizer
>
> Modified Files:
> 	LTStreamTokenizer.java
> Log Message:
> fixed legacy bugs that prevented comments from being parsed  
> reliably, or at all.
>
> Index: LTStreamTokenizer.java
> ===================================================================
> RCS file: /cvsroot/jboost/jboost/src/jboost/tokenizer/ 
> LTStreamTokenizer.java,v
> retrieving revision 1.1.1.1
> retrieving revision 1.2
> diff -C2 -d -r1.1.1.1 -r1.2
> *** LTStreamTokenizer.java	16 May 2007 04:06:02 -0000	1.1.1.1
> --- LTStreamTokenizer.java	16 May 2008 18:50:01 -0000	1.2
> ***************
> *** 71,75 ****
>          this.terminator= terminator;
>          terLen= terminator.length();
> !         minLen= terLen >= 2 ? 2 : terLen;
>          for (int i= 0, terLines= 0; i < terLen; i++)
>              if (terminator.charAt(i) == '\n')
> --- 71,75 ----
>          this.terminator= terminator;
>          terLen= terminator.length();
> !         minLen= terLen <= 2 ? 2 : terLen;
>          for (int i= 0, terLines= 0; i < terLen; i++)
>              if (terminator.charAt(i) == '\n')
> ***************
> *** 135,139 ****
>          while (true) {
>              //	    if(Monitor.logLevel>3) Monitor.log("strLen=" +  
> strLen + " toCopy=" + toCopy + " i=" + i);
> !             if (i > strLen - minLen) {
>                  try {
>                      numRead= br.read(cBuf, 0, bufLen);
> --- 135,139 ----
>          while (true) {
>              //	    if(Monitor.logLevel>3) Monitor.log("strLen=" +  
> strLen + " toCopy=" + toCopy + " i=" + i);
> !         	if (i > strLen - minLen) {
>                  try {
>                      numRead= br.read(cBuf, 0, bufLen);
> ***************
> *** 206,210 ****
>                  if (Util.even(numEscapes)) { // line comment starts!
>                      curTok += strBuf.substring(toCopy, i -  
> numEscapes / 2);
> !                     ongoingComment= true;
>                  } else // escpaed, part of token
>                      curTok
> --- 206,210 ----
>                  if (Util.even(numEscapes)) { // line comment starts!
>                      curTok += strBuf.substring(toCopy, i -  
> numEscapes / 2);
> !                     lineComment= true;
>                  } else // escpaed, part of token
>                      curTok
>
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> jboost-cvs mailing list
> jbo...@li...
> https://lists.sourceforge.net/lists/listinfo/jboost-cvs