Re: [ jEdit-devel ] TextUtilities.getTokenAtOffset()

[Rudi W. <Rud...@we...>]

Hello Robert,
please keep in mind that if your offset points to a whitespace TextUtilities.getTokenAtOffset doesn't return a token.
For XSearch-plugin I have this workaround:

boolean blankChar = true;
for (int i=currPos; blankChar && i<buffer.getLength();i++) {
	if (!Character.isWhitespace(buffer.getText(i,1).charAt(0))) {
		blankChar = false;
		currPos = i;
	}
}
int line = textArea.getLineOfOffset(currPos);
int position = currPos - textArea.getLineStartOffset(line);

DefaultTokenHandler tokens = new DefaultTokenHandler();
buffer.markTokens(line, tokens);
org.gjt.sp.jedit.syntax.Token token =
TextUtilities.getTokenAtOffset(tokens.getTokens(), position);
if(token.id == Token.COMMENT1 || token.id == Token.COMMENT2)
	outsideCmt = false;
else
	outsideCmt = true;


Message: 10
From: Slava Pestov <sl...@je...>
To: de...@je...
Subject: Re: [ jEdit-devel ] TextUtilities.getTokenAtOffset()
Date: Wed, 19 Feb 2003 13:38:55 -0500

Hi,

A TokenHandler is sort of like an event listener, it takes action upon 
receiving a certain syntax token. The DefaultTokenHandler builds a linked 
list of Token instances. However, it needs to actually receive the tokens -- 
and this is done with a Buffer.markTokens() call:

TokenHandler tokenHandler = new DefaultTokenHandler();
buffer.markTokens(lineNumber,tokenHandler);
Token tokens = tokenHandler.getTokens();
Token token = TextUtilities.getTokenAtOffset(tokens,columnNumber);

If 'pos' is an offset from the start of the buffer, turn it into a line/column 
pair like this:

int lineNumber = buffer.getLineOfOffset(pos);
int columnNumber = pos - buffer.getLineStartOffset(lineNumber);
On February 19, 2003 01:18 pm, Robert Fletcher wrote:

>> Hi,
>>
>> I'm trying to identify if a particular offset is
>> inside a comment or literal, so I figured the best way
>> to do it would be something like:
>>
>> TokenHandler tokenhandler = new DefaultTokenHandler();
>> Token tokens = tokenhandler.getTokens();
>> Token token = TextUtilities.getTokenAtOffset(tokens,
>> pos);
>> return token.id == Token.COMMENT1 ||
>>        token.id == Token.COMMENT2 ||
>>        token.id == Token.LITERAL1 ||
>>        token.id == Token.LITERAL2;
>>
>> But the tokens.getTokens() call is returning null.
>> Anyone know if I'm missing something obvious or if I'm
>> totally barking up the wrong tree?
>>
>> 
>