[JEDI.NET-commits] main/run Jedi.Text.Tokenization.Base.pas,NONE,1.1

[Marcel B. <jed...@us...>]

Update of /cvsroot/jedidotnet/main/run
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4679/main/run

Added Files:
	Jedi.Text.Tokenization.Base.pas 
Log Message:
Generic text tokenizer

--- NEW FILE: Jedi.Text.Tokenization.Base.pas ---
{---------------------------------------------------------------------------------------------------
The contents of this file are subject to the Mozilla Public License Version 1.1 (the "License"); you
may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.mozilla.org/MPL/MPL-1.1.html

Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
KIND, either expressed or implied. See the License for the specific language governing rights and
limitations under the License.

The Original Code is: Jedi.Text.Tokenization.Base.pas, released on --.

The Initial Developer of the Original Code is Marcel Bestebroer
Portions created by Marcel Bestebroer are Copyright (C) 2004 Marcel Bestebroer
All Rights Reserved.

Contributor(s):

You may retrieve the latest version of this file at the JEDI.NET home page, located at
http://sf.net/projects/jedidotnet
[...1091 lines suppressed...]

function WhiteSpaceToken.Literal: string;
begin
  if TypeOf(System.String).IsInstanceOfType(FData) then
    Result := System.String(FData)
  else
    Result := '';
end;
{$ENDREGION}

{$REGION 'WhiteSpaceToken.EndPoint'}
constructor WhiteSpaceToken.EndPoint.Create(column, line: Integer);
begin
  inherited Create;
  FColumn := column;
  FLine := line;
end;
{$ENDREGION}

end.