#6 IndexError raised when tokenizing file with

closed
None
5
2007-08-30
2007-02-13
Anonymous
No

I'm running PyMetrics-0.7.6 using Python 2.4 on Windows XP HOME. I get the same error when running from Cygwin or DOS.

I get the following stacktrace from a file with trailing whitespaces at the end of the file:

############start trace ##################
Traceback (most recent call last):
File "../pymetrics-0.7.6/PyMetrics.py", line 261, in ?
main()
File "../pymetrics-0.7.6/PyMetrics.py", line 174, in main
lex.parse( inFileName ) # parse input file
File "C:\cygwin\home\humboldt\pymetrics-0.7.6\lexer.py", line 37, in parse
self.__parseSource()
File "C:\cygwin\home\humboldt\pymetrics-0.7.6\lexer.py", line 44, in __parseSource
tokenize.tokenize( text.readline, self )
File "C:\cygwin\home\humboldt\pymetrics-0.7.6\tokenize.py", line 156, in tokenize
tokenize_loop(readline, tokeneater)
File "C:\cygwin\home\humboldt\pymetrics-0.7.6\tokenize.py", line 162, in tokenize_loop
for token_info in generate_tokens(readline):
File "C:\cygwin\home\humboldt\pymetrics-0.7.6\tokenize.py", line 292, in generate_tokens
while line[pos] in ' \t':
IndexError: string index out of range

########################## end trace ################

I can definitely delete the trailing whitespace as the workaround or fix. But, I thought I might report this for the knowledge base.

Discussion

  • File that raises IndexError

     
    Attachments
    • assigned_to: nobody --> rcharney
    • status: open --> closed
     
  • Logged In: YES
    user_id=411372
    Originator: NO

    This is legitimate bug which was raised when an indented line was terminated with trailing blanks or tabs and no New Line (i.e. last line ended with EOF).