#71 Spurious "invalid syntax" errors for modules

v3.0
closed-fixed
5
2006-03-16
2006-03-16
No

This was recreated in Windows, after I made a change to
the source code so we wouldn't get an ImportError if
the curses module couldn't be imported.

I get strange errors in 3.0 alpha when trying to
process some modules. I've picked a simple example out
for you.

Module:
http://cvs.sourceforge.net/viewcvs.py/\*checkout*/dopal/dopal/src/dopal/__init__.py?content-type=text%2Fplain

Renamed it to dopal_init.py and tried to run epydoc on it.

Output:

------
[
Error: Error during parsing: invalid syntax
(C:\Temp\dopal_init.py,
line 21)
[................................
Warning: Empty group u'Front-end modules' in dopal_init
Warning: Empty group u'Core-level modules' in dopal_init
Warning: Empty group u'Object-level modules' in dopal_init

Traceback (most recent call last):
File "c:\python24\lib\site-packages\epydoc\cli.py",
line 636, in ?
cli()
File "c:\python24\lib\site-packages\epydoc\cli.py",
line 315, in cli
main(options, names)
File "c:\python24\lib\site-packages\epydoc\cli.py",
line 271, in main
add_submodules=(options.action!='text'))
File
"C:\Python24\Lib\site-packages\epydoc\docbuilder.py",
line 203, in build_doc_index
val_doc.init_groups()
File
"C:\Python24\Lib\site-packages\epydoc\apidoc.py", line
689, in init_groups
del self.group_names[i], self.groups[group]
IndexError: list assignment index out of range

Discussion

  • Edward Loper

    Edward Loper - 2006-03-16

    Logged In: YES
    user_id=195958

    It looks like this actually shows two bugs: one, a problem
    in deleting empty groups, which I just fixed in
    subversion; and another, which causes the parse error,
    which I was unable to replicate.

    I don't see anything very unusual on line 21 of the file
    that you sent me, so I'm not sure why it would be
    generating a parse error. When I run it, I don't get
    one. Try running epydoc with "-vvv", and let me know what
    the 3-4 lines of output before the parse error are?

     
  • Allan Crooks

    Allan Crooks - 2006-03-16

    Logged In: YES
    user_id=39733

    First of all, I forgot to mention version of Python I'm
    using - it's 2.4.1.

    Secondly, I forgot to thank you for writing the software -
    it's really great stuff. :)

    The module itself imports fine, so I don't really know why
    it's failing either. Anyway I can get whatever error is
    causing the problem escape to the console? I can modify the
    source code locally to figure it out.

    Output with -vvv switch:

    Anyway, with -vvv flag:

    -----

    Building documentation
    [ 0%] dopal_init (dopal_init.py)
    Info: Parsing C:\Temp\dopal_init.py
    Error: Error during parsing: invalid syntax
    (C:\Temp\dopal_init.py,
    line 21)
    Merging parsed & introspected information
    Linking imported variables
    [ 0%] dopal_init
    Indexing documentation
    [ 0%] dopal_init
    Parsing docstrings
    [ 0%] dopal_init
    Inheriting documentation
    Sorting & Grouping
    [ 0%] dopal_init
    Warning: Empty group u'Front-end modules' in dopal_init
    Warning: Empty group u'Core-level modules' in dopal_init
    Warning: Empty group u'Object-level modules' in dopal_init

    Traceback (most recent call last):
    File "c:\python24\lib\site-packages\epydoc\cli.py", line
    636, in ?
    cli()
    File "c:\python24\lib\site-packages\epydoc\cli.py", line
    315, in cli
    main(options, names)
    File "c:\python24\lib\site-packages\epydoc\cli.py", line
    271, in main
    add_submodules=(options.action!='text'))
    File "C:\Python24\Lib\site-packages\epydoc\docbuilder.py",
    line 203, in build_doc_index
    val_doc.init_groups()
    File "C:\Python24\Lib\site-packages\epydoc\apidoc.py",
    line 689, in init_groups
    del self.group_names[i], self.groups[group]
    IndexError: list assignment index out of range

    ----

     
  • Edward Loper

    Edward Loper - 2006-03-16

    Logged In: YES
    user_id=195958

    Could you try making the following change to
    epydoc/docbuilder.py, on around line 373:

    try:
    parse_doc = parse_docs(
    filename=filename, context=parent_docs[1])
    except ParseError, e:
    - log.error(e)
    + raise

    (i.e., replace "log.error(e)" with "raise), and try
    running it again? That should at least let me know what
    part of the epydoc parsing code is having trouble. (I'd
    do this myself, but as I said earlier, I can't replicate
    the problem when I run epydoc on that file.)

     
  • Allan Crooks

    Allan Crooks - 2006-03-16
     
  • Allan Crooks

    Allan Crooks - 2006-03-16

    Logged In: YES
    user_id=39733

    Figured out what it was.

    First of all, my local copy of the file is different to what
    you've been using - mine seems to have a combination of
    different file endings for some reason. It seems the
    tokenize module doesn't like this, and that's what the
    syntax error is related to.

    After hacking around, I figured out how you can modify the
    code to fix it.

    docparser.py, line 517
    - module_file = codecs.open(module_doc.filename, 'r',
    encoding)
    + module_file = codecs.open(module_doc.filename, 'rU',
    encoding)

    You'll need to check whether that version of Python supports
    universal newlines or not though (it was implemented in 2.3
    - I'm not sure what the minimum version of Python is going
    to be for Epydoc 3).

     
  • Edward Loper

    Edward Loper - 2006-03-16

    Logged In: YES
    user_id=195958

    Would you mind attaching a copy of the file with the mixed
    line endings, so I can play with it?

     
  • Allan Crooks

    Allan Crooks - 2006-03-16

    Logged In: YES
    user_id=39733

    I had already attached the file with the last post - guess I
    should have mentioned it. :)

     
  • Edward Loper

    Edward Loper - 2006-03-16
    • status: open --> closed-fixed
     
  • Edward Loper

    Edward Loper - 2006-03-16

    Logged In: YES
    user_id=195958

    Fixed in subversion (rev. 987)

    I fixed it by adding "U" to the mode, as you suggested. I
    also made the parse error messages a little more detailed,
    which will hopefully make it easier to tell where the
    problem is.

    "U" does require python 2.3, although it appears to be
    silently ignored before python 2.3. Presumably having
    mixed line-termination characters should be fairly
    uncommon, so I'm not too worried about it.

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks