Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#503 frequent KeyDot failed, UnicodeError in interactive

open
nobody
pythonwin (176)
5
2010-09-24
2010-09-24
kxroberto
No

build214

Firing event 'KeyDot' failed.
Traceback (most recent call last):
File "C:\Python23\Lib\site-packages\pythonwin\pywin\scintilla\bindings.py", line 145, in fire
rc = binding.handler(*args)
File "C:\Python23\Lib\site-packages\pythonwin\pywin\scintilla\view.py", line 307, in KeyDotEvent
self._AutoComplete()
File "C:\Python23\Lib\site-packages\pythonwin\pywin\scintilla\view.py", line 467, in _AutoComplete
text = self.GetTextRange(self.LineIndex(minline),endpos)
File "C:\Python23\Lib\site-packages\pythonwin\pywin\scintilla\control.py", line 362, in GetTextRange
ret = ret.decode(default_scintilla_encoding)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 5946: unexpected code byte

Discussion

  • Roger Upole
    Roger Upole
    2010-09-28

    The underlying problem is how Scintilla handles (or fails to handle)
    multibyte characters when in utf-8 mode. None of the text handling
    messages (SCI_GETTEXTRANGE in this case) take continuation characters
    into account. If you ask for say 10 characters, what you get back is
    10 bytes, which may contain partial characters at the start or the end
    of the buffer.
    The only way I can see to deal with discrete characters are the
    SCI_POSITIONBEFORE and SCI_POSITIONAFTER messages, which means all operations would have to be done one character at a time.

     
  • kxroberto
    kxroberto
    2010-10-28

    the start/end for this GetTextRange come unchecked from LineIndex() 's.
    The lineno's unchecked from _GetClassInfoFromBrowser
    Maybe just a line is -1 or so.
    If always complete valid lines are decoded completely, may there should be no such problem?

     
  • Mark Hammond
    Mark Hammond
    2011-04-26

    Yeah - we are sending the GETTESTRANGE message, but the start and end pos have both come directly from LineIndex (EM_LINEINDEX). This is basically the same strategy taken by GetLine():

    start = self.LineIndex(line)
    end = self.LineIndex(line+1)
    return self.GetTextRange(start, end)

    the docs for EM_GETTEXTRANGE say end if "Character position immediately following the last character in the range.", so this looks correct.

    The above all assumes scintilla doesn't have a bug in some corner case:)

    It also seems strange the exception refers to 0xff - best I can tell, 0xff is not a leading utf8 byte.

    So I guess I need instructions for a repro so I can see what is going on via the debugger!

     
  • Michael Toews
    Michael Toews
    2011-05-30

    I frequently (daily) have a similar error, typically while editing code or comments in Python scripts where I happen to type letters followed by the "." character. This edited error was 2000 lines long:

    Firing event 'KeyDot' failed.
    Traceback (most recent call last):
    File "C:\Python25\Lib\site-packages\pythonwin\pywin\scintilla\bindings.py", line 142, in fire
    rc = binding.handler(*args)
    File "C:\Python25\Lib\site-packages\pythonwin\pywin\scintilla\view.py", line 307, in KeyDotEvent
    self._AutoComplete()
    File "C:\Python25\Lib\site-packages\pythonwin\pywin\scintilla\view.py", line 465, in _AutoComplete
    minline, maxline, curclass = self._GetClassInfoFromBrowser()
    File "C:\Python25\Lib\site-packages\pythonwin\pywin\scintilla\view.py", line 520, in _GetClassInfoFromBrowser
    if not browser.list: return (minline,maxline,None) # Not initialized
    File "C:\Python25\Lib\site-packages\pythonwin\pywin\tools\hierlist.py", line 62, in __getattr__
    return getattr(self, attr)
    File "C:\Python25\Lib\site-packages\pythonwin\pywin\tools\hierlist.py", line 62, in __getattr__
    return getattr(self, attr)
    80 lines deleted
    File "C:\Python25\Lib\site-packages\pythonwin\pywin\tools\hierlist.py", line 62, in __getattr__
    return getattr(self, attr)
    File "C:\Python25\Lib\site-packages\pythonwin\pywin\tools\hierlist.py", line 60, in __getattr__
    return getattr(self.listControl, attr)
    RuntimeError: maximum recursion depth exceeded

    I'm using pywin32 build 214, WinXP, Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)]

     
  • kxroberto
    kxroberto
    2011-11-22

    I have still this same "maximum recursion" kind of error in build 216.

    The solution probably:
    browser.list => browser.listControl
    And the recursive "return getattr(self, attr)" in HierList.__getattr__ probably makes no sense. (if attr in .__dict__ then .__getattr__ is not called. )

    ---

    --- _orig/view.py 2011-02-25 11:04:38 +0000
    +++ ./view.py 2011-11-22 14:46:18 +0000
    @@ -510,7 +510,7 @@
    pass
    if browser is None:
    return (minline,maxline,None) # Current window has no browser
    - if not browser.list: return (minline,maxline,None) # Not initialized
    + if not browser.listControl: return (minline,maxline,None) # Not initialized
    path = self.GetDocument().GetPathName()
    if not path: return (minline,maxline,None) # No current path

    --- _orig/hierlist.py 2009-02-16 17:43:26 +0000
    +++ ./hierlist.py 2011-11-22 14:43:35 +0000
    @@ -56,10 +56,10 @@
    self.filledItemHandlesMap = {}
    self.bitmapMask = bitmapMask
    def __getattr__(self, attr):
    - try:
    + ##try:
    return getattr(self.listControl, attr)
    - except AttributeError:
    - return getattr(self, attr)
    + ##except AttributeError:
    + ## return getattr(self, attr)

    def ItemFromHandle(self, handle):
    return self.itemHandleMap[handle]

     
  • Mika
    Mika
    2013-02-08

    I get always this error only when class browser in left panel is visible.

    My workaround is just to outcomment the line from where error comes e.g.
    #if not browser.list: return (minline,maxline,None)
    Seems to work without error after that. I'm just wondering what functionality I'm missing when out-commenting the line.

    AFAIK : The error comes because __getattr__ of browser is overloaded and when there is "browser." in code it starts infinite loop.