#1280 PySTAF.unmarshall fails on multi-byte characters

Unix::Linux
open
nobody
5
5 days ago
2009-07-23
Nathan Parrish
No

from https://sourceforge.net/forum/forum.php?thread_id=3340759&forum_id=104046

I thought I had broken my receive code, but in fact the problem is that PySTAF.unmarshall() fails when there is non-ascii data (this despite having the setdefaultencoding set in sitecustomize.py):

>>> result.result
'@SDT/{:297::6:handle@SDT/$S:2:81:10:handleName@SDT/$S:8:stafmsgr:7:machine@SDT/$S:13:local://local:7:message@SDT/$S:14:test \xc5\x93\xc5\x93\xc5\x93\xc5\x93 test:8:priority@SDT/$S:1:5:19:staf-map-class-name@SDT/$S:24:STAF/Service/Queue/Entry:9:timestamp@SDT/$S:17:20090721-19:19:24:4:type@SDT/$S:4:test:4:user@SDT/$S:16:none://anonymous'
>>> PySTAF.unmarshall(result.result)
@SDT/{:297::6:handle@SDT/$S:2:81:10:handleName@SDT/$S:8:stafmsgr:7:machine@SDT/$S:13:local://local:7:message@SDT/$S:14:test œœœœ test:8:priority@SDT/$S:1:5:19:staf-map-class-name@SDT/$S:24:STAF/Service/Queue/Entry:9:timestamp@SDT/$S:17:20090721-19:19:24:4:type@SDT/$S:4:test:4:user@SDT/$S:16:none://anonymous

and where we do not have the non-ascii:
>>> result.result
'@SDT/{:291::6:handle@SDT/$S:2:81:10:handleName@SDT/$S:8:stafmsgr:7:machine@SDT/$S:13:local://local:7:message@SDT/$S:9:test test:8:priority@SDT/$S:1:5:19:staf-map-class-name@SDT/$S:24:STAF/Service/Queue/Entry:9:timestamp@SDT/$S:17:20090721-19:21:34:4:type@SDT/$S:4:test:4:user@SDT/$S:16:none://anonymous'
>>> PySTAF.unmarshall(result.result)
{
priority : 5
handle : 81
staf-map-class-name: STAF/Service/Queue/Entry
timestamp : 20090721-19:21:34
machine : local://local
user : none://anonymous
message : test test
type : test
handleName : stafmsgr
}

Discussion

  • Nathan Parrish
    Nathan Parrish
    2009-08-05

    I actually worked around this by explicitly decoding the message as UTF-8
    before unmarshalling it:
    if result.rc == 0:
    unimsg = result.result.decode("utf-8")
    mesgdict = PySTAF.unmarshall(unimsg).getRootObject()