Unicode issue

Help
2012-01-09
2013-04-25
  • Eliott Minkovitch

    Hi Eric, I have a document which somehow stores unicode data instead of string data, for some text fields. The fields are editable Text type. These fields cannot be read by Plomino code getItem or doc.itemName, it says Attribute Error. This is quite an issue, since none of the text data is accessible. Is there any way to fix this, without upgrading to a new version?

    Thanks
    Eliott

     
  • Jean Jordaan

    Jean Jordaan - 2012-01-10

    Can you post a full traceback? Perhaps you have an underlying encoding error that gets turned into an AttributeError.

    It should be possible to write an agent to do `doc.setItem('itemid', asUnicode(doc.getItem('itemid')))` for all the relevant items.

    Note that if you have been mixing strings and unicode, you may be better off exporting your design and content, and recreating your Plomino instance from scratch.

    This is because once a mix of unicode and string values have made it into the same catalog index, you will get exceptions, e.g. when you try to sort that index. If a value is the same in unicode and as a string, the value in the catalog will remain a string, and it's tricky to ensure that all the values are unicode.

     
  • Eliott Minkovitch

    Sure, this is what I get if I use doc.fieldName notation:

      File "Script (Python)", line 46, in agent_ExportComments
    AttributeError: CommentsView
    Context is <PlominoAgent> /development-folder/unreleased-plomino-databases/qa/ReqForm/ExportComments

    if I use doc.getItem('fieldName') notation, I get:

      File "Script (Python)", line 46, in agent_ExportComments
    AttributeError: getItem
    Context is <PlominoAgent> /development-folder/unreleased-plomino-databases/qa/ReqForm/ExportComments

    I tried this on all kinds of fields, str and unicode, same result. Maybe it's nothing to do with unicode…

    and this is the source code:

    results=

    def exportFile(doc):
        #try:
           a0 = doc.getItem("CommentsView")
           a1 = str(a0)

           # remove all HTML from text
           a1 = a1.replace('<br>','')
           a1 = a1.replace('<br />','')
           a1 = a1.replace('<hr>','')
           a1 = a1.replace('<p>','')
           a1 = a1.replace('</p>','')
           a1 = a1.replace('&nbsp;','')
           a1 = a1.replace('&amp;','')
           a1 = a1.replace('<pre>','')
           a1 = a1.replace('</pre>','')

           # get three lines of comments, max 80 characters long each
           line1 = a1
           line2 = a1
           line3 = a1

           b1 = doc.getItem("requisitionNumber")

           if line1 != "": results.append()
           if line2 != "": results.append()
           if line3 != "": results.append()
     
           context.REQUEST.RESPONSE.setHeader('content-type', \
                   'text/csv; charset=utf-8')
           context.REQUEST.RESPONSE.setHeader("Content-Disposition", \
                   "attachment; filename=test_export_3.csv")
           context.REQUEST.RESPONSE.setBody(array_to_csv(results))
        #   return 'OK'
        #except:
        #   return 'Error'

    db=context.getParentDatabase()
    view=db.getView('ExportView')
    docs=view.getAllDocuments()
    for doc in docs:
           exportFile(doc)

    So, to summarise, I used version 1.13 (local installation), where it works perfectly and version 1.10 (server), where it does not work at all… any idea what may be causing it?

    Best regards,
    Eliott

     
  • Jean Jordaan

    Jean Jordaan - 2012-01-11

    File "Script (Python)", line 46, in agent_ExportComments
    > AttributeError: CommentsView
    > Context is <PlominoAgent> /development-folder/unreleased-plomino-databases/qa/ReqForm/ExportComments

    This still isn't the traceback, it's just the final error.
    To get the traceback, turn on Debug mode ("If enabled, script and formula
    errors are logged.") on your database edit tab and look in the Zope logfile.

    >   File "Script (Python)", line 46, in agent_ExportComments
    > AttributeError: getItem
    > Context is &lt;PlominoAgent&gt; /development-folder/unreleased-plomino-databases/qa/ReqForm/ExportComments
    

    This shows you that your 'doc' is not in fact a Plomino document. If it were
    a Plomino document, it would have a 'getItem' method.

    Check what 'doc' is.

    > I tried this on all kinds of fields, str and unicode, same result. Maybe
    > it's nothing to do with unicode…

    Probably not.

    docs=view.getAllDocuments()
    for doc in docs:
        exportFile(doc)
    

    The behaviour of 'getAllDocuments' changed a couple of versions ago.
    Now it returns documents.
    In the past it did not return documents, but catalog brains.

     
  • Eliott Minkovitch

    Indeed, you are correct - the view object does not return documents in release 1.10. getParentDatabase().getAllDocuments() will do the trick in this case.

    Thank you for pointing me in the right direction!
    Regards
    Eliott

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks