#41 Yapsy-loaded Python modules have incorrect encoding

[Tiago Nobrega]

Hi, I believe Yapsy's default strategy of open()ing Python modules is not respecting the module-encoding rules defined by the language. I've tested this on Python 3.6 with Yapsy 1.12.2. The attached files define this plugin:

# coding: utf-8

from yapsy.IPlugin import IPlugin


class PluginWithList(IPlugin):

    def activate(self):
        assert b'a\xc3\xa7\xc3\xa3o'.decode('utf-8') == "ação"

Then when I activate the plugin with:

from yapsy.PluginManager import PluginManager

simplePluginManager = PluginManager()
simplePluginManager.setPluginPlaces([r'X:/yapsy-bug'])
simplePluginManager.collectPlugins()

for pluginInfo in simplePluginManager.getAllPlugins():
    simplePluginManager.activatePluginByName(pluginInfo.name)

I get the following error:

  File "X:\yapsy-bug\my_plugin.py", line 9, in activate
    assert b'a\xc3\xa7\xc3\xa3o'.decode('utf-8') == "ação"
AssertionError

I believe this happens because I'm on a Windows machine which has a default encoding of cp-1252(extended ascii with latin characters). The file is written in UTF-8, but a raw open() call without specifying the encoding will use the system's default, which in this case is incorrect.

For Python 3, I believe an immediate improvement would be to default to utf-8 in the open() call, as that is the language's default. A better fix would probably involve sniffing the file's encoding directive, like the interpreter itself does. Specifically, I mean the code in PluginManager._importModule():

    @staticmethod
    def _importModule(plugin_module_name, candidate_filepath):
        """
        Import a module, trying either to find it as a single file or as a directory.

        .. note:: Isolated and provided to be reused, but not to be reimplemented !
        """
        # use imp to correctly load the plugin as a module
        if os.path.isdir(candidate_filepath):
            candidate_module = imp.load_module(plugin_module_name,None,candidate_filepath,("py","r",imp.PKG_DIRECTORY))
        else:
            with open(candidate_filepath+".py","r") as plugin_file:
                candidate_module = imp.load_module(plugin_module_name,plugin_file,candidate_filepath+".py",("py","r",imp.PY_SOURCE))
        return candidate_module

Thank you for your work and please let me know if you have questions/are interested in a patch.

Tiago