#324 Dependency on __file__

[Anthon van der Neut]

When testing with PyOxidizer, which doesnt set __file__, as it is not sensible in that context and also it is not a mandatory module property, ruamel.yaml fails many of its tests (104) because main.py attempts to load official modules using __file__.

The easy fix is to make it optional, and dont allow plugins (i.e. the jinja2 plugin), in that context.

The better fix is to find the list of ruamel.yaml.* packages using other mechanisms that are now part of Python, especially importlib.metadata and importlib.resources, both of which have backports.

In order to avoid backports, perhaps mix strategies by using __file__ if present, or fallback to using importlib.

(originally posted on 2019-11-04 at 04:40:03 by John Vandenberg <John Vandenberg@bitbucket>)

[Anthon van der Neut]

None
(originally posted on 2019-11-04 at 04:43:26 by John Vandenberg <John Vandenberg@bitbucket>)

[Kelly Brazil]

Hello, I am seeing this same issue with PyOxidizer with my command-line application jc: https://github.com/kellyjonbrazil/jc

A user reported that the YAML parser (using ruamel.yaml) is not working with the PyOxidizer binary:

For this test

printf 'blah:\n  thing: stuff' | ./jc -d --yaml

I get this error

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "jc.cli", line 456, in main
  File "jc.parsers.yaml", line 137, in parse
  File "ruamel.yaml.main", line 89, in __init__
  File "ruamel.yaml.main", line 633, in official_plug_ins
NameError: name '__file__' is not defined

The pypi install is fine which is why I think it's a packaging bug - not a bug in jc as such.
But I'm happy to open an issue on the jc project if you prefer.

I'm not sure how you would fix it since ruamel.yaml does this

    # helpers
    def official_plug_ins(self):
        # type: () -> Any
        bd = os.path.dirname(__file__)
        gpbd = os.path.dirname(os.path.dirname(bd))
        res = [x.replace(gpbd, "")[1:-3] for x in glob.glob(bd + '/*/__plug_in__.py')]
        return res

[Kelly Brazil]

Hi there,

I just wanted to bump this issue since it is user impacting and if it happens to be an easy fix I'd like to re-package my application so it will have full functionality.

Thanks in advance,
Kelly

[Anthon van der Neut]

It is an easy fix if the packaging doesn't need the plug-ins. In that case it is a question of catching the NameError and returning the empty list. It is scheduled for 0.17.0.
Until then you can of course do something like:

ruamel.yaml.YAML.official_plug_ins = lambda a: []

before instantiating YAML()

[Kelly Brazil]

Hi Anthon,

Thanks for the workaround - I'll try that out!

[Kelly Brazil]

Hi Anthon - looks like that worked. Thanks for your help!

[Anthon van der Neut]

If you ever need to include one of the plug-ins, e.g. jinja2 then you can do:

   ruamel.yaml.YAML(plug_ins=['ruamel/yaml/jinja2/__plug_in__'])

assuming that you package the plug-in as well.

[Anthon van der Neut]

This is solved in 0.17 by catching the NameError that accessing __file__ throws.

This works out of the box if no plug-ins are necessary. If plugins are bundled in the oxidized executable, than YAML.official_plug_ins must be monkey patched with a lambda that gives the correct list for the bundled plugins sub-paths.