- If that's the case, any idea on why Jython don't see the plugin
classes by default?

In my case it's not enough for Jython to see the same classes as 'ourCrazyClassloader', they really do have to be loaded by 'ourCrazyClassloader'.  

 - Anyway, why is it needed to change the class loader "on the fly",
instead of setting it once when calling initialize()?

I don't need to change it on the fly. Some more code:

PySystemState state = new PySystemState();
interpreter = new PythonInterpreter(ourCrazyDictionary, state);

After then we never touch it and just keep using interpreter.