Python Support for Encoders
Brought to you by:
kimmov
Menu
▾
▴
#24 Python Support for Encoders
Add Python Support for encoders - patch attached.
This also fixes a bug in the existing dynamic coder code which previously could only load 1 plugin.
Place python encoders in the directory <Frhed Installation>\PyEncoders
Python encoders can do everything c encoders can do with these current bugs.
* For large selections 100 MB+ the encoder doesn't work because a python return string
can't be generated without an out of memory error in the python interpreter - a custom c++
class is needed which wraps the selection buffer, so the data doesn't need replicated as a
python string.
* If pythonXX.dll can't be found the encoder doesn't load - it is dependent on a very specific
python version - It is recommend that the python26.dll be included in the distro in the
<Frhed Installation> directory.
* Built with VC 2008 Express, so the project version for pyscripts isn't for 2005 like the rest
of the projects
* Currently this plugin uses no unicode aware strings for paths - research is needed to determine
if python supports unicode in module names or paths and how to use them through the C-interfaces.
* If a script fails it shows no error output to the user and the data remains unchanged (currently new pyscripts.err and pyscripts.out files are created under <frhed install> to help in debugging new scripts).
See the readme.txt for the full details.
There are a few example sample encoders under EncoderPlugins\pyscripts\pyEncoders\sample.py.
Before before requesting checkin to the trunk, I will see about creating a C++ wrapper for the buffer sent to python encoder. I'm thinking I may give it a similar interface in python as a file handle. Python scripts can read from it and write to it as needed but don't need necessarily need to create insanely long strings like they do now.
Discussion
working version
Moving to the patch tracker.
I took a short look at the code and this looks good!
Thanks a lot for great documentation you have written. Helps a lot. I'll borrow some of that for the user manual too. But can you move the doc to /Docs/Developers/ -folder. IMO its better to have one folder for docs than spread them around the source tree (even though it means you need to cd to docs folder).
One thing I'm not very thrilled about is this makes compiling Frhed to require Python. It would be great it people still can compile Frhed without Python (won't be able to use Python encoders but not everybody need to use them). Not everybody wants to install Python to compile Frhed.
I still prefer project files to be in VS2003 format. Of course this is not possible to achieve for everybody (or even most developers?). I have VS2003 in one machine but have rarely access to it. But I can access VS2005 so I can convert project to it later.
I also just found this conversion tool:
https://sourceforge.net/projects/vspc/
but haven't yet tried it...
I think the interface is OK as first version. Later on we need to add more info about the plugin for user. Like author, version etc.
The bugfix for loading more than one plugins is this? :
}
- prev_ind = ind;
- ind = buffer.find_first_of(';', prev_ind + 1);
+ prev_ind = ind + 1;
+ ind = buffer.find_first_of(';', prev_ind);
}
I'll commit it as separate commit, Thanks for fixing it.
One thing to consider later is to load (Python) encoders from user's folder. If user is normal user without admin privileges one cannot install new plugins to program files folder.
>>But can you move the doc to /Docs/Developers/ -folder.
ok
>>One thing I'm not very thrilled about is this makes compiling Frhed to require Python.
The pyscripts project has all the python dependencies on it not the top level solution. Currently the top level solution doesn't even know about the pyscripts project, you have to import it before build-all will even know about it. So frhed is no more dependent on pyscripts than addone (I based pyscripts on addone).
>> I still prefer project files to be in VS2003 format.
pyscripts is just based on the addone project file. It contains 1 source file with a change to add one new header directory and one new lib file (but VC auto updated it for me to the 2008 format).
So I can see about just hand editing the project file from addone to make it into the new pyscripts project, or using the tool you describe to submit it next time. (The details on how to add the header file, and lib dependencies are also in the docs, I did that in case a user had a different version of the compiler and wanted to know how to recreate the pyscripts project file).
>> The bugfix for loading more than one plugins is this? :
yes. Otherwise the second plugin string looks like this ;plugintwo.dll; and that didn't load.
>>One thing to consider later is to load (Python) encoders from user's folder.
ok. I'll add that to the todo in the readme. Is it ok if I add that later after we get the initial checkin?
> The pyscripts project has all the python dependencies on
> it not the top level solution.
Missed that, thanks for clarifying.
> pyscripts is just based on the addone project file.
Ok. I made that with VS2005. Don't worry about the projec file format we can always convert it later.
> ok. I'll add that to the todo in the readme. Is it ok
> if I add that later after we get the initial checkin?
Definitely. Lets get this in first and further modifications and improvements are then a lot easier. I personally prefer smaller commits which makes it easier to track things later.
Committed the bug fix to load more than one encoder dll to SVN trunk:
Completed: At revision: 820
Can you add the updated patch addressing my comments? Like moving document to correct folder.
I'm again sorry it has taken so much time to respond. I simply haven't had any time for Frhed development in past weeks and also had problems with e-mails coming through so I may have missed some notifications from Sf.net trackers...
Will do. I've been busy as well. I'll get to this this weekend.