Donate Share

Frhed

Tracker: Patches

5 Python Support for Encoders - ID: 2834337
Last Update: Comment added ( nobody )

Add Python Support for encoders - patch attached.

This also fixes a bug in the existing dynamic coder code which previously
could only load 1 plugin.

Place python encoders in the directory <Frhed Installation>\PyEncoders

Python encoders can do everything c encoders can do with these current
bugs.

* For large selections 100 MB+ the encoder doesn't work because a python
return string
can't be generated without an out of memory error in the python
interpreter - a custom c++
class is needed which wraps the selection buffer, so the data doesn't
need replicated as a
python string.
* If pythonXX.dll can't be found the encoder doesn't load - it is dependent
on a very specific
python version - It is recommend that the python26.dll be included in the
distro in the
<Frhed Installation> directory.
* Built with VC 2008 Express, so the project version for pyscripts isn't
for 2005 like the rest
of the projects
* Currently this plugin uses no unicode aware strings for paths - research
is needed to determine
if python supports unicode in module names or paths and how to use them
through the C-interfaces.
* If a script fails it shows no error output to the user and the data
remains unchanged (currently new pyscripts.err and pyscripts.out files are
created under <frhed install> to help in debugging new scripts).

See the readme.txt for the full details.

There are a few example sample encoders under
EncoderPlugins\pyscripts\pyEncoders\sample.py.

Before before requesting checkin to the trunk, I will see about creating a
C++ wrapper for the buffer sent to python encoder. I'm thinking I may
give it a similar interface in python as a file handle. Python scripts can
read from it and write to it as needed but don't need necessarily need to
create insanely long strings like they do now.


Chris Nebergall ( cneb96 ) - 2009-08-09 00:42

5

Open

None

Nobody/Anonymous

None

trunk

Public


Comments ( 8 )




Date: 2009-10-27 04:04
Sender: nobody

Will do. I've been busy as well. I'll get to this this weekend.


Date: 2009-10-21 19:04
Sender: kimmovProject AdminAccepting Donations

Can you add the updated patch addressing my comments? Like moving document
to correct folder.

I'm again sorry it has taken so much time to respond. I simply haven't had
any time for Frhed development in past weeks and also had problems with
e-mails coming through so I may have missed some notifications from Sf.net
trackers...


Date: 2009-08-13 16:40
Sender: kimmovProject AdminAccepting Donations

Committed the bug fix to load more than one encoder dll to SVN trunk:
Completed: At revision: 820



Date: 2009-08-10 05:42
Sender: kimmovProject AdminAccepting Donations

> The pyscripts project has all the python dependencies on
> it not the top level solution.

Missed that, thanks for clarifying.

> pyscripts is just based on the addone project file.

Ok. I made that with VS2005. Don't worry about the projec file format we
can always convert it later.

> ok. I'll add that to the todo in the readme. Is it ok
> if I add that later after we get the initial checkin?

Definitely. Lets get this in first and further modifications and
improvements are then a lot easier. I personally prefer smaller commits
which makes it easier to track things later.


Date: 2009-08-09 16:53
Sender: nobody

>>But can you move the doc to /Docs/Developers/ -folder.

ok

>>One thing I'm not very thrilled about is this makes compiling Frhed to
require Python.

The pyscripts project has all the python dependencies on it not the top
level solution. Currently the top level solution doesn't even know about
the pyscripts project, you have to import it before build-all will even
know about it. So frhed is no more dependent on pyscripts than addone (I
based pyscripts on addone).

>> I still prefer project files to be in VS2003 format.

pyscripts is just based on the addone project file. It contains 1 source
file with a change to add one new header directory and one new lib file
(but VC auto updated it for me to the 2008 format).

So I can see about just hand editing the project file from addone to make
it into the new pyscripts project, or using the tool you describe to submit
it next time. (The details on how to add the header file, and lib
dependencies are also in the docs, I did that in case a user had a
different version of the compiler and wanted to know how to recreate the
pyscripts project file).

>> The bugfix for loading more than one plugins is this? :

yes. Otherwise the second plugin string looks like this ;plugintwo.dll;
and that didn't load.

>>One thing to consider later is to load (Python) encoders from user's
folder.

ok. I'll add that to the todo in the readme. Is it ok if I add that
later after we get the initial checkin?


Date: 2009-08-09 10:13
Sender: kimmovProject AdminAccepting Donations

Thanks a lot for great documentation you have written. Helps a lot. I'll
borrow some of that for the user manual too. But can you move the doc to
/Docs/Developers/ -folder. IMO its better to have one folder for docs than
spread them around the source tree (even though it means you need to cd to
docs folder).

One thing I'm not very thrilled about is this makes compiling Frhed to
require Python. It would be great it people still can compile Frhed without
Python (won't be able to use Python encoders but not everybody need to use
them). Not everybody wants to install Python to compile Frhed.

I still prefer project files to be in VS2003 format. Of course this is not
possible to achieve for everybody (or even most developers?). I have VS2003
in one machine but have rarely access to it. But I can access VS2005 so I
can convert project to it later.

I also just found this conversion tool:
https://sourceforge.net/projects/vspc/
but haven't yet tried it...

I think the interface is OK as first version. Later on we need to add more
info about the plugin for user. Like author, version etc.

The bugfix for loading more than one plugins is this? :
}
- prev_ind = ind;
- ind = buffer.find_first_of(';', prev_ind + 1);
+ prev_ind = ind + 1;
+ ind = buffer.find_first_of(';', prev_ind);
}

I'll commit it as separate commit, Thanks for fixing it.

One thing to consider later is to load (Python) encoders from user's
folder. If user is normal user without admin privileges one cannot install
new plugins to program files folder.


Date: 2009-08-09 06:44
Sender: kimmovProject AdminAccepting Donations

I took a short look at the code and this looks good!


Date: 2009-08-09 06:38
Sender: kimmovProject AdminAccepting Donations

Moving to the patch tracker.


Log in to comment.




Attached File ( 1 )

Filename Description Download
pyencoder_2009_08_08.patch working version Download

Changes ( 3 )

Field Old Value Date By
artifact_group_id None 2009-08-13 16:40 kimmov
data_type 1100165 2009-08-09 06:38 kimmov
File Added 338471: pyencoder_2009_08_08.patch 2009-08-09 00:42 cneb96