UDT wrapping for python
Brought to you by:
lilyco
Menu
▾
▴
I had some free time to play around with UDT. I liked it, but I needed it for a project
in python so a SWIG wrapping emerged. Here it is: https://bitbucket.org/marto1/udtwrap
I made it with a schoolmate for few days so it's not tested and probably has memory leaks, but that's going to change of course.
Also, examples(located in src/) for client and server can be found there.
I've also been working on a Python C extension to UDT written in native C++ which allow for true PyTypes found https://github.com/cjhanks/PyUDT . It's still in development with the majority of features tested as working.
I would be a little surprised if the SWIG wrappers appropriately deal with the global interpreter lock for blocking UDT calls (eg: accept()), but I could be wrong.
I plan on finishing mine next week, as I need to use it as a dependency.
From what I can observe, you might have saved me a lot of time. The extension looks great, although I have trouble understanding the way that epoll works in this case. Can you elaborate on why it is in a separate thread (especially when GIL is still present and this can seriously hurt performance) and is it a necessity? Also, do you intend this to stay a raw/semi-raw wrapping around UDT, because my intention was to integrate udt in twisted as a transport protocol (so no work overlaps occur)?
Again, thanks for what you have done, I will probably use your solution.
P.S.: If i understood correctly "deal with the global interpreter lock" you mean this: http://stackoverflow.com/questions/1576737/releasing-python-gil-in-c-code .
The extension separates the code into two variations. Functions found in module udt4 directly wrap the C++-api except for noted in pydoc (ie: when C++ would return by reference or pointer). The pyudt implementation wrap the basic udt4 module to give a 'pythonic-style' implementation.
Example epoll in pyudt:
https://github.com/cjhanks/PyUDT/tree/master/example/pyudt/epoll
The previous example was launching threads as a testing convenience, however this is not a requirement.
This implementation will always be a (module udt4) raw extension around the UDT C++ interface and then a (module udt4.pyudt) python-wrap around the udt4 module.
As for the GIL, I'm not greatly familiar with SWIG. However, it appears they are wrapping functions with the Python.h C macros necessary to release the lock. In this implementation you would have to wrap every blocking UDT function with an inline function which releases and re-acquires the lock.
A stable version should trickle out over the next couple of weeks, if you have issues please let me know (https://github.com/cjhanks/PyUDT/issues). Currently all intended features have been added and tested at a minimal level. Memory tracing and edge cases will follow.
Also note that udt4 module function raise udt4.UDTException errors (base classing RuntimeError) and should be caught appropriately, exceptions map to a tuple of the UDT::ERROR rc and the UDT generated string error.
The latest stable version should be available in Pip via `pip install pyudt4`