Hi Jim,

 

i just changed the mailing list becaue i want do dig a bit more into the jython ssl internals.

 

before i come to the concrete socket-reboot project you mentioned i first want to understant the basic issue (you mentioned blocking-non blocking api switches in python).

 

I had a look into the test code of cpython and jython ssl modules. (cpython: Lib/test/test_ssl.py and jython: Lib/test/test_socket_ssl.py)

There i come to the first (perhaps stupid) question: why the test code in jython does not contain any unittest.TestCase at all? Is there an obvious reason for it that i just oversee?

 

Next question that i was ask myself: if jython wants to guarantee api compatbility with cpython wouldn't it be just simply to take the ssl test code and implement something that passes this test? This would include then The BasicSocketTests as well.

 

Sorry for the stupid questions but i'm not in the matter how jython models all the low level system calls. But is seems that most of os package is not implemented like os.fork or os.pipe. But wouldn't it be possible to map these low level functions to java threads or even with a jni package?

 

--

 

Sven


On Sunday 09 February 2014 17:24:38 Jim Baker wrote:

Sven,


I updated http://bugs.jython.org/issue2066 to describe a workaround that can be used for easy_install, but not pip, namely to use this branch: https://bitbucket.org/jimbaker/jython-ssl


However this specific branch, useful as it is for right now for testing things out on Jython 2.7, is not how we will implement SSL support for Jython because its design does not support nonblocking SSL, as required by pip.


Instead, I'm actively working on a project that I call socket-reboot, https://github.com/jimbaker/socket-reboot, which is a brand new implementation of socket, select, and ssl support that uses Netty 4; quoting from the README for that repo:


The socket, select, and ssl modules provide the core, low-level networking semantics for Python. However, implementing these modules in their entirety is difficult for Jython given some differences between these APIs and the underlying Java platform. This is especially true with respect to supporting IO completions on nonblocking sockets (using either select or poll) and SSL handshaking.

 

This spike demonstrates with working code that for Jython 2.7 we can use Netty 4 to readily implement these core semantics, with minor exceptions. In particular, this spike mostly looks at the implications of implementing the select function, which is both minimally documented and tested in Python. The other major element addressed is the management of Netty's thread pool, especially with respect to cleaning up a PySystemState; such management is quite easy to implement in practice.


I encourage anyone who is interested to read this document, since it goes into extensive detail on how this support can be implemented. In addition, it has seen extensive review by experts in the Python networking community.


One thing I would add to that intro is that I have been recently functionally testing this spike with respect to asyncore/asynchat, in conjunction with ssl, and this has worked well for client usage. Since the select loop used by asyncore is reasonably close to how pip (through requests) works, I think this bodes well for my approach.


The other thing to know is that I haven't yet looked at server sockets or UDP, however, I cannot see why the socket-reboot approach would not apply to either situation given how well Netty 4 supports both cases, so it's just a simple matter of programming ;).


Progress on this has been slower than I would like, especially since supporting pip/virtualenv is a blocker for Jython 2.7 beta 2. Having said this, I fully expect we will have this support implemented very soon now that this approach has started to really solidify.


- Jim




On Sun, Feb 9, 2014 at 10:17 AM, Sven Aßmann <sven.assmann@lubico.biz> wrote:

I found the bug already existing:

http://bugs.jython.org/issue2066

Is someone already taking care of that?



On Sunday 09 February 2014 18:01:08 Sven Aßmann wrote:
> Hi Folks,
>
> on twitter i was kindly inveted to this list. Thanks for that.
>
> I had some issues to with getting setuptools and pip working to be finally
> able to prepare a virtualenv, because setuptools and pip are prerequisits
> for a virtualenv.
>
> However I tried to use setuptools as described in the appendixA
> http://www.jython.org/jythonbook/en/1.0/appendixA.html#setuptools
>
> But it turned out to be not working (at least on my system with ArchLinux)
> with jyton27.
>
> So I did the stuff manually (https://gist.github.com/sassman/8901056) so far
> so good. Now i only had to install pip.
> But as you can see here (http://goo.gl/8RRjtA) this failed with a strange
> issue form ssl support of setuptools module:
>
> AttributeError: 'module' object has no attribute 'CERT_REQUIRED'
>
> if found that:
> https://mail.python.org/pipermail/distutils-sig/2013-July/021449.html
>
> So, i would really appreciate some input, any one an idea? Does this stuff
> work for you guys?
> Should i file a bug for jython 2.7?
>
>
> Cheers,
> Sven Aßmann
>
>
>
>
> ----------------------------------------------------------------------------

> -- Managing the Performance of Cloud-Based Applications

> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
> _______________________________________________
> Jython-users mailing list
> Jython-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/jython-users

------------------------------------------------------------------------------
Managing the Performance of Cloud-Based Applications
Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
Read the Whitepaper.
http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
_______________________________________________
Jython-users mailing list
Jython-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jython-users