Jython

On Fri, Sep 21, 2012 at 1:32 AM, Julian Kennedy <juliankenn@...:

> Hi Guys
>
> Ive decided to start work on implementing the bz2 module, unless
> anyone else is working on it already. My plan is to use apache commons
> compress. I will update the roadmap wiki when I get a chance.
>
Hi Julian,

I think someone has tried using Apache's compression code before and found
that some parts of CPython's bz2 module are not implementable that way - on
the other hand this was many years ago so maybe things are different now.
It sounds like a good starting place in any case - let us know how it goes!

BTW apologies that I haven't made it back to test your test_csv patch yet,
I think I should have a chance today.

-Frank

BTW there is a small pointer on porting modules here:
http://wiki.python.org/jython/JythonDeveloperGuide/PortingPythonModulesToJython

-Frank

On Sep 21, 2012, at 1:32 AM, Julian Kennedy wrote:

> Hi Guys
> 
> Ive decided to start work on implementing the bz2 module, unless
> anyone else is working on it already. My plan is to use apache commons
> compress. I will update the roadmap wiki when I get a chance.

Take a look at:

http://bugs.jython.org/issue1445

So the apache commons version does not support streaming of decompression. However the rather new jbzip2 lib might support it:

http://code.google.com/p/jbzip2/

I'd take a look at the code on issue1445 as a starting point and if jbzip2 supports the decompression streaming, move it over to support that.

--
Philip Jenvey

Thx for the pointers. I will have a look at those.

On 21 September 2012 18:41, Philip Jenvey <pjenvey@...> wrote:
>
> On Sep 21, 2012, at 1:32 AM, Julian Kennedy wrote:
>
>> Hi Guys
>>
>> Ive decided to start work on implementing the bz2 module, unless
>> anyone else is working on it already. My plan is to use apache commons
>> compress. I will update the roadmap wiki when I get a chance.
>
> Take a look at:
>
> http://bugs.jython.org/issue1445
>
> So the apache commons version does not support streaming of decompression. However the rather new jbzip2 lib might support it:
>
> http://code.google.com/p/jbzip2/
>
> I'd take a look at the code on issue1445 as a starting point and if jbzip2 supports the decompression streaming, move it over to support that.
>
> --
> Philip Jenvey
>

Just an update. I managed to get all the tests passing except the
BZ2Decompressor tests. Havent gotten to it yet. I got the streaming
compressor working by using a custom OutputStream class that captures
the compressed output in chunks.

Im using Apache commons compress. It seems the projects you guys
mentioned was somehow integrated into the apache commons project. At
least, thats what I conclude based on the API's.

I'll complete the BZ2Decompressor sometime this week.

Regards

On 23 September 2012 15:45, Julian Kennedy <juliankenn@...> wrote:
> Thx for the pointers. I will have a look at those.
>
> On 21 September 2012 18:41, Philip Jenvey <pjenvey@...> wrote:
>>
>> On Sep 21, 2012, at 1:32 AM, Julian Kennedy wrote:
>>
>>> Hi Guys
>>>
>>> Ive decided to start work on implementing the bz2 module, unless
>>> anyone else is working on it already. My plan is to use apache commons
>>> compress. I will update the roadmap wiki when I get a chance.
>>
>> Take a look at:
>>
>> http://bugs.jython.org/issue1445
>>
>> So the apache commons version does not support streaming of decompression. However the rather new jbzip2 lib might support it:
>>
>> http://code.google.com/p/jbzip2/
>>
>> I'd take a look at the code on issue1445 as a starting point and if jbzip2 supports the decompression streaming, move it over to support that.
>>
>> --
>> Philip Jenvey
>>

On Sun, Oct 21, 2012 at 1:08 PM, Julian Kennedy <juliankenn@...:

> Just an update. I managed to get all the tests passing except the
> BZ2Decompressor tests. Havent gotten to it yet. I got the streaming
> compressor working by using a custom OutputStream class that captures
> the compressed output in chunks.
>
> Im using Apache commons compress. It seems the projects you guys
> mentioned was somehow integrated into the apache commons project. At
> least, thats what I conclude based on the API's.
>
> I'll complete the BZ2Decompressor sometime this week.
>
Great! I just ran into some trouble with our lack of bzip2 this week -
trying to get celery installed via pip threw an error as it tried to use
the bzip2 version of some dependencies. Thanks for looking into this!

-Frank

Update:

Ive completed the module. All tests are passing. Im just having issues
with mercurial. Im a git user so I tried to do my development in a
named branch and rebase onto default when done. This turned out to be
not such a good idea. Im now trying to figure out how to get a patch
with just my commits.

On 22 October 2012 04:01, fwierzbicki@... <fwierzbicki@...> wrote:
>
>
> On Sun, Oct 21, 2012 at 1:08 PM, Julian Kennedy <juliankenn@...>
> wrote:
>>
>> Just an update. I managed to get all the tests passing except the
>> BZ2Decompressor tests. Havent gotten to it yet. I got the streaming
>> compressor working by using a custom OutputStream class that captures
>> the compressed output in chunks.
>>
>> Im using Apache commons compress. It seems the projects you guys
>> mentioned was somehow integrated into the apache commons project. At
>> least, thats what I conclude based on the API's.
>>
>> I'll complete the BZ2Decompressor sometime this week.
>
> Great! I just ran into some trouble with our lack of bzip2 this week -
> trying to get celery installed via pip threw an error as it tried to use the
> bzip2 version of some dependencies. Thanks for looking into this!
>
> -Frank

Oh, and I also tested the module out by install twisted via pip. This
downloaded the tar.bz2 file and installed correctly. So I know at
least pip installations will now work.

On 26 October 2012 00:17, Julian Kennedy <juliankenn@...> wrote:
> Update:
>
> Ive completed the module. All tests are passing. Im just having issues
> with mercurial. Im a git user so I tried to do my development in a
> named branch and rebase onto default when done. This turned out to be
> not such a good idea. Im now trying to figure out how to get a patch
> with just my commits.
>
> On 22 October 2012 04:01, fwierzbicki@... <fwierzbicki@...> wrote:
>>
>>
>> On Sun, Oct 21, 2012 at 1:08 PM, Julian Kennedy <juliankenn@...>
>> wrote:
>>>
>>> Just an update. I managed to get all the tests passing except the
>>> BZ2Decompressor tests. Havent gotten to it yet. I got the streaming
>>> compressor working by using a custom OutputStream class that captures
>>> the compressed output in chunks.
>>>
>>> Im using Apache commons compress. It seems the projects you guys
>>> mentioned was somehow integrated into the apache commons project. At
>>> least, thats what I conclude based on the API's.
>>>
>>> I'll complete the BZ2Decompressor sometime this week.
>>
>> Great! I just ran into some trouble with our lack of bzip2 this week -
>> trying to get celery installed via pip threw an error as it tried to use the
>> bzip2 version of some dependencies. Thanks for looking into this!
>>
>> -Frank

On Thu, Oct 25, 2012 at 3:18 PM, Julian Kennedy <juliankenn@...:

> Oh, and I also tested the module out by install twisted via pip. This
> downloaded the tar.bz2 file and installed correctly. So I know at
> least pip installations will now work.

Great!

-Frank

Hi Guys

I must apologize for not getting back to this module recently. There were
some comments on the pull request about housekeeping issues. The code is
fine, though. I will try to get those cleaned up soon. If anyone is willing
to jump in and help with those, I'd be more than happy about that :-)

Im aware that bz2 support is VERY important to get jython properly working
with virtualenv installs because some module are only released in tar.bz2
format when installed via easy_install/pip. And I think these kind of
compatibility issues are really holding jython back.

I will post another update soon.

Regards


On 27 October 2012 20:20, fwierzbicki@... <fwierzbicki@...:

>
>
> On Thu, Oct 25, 2012 at 3:18 PM, Julian Kennedy <juliankenn@...:
>
>> Oh, and I also tested the module out by install twisted via pip. This
>> downloaded the tar.bz2 file and installed correctly. So I know at
>> least pip installations will now work.
>
> Great!
>
> -Frank
>

On Wed, Nov 21, 2012 at 2:47 AM, Julian Kennedy <juliankenn@...> wrote:
> Hi Guys
>
> I must apologize for not getting back to this module recently. There were
> some comments on the pull request about housekeeping issues.
No worries! We really appreciate the work. The lack of bz2 in Jython
has been a sore point for a while.

> The code is
> fine, though. I will try to get those cleaned up soon. If anyone is willing
> to jump in and help with those, I'd be more than happy about that :-)
>
> Im aware that bz2 support is VERY important to get jython properly working
> with virtualenv installs because some module are only released in tar.bz2
> format when installed via easy_install/pip. And I think these kind of
> compatibility issues are really holding jython back.
Agreed.

> I will post another update soon.
Great!

-Frank