cclib-devel

[cclib-devel] Further JSON support

[Karol L. <kar...@kn...>]

Hi,

 A clarification on my latest commit. Since I started to use a different JSON 
encoder/decoder than simplejson, I decided to add support for them in cclib.

 One of simplejson, cjson ot json is loaded dynamically when load_json() is 
called in data.py. It is only loaded once, unless an optional argument for 
the module name is passed (can be passed through read_json, etc.). An 
example:

>>> import cclib
>>> m = cclib.parser.ccopen("dvb_sp.out").parse()
(...)
>>> m.writejson(outfile="x.simplejson", module="simplejson")
>>> m.writejson(outfile="x.json", module="json")
>>> m.writejson(outfile="x.cjson", module="cjson")

This writes three files using the three different modules. So,

x.cjson is 63K
x.json is 69K
x.simplejson is 105K

I don't know if any of these modules is better or faster or if it matters, but 
since simplejson was first it is the default (being the first to try to 
load).

Cheers,
Karol

-- 
written by Karol Langner
Mon Nov 19 14:26:59 CET 2007

Re: [cclib-devel] Further JSON support

[Noel O'B. <bao...@gm...>]

Do we really need support for three JSON libraries? We can just bundle
simplejson (or one of the others if their licenses permit) with cclib.

Noel

On 19/11/2007, Karol Langner <kar...@kn...> wrote:
> Hi,
>
>  A clarification on my latest commit. Since I started to use a different JSON
> encoder/decoder than simplejson, I decided to add support for them in cclib.
>
>  One of simplejson, cjson ot json is loaded dynamically when load_json() is
> called in data.py. It is only loaded once, unless an optional argument for
> the module name is passed (can be passed through read_json, etc.). An
> example:
>
> >>> import cclib
> >>> m = cclib.parser.ccopen("dvb_sp.out").parse()
> (...)
> >>> m.writejson(outfile="x.simplejson", module="simplejson")
> >>> m.writejson(outfile="x.json", module="json")
> >>> m.writejson(outfile="x.cjson", module="cjson")
>
> This writes three files using the three different modules. So,
>
> x.cjson is 63K
> x.json is 69K
> x.simplejson is 105K
>
> I don't know if any of these modules is better or faster or if it matters, but
> since simplejson was first it is the default (being the first to try to
> load).
>
> Cheers,
> Karol
>
> --
> written by Karol Langner
> Mon Nov 19 14:26:59 CET 2007
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>

Re: [cclib-devel] Further JSON support

[Karol L. <kar...@kn...>]

On Monday 19 November 2007 14:38, Noel O'Boyle wrote:
> Do we really need support for three JSON libraries? We can just bundle
> simplejson (or one of the others if their licenses permit) with cclib.

I don't like it either, but it seems to be the simplest fix for now if I want 
to use cjson. Bundling is probably the best final solution. I would opt for 
cjson in that case, although it is a C extension and would require 
compilation upon install, which cclib has avoided up to now.

- Karol

-- 
written by Karol Langner
Mon Nov 19 15:02:50 CET 2007

Re: [cclib-devel] Further JSON support

[Noel O'B. <bao...@gm...>]

On 19/11/2007, Karol Langner <kar...@kn...> wrote:
> On Monday 19 November 2007 14:38, Noel O'Boyle wrote:
> > Do we really need support for three JSON libraries? We can just bundle
> > simplejson (or one of the others if their licenses permit) with cclib.
>
> I don't like it either, but it seems to be the simplest fix for now if I want
> to use cjson. Bundling is probably the best final solution. I would opt for
> cjson in that case, although it is a C extension and would require
> compilation upon install, which cclib has avoided up to now.

Yes - I wouldn't like to do this for the 'core' parsing. How about
changing so that rather than passing in a module, you just pass in the
dump/load function? Then cclib wouldn't have to do any of this. The
user would do "from simplejson import dumps", and then pass "dumps" as
a parameter to write_json.

> - Karol
>
> --
> written by Karol Langner
> Mon Nov 19 15:02:50 CET 2007
>

Re: [cclib-devel] Further JSON support

[Karol L. <kar...@kn...>]

On Monday 19 November 2007 15:21, Noel O'Boyle wrote:
> On 19/11/2007, Karol Langner <kar...@kn...> wrote:
> > On Monday 19 November 2007 14:38, Noel O'Boyle wrote:
> > > Do we really need support for three JSON libraries? We can just bundle
> > > simplejson (or one of the others if their licenses permit) with cclib.
> >
> > I don't like it either, but it seems to be the simplest fix for now if I
> > want to use cjson. Bundling is probably the best final solution. I would
> > opt for cjson in that case, although it is a C extension and would
> > require compilation upon install, which cclib has avoided up to now.
>
> Yes - I wouldn't like to do this for the 'core' parsing.

Does this mean you would consider C extensions appropriate for ex. 
cclib.methods if they made them ore efficient?

> How about 
> changing so that rather than passing in a module, you just pass in the
> dump/load function? Then cclib wouldn't have to do any of this. The
> user would do "from simplejson import dumps", and then pass "dumps" as
> a parameter to write_json.

I'm not sure I understand: you propose to bundle a pure python JSON module for 
the default (simplejson or json), and add an optional dumps/loads argument if 
the user wants an alternate function?

- Karol

-- 
written by Karol Langner
Mon Nov 19 15:36:43 CET 2007

Re: [cclib-devel] Further JSON support

[Noel O'B. <bao...@gm...>]

On 19/11/2007, Karol Langner <kar...@kn...> wrote:
> On Monday 19 November 2007 15:21, Noel O'Boyle wrote:
> > On 19/11/2007, Karol Langner <kar...@kn...> wrote:
> > > On Monday 19 November 2007 14:38, Noel O'Boyle wrote:
> > > > Do we really need support for three JSON libraries? We can just bundle
> > > > simplejson (or one of the others if their licenses permit) with cclib.
> > >
> > > I don't like it either, but it seems to be the simplest fix for now if I
> > > want to use cjson. Bundling is probably the best final solution. I would
> > > opt for cjson in that case, although it is a C extension and would
> > > require compilation upon install, which cclib has avoided up to now.
> >
> > Yes - I wouldn't like to do this for the 'core' parsing.
>
> Does this mean you would consider C extensions appropriate for ex.
> cclib.methods if they made them ore efficient?

I think I've answered this on another thread a few weeks ago. The
short answer is yes.

> > How about
> > changing so that rather than passing in a module, you just pass in the
> > dump/load function? Then cclib wouldn't have to do any of this. The
> > user would do "from simplejson import dumps", and then pass "dumps" as
> > a parameter to write_json.
>
> I'm not sure I understand: you propose to bundle a pure python JSON module for
> the default (simplejson or json), and add an optional dumps/loads argument if
> the user wants an alternate function?
Maybe - what do you think? Alternatively, we could forget about
bundling or importing the module ourselves, and just leave it to the
user to pass in the dumps/loads function. This would make our code
much cleaner, and fits the "Only one good way to do it" philosophy of
Python. I will go with this as my final proposal...

> - Karol
>
> --
> written by Karol Langner
> Mon Nov 19 15:36:43 CET 2007
>

Re: [cclib-devel] Further JSON support

[Karol L. <kar...@kn...>]

On Monday 19 November 2007 15:56, Noel O'Boyle wrote:
> > > How about
> > > changing so that rather than passing in a module, you just pass in the
> > > dump/load function? Then cclib wouldn't have to do any of this. The
> > > user would do "from simplejson import dumps", and then pass "dumps" as
> > > a parameter to write_json.
> >
> > I'm not sure I understand: you propose to bundle a pure python JSON
> > module for the default (simplejson or json), and add an optional
> > dumps/loads argument if the user wants an alternate function?
>
> Maybe - what do you think? Alternatively, we could forget about
> bundling or importing the module ourselves, and just leave it to the
> user to pass in the dumps/loads function. This would make our code
> much cleaner, and fits the "Only one good way to do it" philosophy of
> Python. I will go with this as my final proposal...

I have no definite opinion. If there is no actual JSON support in cclib 
(bundled or imported), then there's no point in having a 'writejson' method 
is there? Just an ambigious 'write' method, since you can pass a function 
that does _anything_.

Karol

-- 
written by Karol Langner
Mon Nov 19 16:22:20 CET 2007

Re: [cclib-devel] Further JSON support

[Noel O'B. <bao...@gm...>]

Good point. The user just needs to call listify() beforehand. In that
case, listify() should probably return a dictionary, rather than
altering the ccData object itself. The dictionary can be written with
whatever JSON library the user wants to use...

Great! I think we have a winner :-)

Noel

On 19/11/2007, Karol Langner <kar...@kn...> wrote:
> I have no definite opinion. If there is no actual JSON support in cclib
> (bundled or imported), then there's no point in having a 'writejson' method
> is there? Just an ambigious 'write' method, since you can pass a function
> that does _anything_.
>
> Karol
>
> --
> written by Karol Langner
> Mon Nov 19 16:22:20 CET 2007
>

Re: [cclib-devel] Further JSON support

[Noel O'B. <bao...@gm...>]

On 19/11/2007, Karol Langner <kar...@kn...> wrote:
> On Monday 19 November 2007 16:29, you wrote:
> > Good point. The user just needs to call listify() beforehand. In that
> > case, listify() should probably return a dictionary, rather than
> > altering the ccData object itself. The dictionary can be written with
> > whatever JSON library the user wants to use...
> >
> > Great! I think we have a winner :-)
>
> I recently added the "getattributes" method, which listifies and returns such
> a dictionary (and later arrayifies) - so that's all that needs to be called.

Well, my point was just that there's no need to call arrayify() if
listify just returns a dictionary. It's also a cleaner solution as it
means that the ccData object will not be perturbed. listify() is not
used by any other code, as far as I know.

> I'll go with anything here. So we are choosing not to provide any intrinsic
> JSON support?
I think so. If there are lots of different libraries that a user might
want to use, the best solution is to allow the user to choose
whichever they want.
> What about pickle?
pickle works already just like for any other Python object. See the
docs on the wiki at:
http://cclib.sourceforge.net/wiki/index.php/Using_cclib

 > Karol
>
> --
> written by Karol Langner
> Mon Nov 19 16:55:31 CET 2007
>

Re: [cclib-devel] Further JSON support

[Karol L. <kar...@kn...>]

I removed all pickle/json functions and methods from data.py. It does seem 
healthier to leave the read/write layer outside of cclib.

I left arrayify, listify, getattributes and setatttributes as they are. So, if 
someone wants a dictionary of attributes for JSON, getattributes is the best 
function, after calling listify (also just added an optional 'tolists' 
argument that does this).

Karol

-- 
written by Karol Langner
Mon Nov 19 19:09:37 CET 2007