CEDET

Hi,

I would like to commit support for "decorator" and "with" syntax
elements to semantic/wisent/wisent-python.wy. These are only present in
newer Python versions, but extending the grammar should not harm the
parsing of legacy programs. Any objections?

Kind regards,
Jan

Re: [CEDET-devel] Extension of Python syntax From: Eric M. Ludlam <eric@si...> - 2010-02-04 17:38 Hi Jan, I did get your patch earlier, but haven't had time to read it to provide feedback yet. There are quite a few emails I need to think on in fact. (Sorry to all who are waiting.) I did notice that your patch included functions/variables with no doc strings. All vars and fcns should have good doc strings, otherwise I get lost in my own code. I do not have issues with upgrading the grammar as you suggest below. I think it's a fine idea. I also know that there is very poor testing of python parsing in cedet still. (ie - only very basic parsing.) If you happen to be using the python parser with smart completion, srecode, cogre, or any other features, adding tests there would be a huge win while developing changes as the one you propose below. Thanks Eric On 02/03/2010 07:18 PM, Jan Moringen wrote: > Hi, > > I would like to commit support for "decorator" and "with" syntax > elements to semantic/wisent/wisent-python.wy. These are only present in > newer Python versions, but extending the grammar should not harm the > parsing of legacy programs. Any objections? > > Kind regards, > Jan

Re: [CEDET-devel] Extension of Python syntax From: Eric M. Ludlam <eric@si...> - 2010-02-05 01:21 On 02/04/2010 07:50 PM, Jan Moringen wrote: >> I also know that there is very poor testing of python parsing in cedet >> still. (ie - only very basic parsing.) If you happen to be using the >> python parser with smart completion, srecode, cogre, or any other >> features, adding tests there would be a huge win while developing >> changes as the one you propose below. > > I will to commit the changes together with proper tests. Thanks for the > suggestion. There are a lot of test harnesses around for different things. I don't know what bits of the python parser are working for you, so if you need a suggestion of which harness to use, let me know. Eric

Re: [CEDET-devel] Extension of Python syntax From: Jan Moringen <jan.moringen@un...> - 2010-02-05 07:49 > I did get your patch earlier, but haven't had time to read it to > provide > feedback yet. There are quite a few emails I need to think on in > fact. > (Sorry to all who are waiting.) Thanks for the reply. > I did notice that your patch included functions/variables with no doc > strings. All vars and fcns should have good doc strings, otherwise I > get lost in my own code. I already committed some of these changes (patch sets 8001 - 8003 according to sourceforge), since I took the absence of replies to indicate no objections. I added doc strings before committing anyway. > I do not have issues with upgrading the grammar as you suggest below. > I think it's a fine idea. > > I also know that there is very poor testing of python parsing in cedet > still. (ie - only very basic parsing.) If you happen to be using the > python parser with smart completion, srecode, cogre, or any other > features, adding tests there would be a huge win while developing > changes as the one you propose below. I will to commit the changes together with proper tests. Thanks for the suggestion. Kind regards, Jan

Re: [CEDET-devel] Extension of Python syntax From: Jan Moringen <jan.moringen@un...> - 2010-02-05 23:41 On Thu, 2010-02-04 at 20:24 -0500, Eric M. Ludlam wrote: > On 02/04/2010 07:50 PM, Jan Moringen wrote: > >> I also know that there is very poor testing of python parsing in > cedet > >> still. (ie - only very basic parsing.) If you happen to be using > the > >> python parser with smart completion, srecode, cogre, or any other > >> features, adding tests there would be a huge win while developing > >> changes as the one you propose below. > > > > I will to commit the changes together with proper tests. Thanks for > the > > suggestion. > > There are a lot of test harnesses around for different things. I > don't know what bits of the python parser are working for you, so if you > need a suggestion of which harness to use, let me know. Apart from minor problems (which I have partly fixed in my CEDET copy), the parser works quite well. Since the enhancements are on the grammar level, I would like to do unit tests on the tag level. Is there a framework for that? I have more improvements for the Python parser queued. These will mostly be on the same level (grammar and some support functions), so any test advice would still apply. Jan

Re: [CEDET-devel] Extension of Python syntax From: Eric M. Ludlam <eric@si...> - 2010-02-06 03:01 On 02/05/2010 06:41 PM, Jan Moringen wrote: > On Thu, 2010-02-04 at 20:24 -0500, Eric M. Ludlam wrote: >> On 02/04/2010 07:50 PM, Jan Moringen wrote: >>>> I also know that there is very poor testing of python parsing in >> cedet >>>> still. (ie - only very basic parsing.) If you happen to be using >> the >>>> python parser with smart completion, srecode, cogre, or any other >>>> features, adding tests there would be a huge win while developing >>>> changes as the one you propose below. >>> >>> I will to commit the changes together with proper tests. Thanks for >> the >>> suggestion. >> >> There are a lot of test harnesses around for different things. I >> don't know what bits of the python parser are working for you, so if you >> need a suggestion of which harness to use, let me know. > > Apart from minor problems (which I have partly fixed in my CEDET copy), > the parser works quite well. > > Since the enhancements are on the grammar level, I would like to do unit > tests on the tag level. Is there a framework for that? Hi Jan, The semantic-utest.el file has the basics to a python parsing unit test. As you can see, that, and many others, are all very basic. For C++, must of the parsing is tested as part of the smart completion tests, where those can't pass unless the parser works well. At the same time, the C test also is a bit more advanced, in that it has some minor edits testing the incremental parser, though it didn't go very far. Anyway, there is basic infrastructure there to extend for Python for the parsing of constructs, even though the entirety of the test should be expanded. Thanks! Eric

Re: [CEDET-devel] Extension of Python syntax From: Jan Moringen <jan.moringen@un...> - 2010-02-09 01:29 Hi Eric. > The semantic-utest.el file has the basics to a python parsing unit > test. As you can see, that, and many others, are all very basic. [...] > Anyway, there is basic infrastructure there to extend for Python for > the parsing of constructs, even though the entirety of the test should be > expanded. I extended the Python test code in semantic-utest.el to cover a greater fraction of the (unmodified) Python grammar. Could you have a look? I'm not sure why there is overlay information in the expected tags for most languages. I did not add (much) overlay information since it does not seem to get used anywhere. Next, I will add the grammar extensions and corresponding tests. Jan

Re: [CEDET-devel] Extension of Python syntax From: Eric M. Ludlam <eric@si...> - 2010-02-09 04:07 On 02/08/2010 07:29 PM, Jan Moringen wrote: > Hi Eric. > >> The semantic-utest.el file has the basics to a python parsing unit >> test. As you can see, that, and many others, are all very basic. > > [...] > >> Anyway, there is basic infrastructure there to extend for Python for >> the parsing of constructs, even though the entirety of the test should be >> expanded. > > I extended the Python test code in semantic-utest.el to cover a greater > fraction of the (unmodified) Python grammar. > > Could you have a look? That's certainly the right idea, and it passes tests when I run it. > I'm not sure why there is overlay information in > the expected tags for most languages. I did not add (much) overlay > information since it does not seem to get used anywhere. I'm not sure about the (overlay ...) tokens. I'd guess they need to exist to pass some test elsewhere testing if the tag has a position. > Next, I will add the grammar extensions and corresponding tests. Thanks! Eric

1 message has been excluded from this view by a project administrator.