From: Louis-Philippe H. <lph...@lp...> - 2010-10-26 18:41:05
|
Hello everyone, As some of you might have noticed, I have been a bit more active than usual lately. With the LTS being just about out of the door, it's time for me to begin focusing on some of the larger changes that await Tiki. I began today by replacing the plugin parser which has been lingering in subversion for a long time. Some breakage is expected, but the change was a required step to allow a more powerful syntax for the new search/reporting plugin that will come with some major changes. The plugin won't be build just yet, but I wanted to make this major change as early on as possible in the process so that the issues can be identified and fixed long before the release. Essentially, the parser was inverted. In the past, when plugins were nested, the inner portions would be evaluated prior to the outer portion, leading the outer portion to read HTML (or mixed wiki/HTML) rather than wiki syntax when looking at the incoming data. It also required hacks in the parser to get around the code plugin which requires to keep the content verbatim. With the new implementation, the input to plugin is precisely what is written in the text, allowing to inspect it like plain text. The plugins that require the inner parts to be executed first will be allowed to do it themselves. Otherwise, the parser should behave the same for normal cases where the nesting is correct. Incorrect matchings may lead to different artifacts, but the page was wrong. There is an extensive suite of test on the matcher and more cases will be added if issues are found. I also tested with the content of the home page on doc.tiki.org to validate a complex case (complex does not begin to express what that page is, it's actually not matching correctly), and the rendering is about correct. The parser is now implemented with code rather than regular expressions and is guaranteed to finish at some point. No more "loop until cooked", for those who dared to look at the old implementation. Now, for the upcoming changes. I intend on modifying the way Tiki handles the listing of objects globally. Once completed, it will affect the site search, individual feature listing and most listing plugins. It's a major undertaking that will simplify the codebase significantly, as well as provide more consistency for users. More details here: http://dev.tiki.org/Unified+Search Comments are welcome. It's still a fairly early draft and few decisions are final. -- LP |
From: Jonny B. <jo...@ti...> - 2010-11-02 11:56:48
|
(oops, found in drafts from a while ago) Hi LP - welcome back! ;) This all sounds good to me, having spent most of the summer grubbing about in parse_data i know it needs some straightening out! We still have some issues in 6.x (i.e. 6.0) where the treatment of plugin contents seems to have changed already (e.g. SPLIT and SORT) leading to regressions that i don't think i can fix in the current architecture... Also the plan for the new unified search (and presumably more unified access to objects) sounds great, and long overdue. Sorry i don't have time now to add more, but i just wanted you to know someone had read your mail (and have been keeping an eye on the diffs from the wiki page) and agree this is certainly the way forward! :) Thanks! jb On 26 Oct 2010, at 19:40, Louis-Philippe Huberdeau wrote: > Hello everyone, > > As some of you might have noticed, I have been a bit more active than usual lately. With the LTS being just about out of the door, it's time for me to begin focusing on some of the larger changes that await Tiki. I began today by replacing the plugin parser which has been lingering in subversion for a long time. Some breakage is expected, but the change was a required step to allow a more powerful syntax for the new search/reporting plugin that will come with some major changes. The plugin won't be build just yet, but I wanted to make this major change as early on as possible in the process so that the issues can be identified and fixed long before the release. > > Essentially, the parser was inverted. In the past, when plugins were nested, the inner portions would be evaluated prior to the outer portion, leading the outer portion to read HTML (or mixed wiki/HTML) rather than wiki syntax when looking at the incoming data. It also required hacks in the parser to get around the code plugin which requires to keep the content verbatim. With the new implementation, the input to plugin is precisely what is written in the text, allowing to inspect it like plain text. The plugins that require the inner parts to be executed first will be allowed to do it themselves. > > Otherwise, the parser should behave the same for normal cases where the nesting is correct. Incorrect matchings may lead to different artifacts, but the page was wrong. There is an extensive suite of test on the matcher and more cases will be added if issues are found. I also tested with the content of the home page on doc.tiki.org to validate a complex case (complex does not begin to express what that page is, it's actually not matching correctly), and the rendering is about correct. > > The parser is now implemented with code rather than regular expressions and is guaranteed to finish at some point. No more "loop until cooked", for those who dared to look at the old implementation. > > Now, for the upcoming changes. I intend on modifying the way Tiki handles the listing of objects globally. Once completed, it will affect the site search, individual feature listing and most listing plugins. It's a major undertaking that will simplify the codebase significantly, as well as provide more consistency for users. > > More details here: http://dev.tiki.org/Unified+Search > > Comments are welcome. It's still a fairly early draft and few decisions are final. > > -- > LP > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America contest > Create new apps & games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev_______________________________________________ > Tikiwiki-devel mailing list > Tik...@li... > https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel |
From: Sylvie G. <sgr...@gm...> - 2010-11-02 13:57:21
|
Louis Philippe, Btw I tried this in trunk {SPLIT(first=col)}r1c1---r2c1---{SPLIT()}m1c1---m2c1@@@m1c2{SPLIT}@@@r1c2---{SPLIT()}g1c1@@@g1c2{SPLIT}---r3c2{SPLIT} {SPLIT()}r1c1---r2c1@@@rrr{SPLIT} This is not correctly parsed. I thought you were parsing now from incuding to included and with a real parser not a preg_match My 2 cents sylvie On Tue, 2010-10-26 at 14:40 -0400, Louis-Philippe Huberdeau wrote: > Hello everyone, > > > As some of you might have noticed, I have been a bit more active than > usual lately. With the LTS being just about out of the door, it's time > for me to begin focusing on some of the larger changes that await > Tiki. I began today by replacing the plugin parser which has been > lingering in subversion for a long time. Some breakage is expected, > but the change was a required step to allow a more powerful syntax for > the new search/reporting plugin that will come with some major > changes. The plugin won't be build just yet, but I wanted to make this > major change as early on as possible in the process so that the issues > can be identified and fixed long before the release. > > > Essentially, the parser was inverted. In the past, when plugins were > nested, the inner portions would be evaluated prior to the outer > portion, leading the outer portion to read HTML (or mixed wiki/HTML) > rather than wiki syntax when looking at the incoming data. It also > required hacks in the parser to get around the code plugin which > requires to keep the content verbatim. With the new implementation, > the input to plugin is precisely what is written in the text, allowing > to inspect it like plain text. The plugins that require the inner > parts to be executed first will be allowed to do it themselves. > > > Otherwise, the parser should behave the same for normal cases where > the nesting is correct. Incorrect matchings may lead to different > artifacts, but the page was wrong. There is an extensive suite of test > on the matcher and more cases will be added if issues are found. I > also tested with the content of the home page on doc.tiki.org to > validate a complex case (complex does not begin to express what that > page is, it's actually not matching correctly), and the rendering is > about correct. > > > The parser is now implemented with code rather than regular > expressions and is guaranteed to finish at some point. No more "loop > until cooked", for those who dared to look at the old implementation. > > > Now, for the upcoming changes. I intend on modifying the way Tiki > handles the listing of objects globally. Once completed, it will > affect the site search, individual feature listing and most listing > plugins. It's a major undertaking that will simplify the codebase > significantly, as well as provide more consistency for users. > > > More details here: http://dev.tiki.org/Unified+Search > > > Comments are welcome. It's still a fairly early draft and few > decisions are final. > > > -- > LP > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America contest > Create new apps & games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ Tikiwiki-devel mailing list Tik...@li... https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel |
From: Louis-Philippe H. <lph...@lp...> - 2010-11-02 14:24:58
|
I checked and the execution is made in the right order, as expected. I don't understand from the plugin code what the output should look like. The inner parts appear to be parsed because the plugin converts some new lines to <br/> itself. I can't imagine a valid reason why it would do that. Moreover, there is quite a terrific amount of code in there. I can't possibly imagine it takes so much to do something that simple. -- LP On Tue, Nov 2, 2010 at 9:57 AM, Sylvie Greverend <sgr...@gm...>wrote: > Louis Philippe, > Btw I tried this in trunk > {SPLIT(first=col)}r1c1---r2c1---{SPLIT()}m1c1---m2c1@@@m1c2{SPLIT}@ > @@r1c2---{SPLIT()}g1c1@@@g1c2{SPLIT}---r3c2{SPLIT} > {SPLIT()}r1c1---r2c1@@@rrr{SPLIT} > This is not correctly parsed. I thought you were parsing now from > incuding to included and with a real parser not a preg_match > My 2 cents > sylvie > > On Tue, 2010-10-26 at 14:40 -0400, Louis-Philippe Huberdeau wrote: > > Hello everyone, > > > > > > As some of you might have noticed, I have been a bit more active than > > usual lately. With the LTS being just about out of the door, it's time > > for me to begin focusing on some of the larger changes that await > > Tiki. I began today by replacing the plugin parser which has been > > lingering in subversion for a long time. Some breakage is expected, > > but the change was a required step to allow a more powerful syntax for > > the new search/reporting plugin that will come with some major > > changes. The plugin won't be build just yet, but I wanted to make this > > major change as early on as possible in the process so that the issues > > can be identified and fixed long before the release. > > > > > > Essentially, the parser was inverted. In the past, when plugins were > > nested, the inner portions would be evaluated prior to the outer > > portion, leading the outer portion to read HTML (or mixed wiki/HTML) > > rather than wiki syntax when looking at the incoming data. It also > > required hacks in the parser to get around the code plugin which > > requires to keep the content verbatim. With the new implementation, > > the input to plugin is precisely what is written in the text, allowing > > to inspect it like plain text. The plugins that require the inner > > parts to be executed first will be allowed to do it themselves. > > > > > > Otherwise, the parser should behave the same for normal cases where > > the nesting is correct. Incorrect matchings may lead to different > > artifacts, but the page was wrong. There is an extensive suite of test > > on the matcher and more cases will be added if issues are found. I > > also tested with the content of the home page on doc.tiki.org to > > validate a complex case (complex does not begin to express what that > > page is, it's actually not matching correctly), and the rendering is > > about correct. > > > > > > The parser is now implemented with code rather than regular > > expressions and is guaranteed to finish at some point. No more "loop > > until cooked", for those who dared to look at the old implementation. > > > > > > Now, for the upcoming changes. I intend on modifying the way Tiki > > handles the listing of objects globally. Once completed, it will > > affect the site search, individual feature listing and most listing > > plugins. It's a major undertaking that will simplify the codebase > > significantly, as well as provide more consistency for users. > > > > > > More details here: http://dev.tiki.org/Unified+Search > > > > > > Comments are welcome. It's still a fairly early draft and few > > decisions are final. > > > > > > -- > > LP > > > ------------------------------------------------------------------------------ > > Nokia and AT&T present the 2010 Calling All Innovators-North America > contest > > Create new apps & games for the Nokia N8 for consumers in U.S. and > Canada > > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > marketing > > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > > http://p.sf.net/sfu/nokia-dev2dev > > _______________________________________________ Tikiwiki-devel mailing > list Tik...@li... > https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel > > > > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North America > contest > Create new apps & games for the Nokia N8 for consumers in U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly $6M in > marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > Tikiwiki-devel mailing list > Tik...@li... > https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel > |
From: Sylvie G. <sgr...@gm...> - 2010-11-02 14:52:20
|
{SPLIT(first=col)}r1c1---r2c1---{SPLIT()}m1c1---m2c1@@@m1c2{SPLIT}@@@r1c2---{SPLIT()}g1c1@@@g1c2{SPLIT}---r3c2{SPLIT} should give something like r1c1 r1c2 r2c2 g1c g1c2 m1c1 m2c1 r3c2 m1c2 On trunk I have something like r1c1 m1c2 r1c2 g1c2 r2c1 g1c1 r3c2 m1c1 m2c1 Strange.... I confirm that in trunk and tiki6 {GROUP(groups=>Registered)}{CODE()}CCCCC{CODE}{ELSE}{CODE()}AAAA{CODE}{GROUP} CODE is executed only one time On Tue, 2010-11-02 at 10:24 -0400, Louis-Philippe Huberdeau wrote: > I checked and the execution is made in the right order, as expected. I > don't understand from the plugin code what the output should look > like. > > > The inner parts appear to be parsed because the plugin converts some > new lines to <br/> itself. I can't imagine a valid reason why it would > do that. > > > Moreover, there is quite a terrific amount of code in there. I can't > possibly imagine it takes so much to do something that simple. > > > -- > LP > > On Tue, Nov 2, 2010 at 9:57 AM, Sylvie Greverend > <sgr...@gm...> wrote: > Louis Philippe, > Btw I tried this in trunk > {SPLIT(first=col)}r1c1---r2c1---{SPLIT()}m1c1---m2c1@@@m1c2{SPLIT}@@@r1c2---{SPLIT()}g1c1@@@g1c2{SPLIT}---r3c2{SPLIT} > {SPLIT()}r1c1---r2c1@@@rrr{SPLIT} > This is not correctly parsed. I thought you were parsing now > from > incuding to included and with a real parser not a preg_match > My 2 cents > sylvie > > > On Tue, 2010-10-26 at 14:40 -0400, Louis-Philippe Huberdeau > wrote: > > Hello everyone, > > > > > > As some of you might have noticed, I have been a bit more > active than > > usual lately. With the LTS being just about out of the door, > it's time > > for me to begin focusing on some of the larger changes that > await > > Tiki. I began today by replacing the plugin parser which has > been > > lingering in subversion for a long time. Some breakage is > expected, > > but the change was a required step to allow a more powerful > syntax for > > the new search/reporting plugin that will come with some > major > > changes. The plugin won't be build just yet, but I wanted to > make this > > major change as early on as possible in the process so that > the issues > > can be identified and fixed long before the release. > > > > > > Essentially, the parser was inverted. In the past, when > plugins were > > nested, the inner portions would be evaluated prior to the > outer > > portion, leading the outer portion to read HTML (or mixed > wiki/HTML) > > rather than wiki syntax when looking at the incoming data. > It also > > required hacks in the parser to get around the code plugin > which > > requires to keep the content verbatim. With the new > implementation, > > the input to plugin is precisely what is written in the > text, allowing > > to inspect it like plain text. The plugins that require the > inner > > parts to be executed first will be allowed to do it > themselves. > > > > > > Otherwise, the parser should behave the same for normal > cases where > > the nesting is correct. Incorrect matchings may lead to > different > > artifacts, but the page was wrong. There is an extensive > suite of test > > on the matcher and more cases will be added if issues are > found. I > > also tested with the content of the home page on > doc.tiki.org to > > validate a complex case (complex does not begin to express > what that > > page is, it's actually not matching correctly), and the > rendering is > > about correct. > > > > > > The parser is now implemented with code rather than regular > > expressions and is guaranteed to finish at some point. No > more "loop > > until cooked", for those who dared to look at the old > implementation. > > > > > > Now, for the upcoming changes. I intend on modifying the way > Tiki > > handles the listing of objects globally. Once completed, it > will > > affect the site search, individual feature listing and most > listing > > plugins. It's a major undertaking that will simplify the > codebase > > significantly, as well as provide more consistency for > users. > > > > > > More details here: http://dev.tiki.org/Unified+Search > > > > > > Comments are welcome. It's still a fairly early draft and > few > > decisions are final. > > > > > > -- > > LP > > > > > ------------------------------------------------------------------------------ > > Nokia and AT&T present the 2010 Calling All Innovators-North > America contest > > Create new apps & games for the Nokia N8 for consumers in > U.S. and Canada > > $10 million total in prizes - $4M cash, 500 devices, nearly > $6M in marketing > > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish > to Ovi Store > > http://p.sf.net/sfu/nokia-dev2dev > > _______________________________________________ > Tikiwiki-devel mailing list > Tik...@li... > https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel > > > > ------------------------------------------------------------------------------ > Nokia and AT&T present the 2010 Calling All Innovators-North > America contest > Create new apps & games for the Nokia N8 for consumers in > U.S. and Canada > $10 million total in prizes - $4M cash, 500 devices, nearly > $6M in marketing > Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to > Ovi Store > http://p.sf.net/sfu/nokia-dev2dev > _______________________________________________ > Tikiwiki-devel mailing list > Tik...@li... > https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel > > |