phpwiki-talk

Link tokenization (was Re: [Phpwiki-talk] Re: Zip meta-data)

[Jeff D. <da...@da...>]

In message <147...@da...>,Arno Hollosi writes:
>
> > > Or do you think we should store the intermediate state instead?
> > 
> > Yes, this is an idea I had a long time ago.
> > I thought that [...] it would open up some interesting possibilities
>
>Sure :o)
>I'm in favour of this change. We have to think about some side effects
>like links to pages not yet existing.

As of yet, I'm not at all convinced this is worth the effort.  It's a big
kettle of fish.  I see three reasons to consider saving the pages in
a partially transformed state:

 1. Eliminate name collisions when merging wikis.

    Well, before we can merge wikis we need an interchange format.  (Currently,
    this is shaping up to be my zipfile.)  Then, page name tokenization only
    helps if the interchanged format contains tokenized names.  I'm pretty sure
    this is not the greatest idea.

    So, I think this is a moot point.

 2. Easy automatic link fixing when a page is renamed.

    I don't think this makes it worth the effort.  It will be easy enough
    to translate the links (e.g. with my forthcoming generalized 
wiki_transform)
    in the raw markup.

 3. Faster page rendering.

    This might be an issue.  However, if it is, I think the best way to speed
    page rendering is just to cache the transformed HTML in the database.

Drawbacks of partial pre-transforming:

 1. Complexity.  (Not that I'm not a fan of complexity ;-? )
 2. Bugs in transform code are more likely to cause permanent changes/losses
    of page content.
 3. Inversion of the pre-transform is another kettle of fish, especially if one
    wants to ensure the the output of the inversion matches the original 
markup.
    (Eg. was the link entered as "WikiLink", "[WikiLink]", or "[  WikiLink ]"?)

Jeff

[Phpwiki-talk] Re: Link tokenization

[Steve W. <sw...@wc...>]

On Sun, 16 Jul 2000, Jeff Dairiki wrote:

> As of yet, I'm not at all convinced this is worth the effort.  It's a big
> kettle of fish.  I see three reasons to consider saving the pages in
> a partially transformed state:
> 
>  1. Eliminate name collisions when merging wikis.
> 
>     So, I think this is a moot point.

Yes, I was recounting that I thought about the problem when I was trying
to merge two Wikis, not that it was a motivating factor here. Our criteria
here should be, will this bring real benefits?

>  2. Easy automatic link fixing when a page is renamed.
> 
>     I don't think this makes it worth the effort.  It will be easy enough
>     to translate the links (e.g. with my forthcoming generalized 
> wiki_transform)
>     in the raw markup.

OK...

>  3. Faster page rendering.
> 
>     This might be an issue.  However, if it is, I think the best way to speed
>     page rendering is just to cache the transformed HTML in the database.

Oh, I hadn't thought of that... but I doubt the gain would be much.

> Drawbacks of partial pre-transforming:
> 
>  1. Complexity.  (Not that I'm not a fan of complexity ;-? )
>  2. Bugs in transform code are more likely to cause permanent changes/losses
>     of page content.
>  3. Inversion of the pre-transform is another kettle of fish, especially if one
>     wants to ensure the the output of the inversion matches the original 
> markup.
>     (Eg. was the link entered as "WikiLink", "[WikiLink]", or "[  WikiLink ]"?)

No doubt, there is risk. "Fortune favors the bold." :-)
Point 3 is particularly troublesome though.

I sketched out a simple pair of algorithms to help me think about this:

On submit of a page:
   replace all links with tokens
   update list of links
   save page to db
   save list to db

On select of page:
   fetch page
   fetch link list
   replace tokens with links
   send page to user

So what are the possible benefits? Here's a list of what I thought of:

Undefined pages can be listed separately (ListOfUndefinedPages), page
names can be changed, pages can be deleted (and links to deleted pages
would be replaced with a message, or link to a different page)

Three things, initially. 

sw

...............................ooo0000ooo.................................
   Hear FM quality freeform radio through the Internet: http://wcsb.org/
                     home page: www.wcsb.org/~swain 

Re: [Phpwiki-talk] Re: Link tokenization

[Arno H. <aho...@in...>]

I gave this some more thought. Here's what I've come up with.
Let me state again that the wikilink table can be used
with or without link tokenization. The benefits of this table are not
bound to tokenization.

Pros:
   * Eliminate name collisions when merging wikis -- long term benefit
   * Easy automatic link fixing when a page is renamed -- short term
     benefit for a seldom (or not so seldom) used feature
   * pages (and their referencing links) can be deleted easily -- short
     term benefit for a seldom (or not so seldom) used feature.

Note that the last two points "Seldom vs. often used feature" depends
on what kind of wiki you are running. In common wikis they would be
used seldom I reckon.

Page deletes *without* deleting references can be easily done without
tokenization too.

Con:
   * Complexity and if it becomes too complex bugs may cause "Bad Things".


Other things mentioned:

 > Undefined pages can be listed separately (ListOfUndefinedPages)

This can be done without tokenization as well.
Or is there more to this and I've overlooked something essential?

 > >  3. Inversion of the pre-transform [is hairy]
 > >     (Eg. was the link entered as "WikiLink", "[WikiLink]",
 > >      or "[  WikiLink ]"?)

This is a moot point. Say links are always displayed as [WikiLink]
to editors afterwards. What's the drawback?

 > >  2. Bugs in transform code are more likely to cause
 > >     permanent changes/losses of page content.

Only if the transform code becomes too complex.

 > >  3. Faster page rendering.

Wether or not this is true: it's a moot point.


To sum it up: some (small?) short term benefits plus a long term
benefit weighed against added complexity.

I vote for postponing this change until 1.4. 
Eventually it will be done, but 1.2 is too early for this.

Let's concentrate on the high priority issues first:
- making phpwiki more modular for easier extension and customization
- refactoring the db interface (going oop?)
- adding new navigation possibilities through use of our
  new db schema.

When this is done we can roll out 1.2.
And then we can start the really crazy things.

Jeff, I'd really like to see the class definitions of your
transform code.


/Arno

P.S: I have to switch vacation with a colleague. This means that
I'm on vacation from Thursday until end of July. Probably without
email access, but unable to code on phpwiki for sure :(

-- 
secret plan:
1. world domination
2. get all cookies
3. eat all cookies

Re: [Phpwiki-talk] Re: Link tokenization

[Steve W. <sw...@wc...>]

Great summary, Arno. As long as we architect 1.2 with this possibility in
mind, I'm happy.

sw


On Mon, 17 Jul 2000, Arno Hollosi wrote:

> 
> I gave this some more thought. Here's what I've come up with.
> Let me state again that the wikilink table can be used
> with or without link tokenization. The benefits of this table are not
> bound to tokenization.
> 
> Pros:
>    * Eliminate name collisions when merging wikis -- long term benefit
>    * Easy automatic link fixing when a page is renamed -- short term
>      benefit for a seldom (or not so seldom) used feature
>    * pages (and their referencing links) can be deleted easily -- short
>      term benefit for a seldom (or not so seldom) used feature.
> 
> Note that the last two points "Seldom vs. often used feature" depends
> on what kind of wiki you are running. In common wikis they would be
> used seldom I reckon.
> 
> Page deletes *without* deleting references can be easily done without
> tokenization too.
> 
> Con:
>    * Complexity and if it becomes too complex bugs may cause "Bad Things".
> 
> 
> Other things mentioned:
> 
>  > Undefined pages can be listed separately (ListOfUndefinedPages)
> 
> This can be done without tokenization as well.
> Or is there more to this and I've overlooked something essential?
> 
>  > >  3. Inversion of the pre-transform [is hairy]
>  > >     (Eg. was the link entered as "WikiLink", "[WikiLink]",
>  > >      or "[  WikiLink ]"?)
> 
> This is a moot point. Say links are always displayed as [WikiLink]
> to editors afterwards. What's the drawback?
> 
>  > >  2. Bugs in transform code are more likely to cause
>  > >     permanent changes/losses of page content.
> 
> Only if the transform code becomes too complex.
> 
>  > >  3. Faster page rendering.
> 
> Wether or not this is true: it's a moot point.
> 
> 
> To sum it up: some (small?) short term benefits plus a long term
> benefit weighed against added complexity.
> 
> I vote for postponing this change until 1.4. 
> Eventually it will be done, but 1.2 is too early for this.
> 
> Let's concentrate on the high priority issues first:
> - making phpwiki more modular for easier extension and customization
> - refactoring the db interface (going oop?)
> - adding new navigation possibilities through use of our
>   new db schema.
> 
> When this is done we can roll out 1.2.
> And then we can start the really crazy things.
> 
> Jeff, I'd really like to see the class definitions of your
> transform code.
> 
> 
> /Arno
> 
> P.S: I have to switch vacation with a colleague. This means that
> I'm on vacation from Thursday until end of July. Probably without
> email access, but unable to code on phpwiki for sure :(
> 
> -- 
> secret plan:
> 1. world domination
> 2. get all cookies
> 3. eat all cookies
> 
> _______________________________________________
> Phpwiki-talk mailing list
> Php...@li...
> http://lists.sourceforge.net/mailman/listinfo/phpwiki-talk
> 

................................ooo0000ooo.................................
   Hear FM quality freeform radio through the Internet: http://wcsb.org/
                     home page: www.wcsb.org/~swain