From: Bruce F. <Bruce@Fitzsimons.org> - 2003-11-03 11:04:02
|
I've hacked the wiki a bit so it runs as an appmod to see if Google treats it differently. For those few of you who are unaware, I've got some problems with Google and other search engines not indexing my wiki. My understanding of the problem has progressed. The problem seems to be that it thinks each .yaws is only one page. So showPage.yaws gets one place in the index, with the highest linked page winning (node=home). Pages below that pointing to showPage.yaws are not indexed, despite being explictly allowed by the robots meta tag. My theory is confirmed by the fact that showOldPage.yaws etc etc also appear once in the index. My hack to run the wiki as an appmod is really ugly. I will refine it if it works. Basically I have a /wikifile/ appmod that calls wiki:showPage(), and I've hacked the html generation to change the links to this style. I will keep the list informed about the results of this test (whether they like it or not :-) /Bruce |
From: <kl...@hy...> - 2003-11-03 13:01:05
|
On Tue, Nov 04, 2003 at 12:03:56AM +1300, Bruce Fitzsimons wrote: > I've hacked the wiki a bit so it runs as an appmod to see if Google treats > it differently. For those few of you who are unaware, I've got some problems > with Google and other search engines not indexing my wiki. Do php,asp .... pages have the same problem ? or is google aware of the suffixes for different scripting languges ? /klacke -- Claes Wikstrom -- Caps lock is nowhere and http://www.hyber.org -- everything is under control |
From: Carsten S. <ca...@gn...> - 2003-11-03 22:05:58
|
Hi Bruce! On Tue, Nov 04, 2003 at 12:03:56AM +1300, Bruce Fitzsimons wrote: > I've hacked the wiki a bit so it runs as an appmod to see if Google treats > it differently. For those few of you who are unaware, I've got some probl= ems > with Google and other search engines not indexing my wiki. >=20 > My understanding of the problem has progressed. The problem seems to be t= hat > it thinks each .yaws is only one page. So showPage.yaws gets one place in > the index, with the highest linked page winning (node=3Dhome). You might just have tried replacing showPage.yaws?node=3Dfoo by showPage.yaws/foo. `/foo' would have been available as Arg#arg.pathinfo. > Pages below that pointing to showPage.yaws are not indexed, despite > being explictly allowed by the robots meta tag. Somewhere on google.com they write that they are not keen on traversing a possibly infinite URL space. Greetings, Carsten --=20 Carsten Schultz (2:38, 33:47), FB Mathematik, FU Berlin http://carsten.fu-mathe-team.de/ PGP/GPG key on the pgp.net key servers,=20 fingerprint on my home page. |
From: Bruce F. <Bruce@Fitzsimons.org> - 2003-11-06 07:56:38
|
Hey Carsten, ----- Original Message ----- From: "Carsten Schultz" <ca...@gn...> >You might just have tried replacing showPage.yaws?node=foo by > showPage.yaws/foo. `/foo' would have been available as > Arg#arg.pathinfo. Hmmm. That *would* have been easier. Damn. Then I could have made them foo.html too :-) We should be able to add that to the yaws wiki without harming anyone, unless people violently disagree. > > Pages below that pointing to showPage.yaws are not indexed, despite > > being explictly allowed by the robots meta tag. > Somewhere on google.com they write that they are not keen on > traversing a possibly infinite URL space. I hadn't seen that, but its somewhat reasonable. I'm sure their spiders get tied up in knots occasionally -- I wonder what they max depth is. I'm sure *some* dynamic sites get indexed though, although I can't point to a specific example. In fact I've just seen my local auction site (trademe.co.nz) has done a similar thing to me in order to get indexed. Thanks a lot for your help. /Bruce |
From: Mickael R. <mic...@er...> - 2003-11-06 08:48:08
|
* Bruce Fitzsimons <Bruce@Fitzsimons.org> [2003-11-06 20:55:03 +1300]: > Hey Carsten, >=20 > ----- Original Message -----=20 > From: "Carsten Schultz" <ca...@gn...> >=20 > >You might just have tried replacing showPage.yaws?node=3Dfoo by > > showPage.yaws/foo. `/foo' would have been available as > > Arg#arg.pathinfo. >=20 > Hmmm. That *would* have been easier. Damn. Then I could have made them > foo.html too :-) > We should be able to add that to the yaws wiki without harming anyone, > unless people violently disagree. No problem for my side. I think it is important to produce page that can be indexed by Google. --=20 Micka=EBl R=E9mond http://www.erlang-projects.org/ |