Re: [htdig] Only one page indexed/ noindex_start

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Thanks, Jim. It's possible that  JavaScript is confusing the parser. I can't
hide them as usual due to syntax.

So now I'm thinking that I should employ these tags: noindex_start,
noindex_end 

Because the archives contain conflicting information on noindex, anyone who
has used them please confirm that this is the correct usage:

these tags, <!--htdig_noindex--> and <!--/htdig_noindex-->   surround the
script, thusly:

<!--htdig_noindex-->
<script language="JavaScript" type="text/javascript"
src="http://adserver.com/adscript"></script>
<!--/htdig_noindex-->

And these tags go in the htdig.conf file, like so:
noindex_start: <SCRIPT
noindex_end: </SCRIPT>

Is that right?

Thanks, all!

-- 
Adam Powell
http://theonion.com/

From: Jim Cole <gre...@yg...>
Date: Fri, 8 Nov 2002 21:08:28 -0700
To: Adam Powell <ap...@th...>
Cc: <htd...@li...>
Subject: Re: [htdig] Only one page indexed

On Friday, November 8, 2002, at 12:57 PM, Adam Powell wrote:

> In an interesting new twist, I re-indexed the site, and suddenly have
> only
> the start url in the db. I hadn't changed anything on the
> backend--only the
> html side-- since the last htdig, so I'm puzzled. It's indexing
> correctly,
> but making it through the first page and not following any links.
...
> Tag: </HEAD>, matched -1
> Tag: <BASE TARGET="_parent">, matched 23
> Tag: <BODY BGCOLOR="#FFFFFF" MARGINWIDTH=0 MARGINHEIGHT=0 LEFTMARGIN=0
> TOPMARGIN=0>, matched -1
> Tag: <script language="JavaScript" type="text/javascript"
> src="http://66.216.104.232:80/servlet/ajrotator/79/0/
> viewJScript?pool=52&typ
> e=2137">, matched 29
> Tag: </script>, matched 30
> Tag: </BODY>, matched -1
> Tag: </HTML>, matched -1

I suspect that your JavaScript is confusing the parser. If you don't
enclose your script code in HTML comment tags, some characters in the
code, such as '<', tend to throw the parser into an incorrect state.
The result is that later elements are often not interpreted correctly
(i.e. link elements are not recognized as such). Enclosing JavaScript
code in HTML comment tags is standard advice in many JavaScript
references.

Jim