RE: [htdig] Cold Fusion (.cfm) files and HT://Dig

[Augeri, J. (NM75) <jim...@ho...>]

Geoff,

Sorry to bring this topic back, but it is not working!  I must apologize in
advance, but in order to adequately explain the situation, some background
and detail is necessary.

THE BEFORE CONFIGURATION:

Top page (http://web.../index.html) was a frames based page; dumb header at
the
top, navigation buttons down the left side, main content window to the right
of
the nav buttons and below the header.  The "htdig.conf" file was pointing at
the
root URL of this server and one other at our site.  Life was good, and
everything
was working as it had in preliminary testing.  (We are running HT-Dig 3.1.5
on a
Solaris 5.6/Apache 3.1.12 web server.)

THE CURRENT CONFIGURATION

Top page is now "index.cfm", and takes advantage of several <CFINCLUDE> tags
to
build the actual main page.  Presently, there are three main pieces: the
header
(header.cfm), the main body (index.cfm), and a footer (footer.cfm).
Index.cfm
pulls in the header.cfm and footer.cfm pieces to make a complete page.  The
_new_
header is a lot more intelligent than the old nav panel, with cascading
menus
built out of JavaScript.  The footer.cfm file is pretty dumb right now,
simply
displaying a standard date and Copyright notice.  The entire site was built
using Dreamweaver 4.0.

(I hope it is well understood just how ColdFusions <CFINCLUDE> tags
assembles
individual chunks of HTML and ultimately delivers a complete HTML page back
to
the browser. The page that the viewer sees within his or her browser does
not
actually exist anywhere on the WEB server itself!)

One of the things that is embeded in the new header is the HT-Dig search
link
for our site, which is how I discovered that the spider wasn't indexing.
The
only stuff that is getting indexed under this new scheme is the stuff that
is linked
directly from the main page (the top level "index.cfm" page).  Since most of
the links to "other stuff" within the site is navigated to via the cascading
JavaScript menus (contained in the "header.cfm" file), I tried making it one
of
the URL's in the "htdig.conf" file.  This seems to make no difference
whatsoever
in the outcome.

Under the BEFORE config, when "rundig" was run, it would take about 15-30
minutes
to index the entire site.  There are about 20K documents on the site in
question.
Now, indexing takes all of about 5-10 seconds!  If I run "rundig" with the
"-v"
switch, I of course get some additional diagnostics about the dig itself,
but not
much that is worthwhile.  And there is nothing in the output that would give
me
a clue as to why it isn't doing the same job it did before.

I have started going back to reread some of the documentation, but haven't
seen
anything thus far that is too encouraging.  Within the Config file, there is
something about being able to create a file with a list of URL's that should
be
indexed, but I don't see anything about the format such a file would take.
Would
I enter one URL per line?  Would I want to reference each "top level" page
of
my site in order to pick up all the links contained therein?

If it hasn't been obvious up to now, this is something of an urgent plea for
help, as it now appears that with our newly formatted, ColdFusion based
site,
that HT-Dig is dead in the water!  I will be most appreciative of anyone who
can provide the "magic wand" to get this all working again.

Jim Augeri
Honeywell International Inc.
Defense Avionics Systems
9201 San Mateo Blvd NE
Albuquerque, NM 87113
Off: 505.828.5808
Fax: 505.828.5500
E-mail: Jim...@ho...


-----Original Message-----
From: Geoff Hutchison [mailto:ghu...@ws...]
Sent: Saturday, January 05, 2002 8:10 AM
To: Augeri, Jim (NM75)
Cc: 'htd...@li...'
Subject: Re: [htdig] Cold Fusion (.cfm) files and HT://Dig


At 10:12 AM -0700 1/4/02, Augeri, Jim (NM75) wrote:
>Will Ht://Dig still continue to index everything correctly?

Sure. If the output is HTML (or text or PDF or whatever), it doesn't 
matter how the page is assembled. Indexing works with PHP, SSI, 
ColdFusion, CGI, etc.

>Is there any way to continue to use HTML META tags to some meaningful
>purpose?
>And what about the Ht://Dig unique tags, such as the e-mail notification
>tags, etc?
>Can anyone suggest an appropriate page structure to take ample advantage of
>both
>the CF and Ht://Dig capabilities?

These seem like they have more to do with using ColdFusion, but I'm 
sure you can add HTML META tags (including ht://Dig unique ones) just 
fine with ColdFusion. However, since I've never used the product, 
I'll leave this to anyone who may have more experience.

-- 
--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/