RE: [htdig] merging digs

[Edward C. <ec...@st...>]

> On Tue, 16 Mar 2004, Edward Chase wrote:
>=20
> > I've got 2 individual digs that I'd like to merge into one for=20
> > searches.  It is my plan to dig a piece of my domain at a time each=20
> > night and then merge the results into one search database.
> >=20
> > Below is the script that I'm using to do the merge.  Is=20
> this correct=20
> > with the file copies and moves?  I suspect not because there are=20
> > things I can find searching using server1.conf that I can't=20
> find using=20
> > allpubsvrs.conf
>=20

> From: Jim [mailto:li...@yg...]=20
>=20
> > Are you first running htmerge on each individual database as=20
> it is created? You need to do that before trying to merge one=20
> database with another. There is a note to this effect, along=20
> with a brief explanation of why it is necessary, on the page=20
> that describes the htmerge command.
>=20
> Jim
>=20


I did get this working.  Here is what I did.

I took the results of the rundig from server1 and copied it to =
allpubsvrs.
Then I took the results of the rundig from server2 and merged them into
allpubsvrs. =20

I suppose this is what I had done to originally setup allpubsvrs, but =
from
this point I would keep merging the results from server1 and then =
server2
over and over (after new rundigs).  With this method, allpubsvrs kept
growing and growing and growing.  I don't think old links were getting
weeded out this way.

With my current method, allpubsvrs is getting reinitialized as a dupe of =
the
first server results before any merges are done.

So to answer your question...  Yes, I was going through an htmerge on =
the
individual databases.

Thanks for you thought, I do believe you were the only one to respond to =
my
question.