[htdig] Long URL List causes blow up

[Searcher <sea...@lo...>]

Here's the problem.

I upgraded to 3.20. I set everything up and started a dig. I'm=
 using 
a list of 10,020 URL's, each on it's own line, terminated with a=
 / 
and not .html or anything else.

I fire up the dig and it terminates repeatedly. After repeatedly=
 
trying, I move back to 3.16 b4 or so. Same problem. I finally=
 find 
that if I break my list down into 5000 URL's at a time, it works.=
 I 
do my first dig using -i. I start my second dig using -a so that=
 it 
will add on to the first. Nothing crashes, it's still going.

This means I'll move back to the latest 3.20 x once this is done=
 and 
use multiple lists until I find the problem. The question is,=
 what is 
the problem? Why is this happening? Everything else seems to work=
 
just fine.

Mike