[htdig] Some questions about the database files (db.worddump and db.docs)

[angelm1 <an...@te...>]

Hi!

My name is Daniel Almendra and I'm working on the project of a
security tool that (when implemented) will use htdig (I have installed
ht://Dig 3.2.0b4) to search many security sites for vulnerabilities,
building with such information a database that can be searched through
a web site. In addition to the site, there will be a tool to send
email to administrators alerting them about new vulnerabilities
regarding their systems.

The major difficulty I'm having is in the process of feeding the
database with efficient data (the idea is to use the spider to search
the data, and there'll be a program to parse this data and feed the
database, all done automatically).

What I'd like to know from you is:

1. Do you know a program that performs the operation of reading the
htdig dumpfiles and putting the data on a database (like MySQL,
postgre or similar)? It would make my job really easier...

2. I'm writing a program that searches the file db.worddump, to find 2
or more desired words on the same document and then it reads the
db.docs file, to include the document that contains those words in the
final database (MySQL). So, what I'd like to know is: What do the
fields in the worddump file mean (the explanation I found in the site
is for the db.wordlist file, which is different from db.worddump)?

Thanks a lot for your attention, I hope we can keep in touch! I'll
wait for an answer.

Best regards,

Daniel Almendra