htdig-dev — Developer Discussion for the ht://Dig project

[htdig-dev] New htdig maintainer

[Andy A. <and...@ya...>]

Hi

I recently mirrored the htdig repo on GitHub.

The address is https://github.com/andy5995/htdig

I plan to promote it, and work on it from time-to-time. I'm not an expert coder, so I have no expectations that I'll be able to develop htdig as well as the original developers, or that I'll be able to make any significant improvements.

For the moment, my plan simply is to gradually build up a community through GitHub and a Slack workspace. The more members who join, the better idea we'll have about the level of support for the continued development of htdig.

To create the possibility and opportunity for development to continue, I'll review pull requests in a timely manner, create tickets, and make minor patches myself.

I don't feel a need to manage or control the project for an indefinite period of time, and would happily hand it over to a team of three or four capable devs who could make sure that htdig remains a highly reputable and stable open source project.

If you are interested in passively monitoring the project, there is a link to join the Slack workspace (chat) on the revised README.md file.

I also ask that you submit any known bugs on the GitHub issue tracker.

The version I'm starting with is 3.2.0b6 (using a patch from https://packages.debian.org/stable/web/htdig), and some modifications from Robert Klein (https://github.com/roklein/htdig), who's repo is the one I mirrored.

Thank you,

-- 
-Andy

[htdig-dev] you will need a C++ library. Try installing libstdc++.

[Pawar, S. (S.) <sp...@vi...>]

I am trying to install htdig-3.1.6 on linux server.
but at the time ./configure 
it is giving error 
 you will need a C++ library. Try installing libstdc++.

ldconfig -p | grep libstdc++
        libstdc++.so.6 (libc6,x86-64) => /usr/lib64/libstdc++.so.6
        libstdc++.so.6 (libc6) => /usr/lib/libstdc++.so.6
        libstdc++.so.5 (libc6,x86-64) => /usr/lib64/libstdc++.so.5
        libstdc++-libc6.2-2.so.3 (libc6) => /usr/lib/libstdc++-libc6.2-2.so.3

I exported LD_RUN_PATH=/usr/lib
and then again did ./configure but still it gives the same error.

For libstdc++-devel package, it is showing the below message.
Also for Package libstdc++-devel-4.4.7-16.el6.x86_64 already installed and latest version

Please help me in configuring and installing /htsearch/ 

Thanks,
Sandhya.
_____
___________________________________
From: htd...@li... <htd...@li...>
Sent: Tuesday, January 19, 2016 7:43 PM
To: Pawar, Sandhya (S.)
Subject: Welcome to the "htdig-dev" mailing list

Welcome to the htd...@li... mailing list!

To post to this list, send your email to:

  htd...@li...

General information about the mailing list is at:

  https://lists.sourceforge.net/lists/listinfo/htdig-dev

If you ever want to unsubscribe or change your options (eg, switch to
or from digest mode, change your password, etc.), visit your
subscription page at:

  https://lists.sourceforge.net/lists/options/htdig-dev/spawar3%40visteon.com


You can also make such adjustments via email by sending a message to:

  htd...@li...

with the word `help' in the subject or body (don't include the
quotes), and you will get back a message with instructions.

You must know your password to change your options (including changing
the password, itself) or to unsubscribe.  It is:

  visteon

Normally, Mailman will remind you of your lists.sourceforge.net
mailing list passwords once every month, although you can disable this
if you prefer.  This reminder will also include instructions on how to
unsubscribe or change your account options.  There is also a button on
your options page that will email your current password to you.

[htdig-dev] Italian mirro deactivation

[Marco N. <mn...@pr...>]

Hi,
Italian htdig mirror is about to be deactivated.

Could you please remove it from dns?

Affected DNS names are www.it.htdig.org and ftp.it.htdig.org

Regards,
Marco

-- 
---------------------------------------------------------------------
|    Marco Nenciarini    | Debian/GNU Linux Developer - Plug Member |
| mn...@pr... | http://www.prato.linux.it/~mnencia       |
---------------------------------------------------------------------
Key fingerprint = FED9 69C7 9E67 21F5 7D95  5270 6864 730D F095 E5E4

[htdig-dev] ISO: installation and configuration help

[Tanya R. <ta...@or...>]

Hello - I'm looking for a dev to hire to install, configure and document an instance or two of htdig. Can anyone point me to a good place or have recommendations to that affect? 

Offline please - 

Thanks,
Tanya
------------------------------------------------------
Orchid: Software you can thrive on
------------------------------------------------------
Tanya Renne
CEO
Orchid Suites, Inc. 
1309 Emerson Street NW
Washington, DC 20011
(o) 202-265-1671 x104
(f) 202-265-1672
------------------------------------------------------

[htdig-dev] [ htdig-Patches-3427714 ] Collection patch

[SourceForge.net <no...@so...>]

Patches item #3427714, was opened at 2011-10-24 11:29
Message generated for change (Tracker Item Submitted) made by javaliah
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304593&aid=3427714&group_id=4593

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Jouni Väliaho (javaliah)
Assigned to: Nobody/Anonymous (nobody)
Summary: Collection patch

Initial Comment:
This patch fixed thc compilation problem I met and also reported before by others:
https://sourceforge.net/mailarchive/message.php?msg_id=8180924
https://sourceforge.net/mailarchive/message.php?msg_id=8180929
I was using g++ 4.6.1, but it may apply to 4.1 and later
see. http://www.fourmilab.ch/fourmilog/archives/2006-05/000699.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304593&aid=3427714&group_id=4593

[htdig-dev] new mirror contribution open source

[<co...@fa...>]

hello !

new mirror for you !


-----------------------------------------------------------------------------


mirror          http://htdig.facebook-solution.com/ftp.htdig.org/
country         germany
location        facebook solution enterprise
www            http://htdig.facebook-solution.com/ftp.htdig.org/
www-host      wwww.facebook-solution.com
organisation    facebook solution
contactperson   batut yann
contactemail    co...@fa...
contactkeyid	
contactphone    +33 0981756847
contactaddress  5 boulevard trudaine ,clermont ferrand  ,france

[htdig-dev] limit_url

[Jason R. <jr...@ri...>]

Can I have multiple entries for the 'limit_urls' configuration option?  If so, how do I specify multiple url's?

Thanks!

________________________________
Disclaimer: This e-mail message is intended only for the personal use of
the recipient(s) named above. If you are not an intended recipient, you
may not review, copy or distribute this message. If you have received this
communication in error, please notify us immediately by e-mail and delete
the original message.

This e-mail expresses views only of the sender, which are not to be
attributed to Rite Aid Corporation and may not be copied or distributed
without this statement.

[htdig-dev] Link Haagse Hogeschool on htdig.org

[C&M l. <lin...@hh...>]

Dear sirs/madams,

First let me introduce myself: my name is Ronald and work
for The Hague University (www.dehaagsehogeschool.nl). On
your website you have put a link to our previous domain
(www.hhs.nl) on this page: http://www.htdig.org/uses.html

Would you please be so kind to change the url of our
website from www.hhs.nl to www.dehaagsehogeschool.nl, as
this is the most accurate domain?

Thanks in advance!

Best regards
Ronald

[htdig-dev] mirrors

[svart <ant...@ya...>]

since most mirrors are out of service, I want to announce a working one:

http://mirrors.zerg.biz/htdig



      

[htdig-dev] can ht://Dig exclude headings from excerpts?

[John C. <bi...@um...>]

Hi all,

I know this may not find anyone; it looks as though dev on ht://Dig  
stopped around 2004. However, we've been using this great product  
since the late 90's and still find it very useful.

Just a question: is there a way to get ht://Dig to exclude headings  
text from excerpts in the search results? We use meta description tags  
for our high-value pages, which works wonderfully. However, for our  
other pages, the headings (which are obviously necessary on the pages)  
tend to muck up the excerpts. For an example, see the example result:

ITCS: How do I use one of the e-mail packages supported at U-M  
(mail.umich.edu, Pine, Mulberry, Eudora)?

at

<http://www.itcs.umich.edu/cgi-bin/htsearch?words=e-mail;page=2>

Were the <h2> tag content not repeated in the excerpt, there'd be a  
lot more room for the answer to be shown there.

Google search results exclude heading tag contents from results  
excerpts; I didn't see a way in the ht://Dig attributes to do the same  
thing, but I'm hoping there's some solution.

Thanks for any advice you can offer!

John Cady
University of Michigan - ITCS

[htdig-dev] ht://Dig mirror Romania Atlas Telecom

[<mi...@at...>]

Hello,

Unfortunatly after server crash we can't offer anymore the mirroring for
your site. Can you please remove our server from the mirror list.

Thank you,
Support Team

[htdig-dev] New htdig mirror

[Evgeny P. <evg...@gm...>]

Hello
I've created new files mirror of htdig on my server: http://htdig.emirrors.org/
The server placed in Erfurt, Germany.
It will be very cool if you place link on my mirror from your website.
Thank you for your work.

Best regards,
Evgeny

[htdig-dev] UTF8 workaround

[Cédric C. <cch...@co...>]

I've submitted a workaround for those, like me, who'd like to use 
ht:/Dig on an UTF8 website. It uses iconv and sed to convert pages 
on-fly back from UTF8 to ISO8859-1, with an internal handler.

Ticket with workaround : 
http://sourceforge.net/tracker/index.php?func=detail&aid=2009114&group_id=4593&atid=354593
In your htdig.conf :
...
external_parsers: /path/to/utf8.sh (extlatin1.sed should be in the same 
directory)
...

-- 
Cédric Chantepie - mailto:cha...@no...
Architecte des systèmes d'information
NOZICAA (Groupe SIGIRE) - http://www.nozicaa.com
20 rue de Sardaigne - Z.A. du Danemark - 72100 LE MANS
Tel: +33 (0) 243 82 97 97
Fax: +33 (0) 243 82 97 99

[htdig-dev] faceted searching

[<roy...@im...>]

We have identified a target market for developing an expert knowledge-base with 
access via a fuzzy matching with a profile defined in terms of a set of values of some 
or all of N parameters, each parameter representing an aspect or facet of the features 
of a chunk of knowledge or experience.

We note that htdig admits to adding of keywords to documents at the meta-level. This 
would be necessary for a parametric indexing system to work. Has anyone adapted 
htdig by adding parameters and their value-sets (symbolic keywords or numerical), 
such as to be set up by expert users, in such a way as to characterise meaningfully 
their knowledge-domain, in terms of expert-specifird parameters and value-sets?

I am copying this to myself at home, as I am bi-locating.

RoyJ

Roy H W Johnston PhD FInstP CIEI
Applied-scientific consultant, Techne Associates,
e-mail rjt...@io..., phone +353-1-490-4040; website
http://www.iol.ie/~rjtechne/.
When 'on site' c/o IMS, 20 Years in Healthcare IT
Clara House, Glenageary Park, Co Dublin, 
roy...@im..., phone +353-1-284-0555;
website http://www.imsmaxims.com; also 
http://www.coastlinesolutions.com. phone +353-1-235-2166

[htdig-dev] How to Mirror the ht://Dig Project

[<apa...@gm...>]

Are there any plans on using rsync instead of the CSV for mirroring?

Thanks,

Ionel Roiban
usphp.com

[htdig-dev] [ htdig-Patches-1870947 ] build error on Mac OS X

[SourceForge.net <no...@so...>]

Patches item #1870947, was opened at 2008-01-14 18:44
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304593&aid=1870947&group_id=4593

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: devel
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Yuji Yamano (yyamano)
Assigned to: Nobody/Anonymous (nobody)
Summary: build error on Mac OS X

Initial Comment:
When I build htdig 3.2.0b6 on Mac OS X, I gor the following error message. 
The attached patch will fix this problem.


htsearch.cc:71: error: non-local variable '<anonymous struct> colonPrefix []' uses anonymous type

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304593&aid=1870947&group_id=4593

[htdig-dev] XSS error in sytnax.html

[Michael S. <ms...@su...>]

Hi,

there is a XSS error in syntax.html of htdig.

you can reproduce this like this:
http://foo.bar/cgi-bin/htsearch?config=&restrict=&exclude=&method=and&format=builtin-long&sort=<script>alert("foo")</script>&words=foo

$(SYNTAXERROR) must be quoted by htdig before filling it in.

greetings
Michael
-- 
Michael Skibbe <ms...@su...>
Core Services
SUSE Linux Products GmbH                      GF: Markus Rex
Nuernberg, Germany                            HRB 16746 (AG Nuernberg)

[htdig-dev] grub now open source search

[G. T. Stresen-R. <ted...@ma...>]

Not sure if this is of interest to anyone on the list, but thought  
I'd pass it on in case you hadn't already heard...

http://gigaom.com/2007/07/30/google-vs-jimmy-wales-and-open-source- 
search/

Ted Stresen-Reuter

(still eager to participate on this project, or wherever it morphs...)

[htdig-dev] Segfault with htstat

[B. N. P. <poo...@in...>]

Hello,
We were getting seg fault while running the htstat command in the latest 
version of htdig i.e., htdig-3.2.0b6-21.2.:
**************************************

ls -la /var/lib/htdig/db/
total 92
drwxr-xr-x 2 root root  4096 Jun 19 03:53 .
drwxr-xr-x 3 root root  4096 Jun 19 03:53 ..
-rw-r--r-- 1 root root 24576 Jun 20 14:55 db.docdb
-rw-r--r-- 1 root root    91 Jun 20 14:55 db.docs
-rw-r--r-- 1 root root 24576 Jun 20 14:55 db.docs.index
-rw-r--r-- 1 root root 24576 Jun 13 17:03 db.excerpts
-rw-r--r-- 1 root root    40 Jun 20 14:55 db.worddump
-rw-r--r-- 1 root root  2048 Jun 13 17:03 db.words.db
-rw-r--r-- 1 root root 16384 Jun 13 17:03 db.words.db_weakcmpr

# htdig && htdig -t && htstat && htload && htstat
htstat: Total documents: 1
htstat: Total words: 0
Segmentation fault
***************************************************************************
On investigation, found that the word DB is empty and we are getting a seg fault as we are trying 
to get the total unique words count even though the total word count is '0'. Hence we made the following 
changes to htstat.cc (checking if words.WordRefs() count is zero or not) and it works fine now.
****************************************************************************
HtWordList words(*config);
    if(words.Open(config->Find("word_db"), O_RDONLY) == OK)
      {
        cout << "htstat: Total words: " << words.WordRefs()->Count() << endl;
     +   if (words.WordRefs()->Count() != 0)
        cout << "htstat: Total unique words: " << words.Words()->Count() << endl;
        words.Close();
      }
********************************************************************************
Can you please let us know if the proposed patch is OK or not?

Regards,
Poornima

[htdig-dev] htdig mirror

[Pelle B. <p....@gm...>]

Hello there!

Iam a big friend of open source and thus I want to dedicate some of my
bandwith to it. For that reason i set up a htdig mirror. The mirror is
located at

http://openmirror.org/pub/htdig/maindocs/
http://openmirror.org/pub/htdig/maindocs/files/

and is updated daily via a cronjob. The server has a redundant 100mbit
connection and is located in Berlin/Germany.

With kind regards, Pelle Boese.

-- 
Quidquid latinum dictum sit, altum viditur.

Pelle Boese
Clemensstr. 6
50676 Cologne
Germany

[htdig-dev] Entities problems

[Dominique A. <dom...@es...>]

Hi,

  We had a problem with some missing entities in the description result
(for instance: –  ’ and ™).  We did some modifications
to filter those entities in the code:

  htcommon/HtSGMLCodec.cc
  myTextFromString << "&ndash;|&rsquo;|&trade;";

  This solved the problem for the description field $(EXCERPT), but we
still have problems with those entities in the title list of the result,
$(TITLE).

  What can we do to resolve this issue?

  Thanks.


-- 
Dominique Arpin, administrateur réseau
A+,Linux+,Server+,MCP
Espace Courbe inc.  http://www.espacecourbe.com/
642 de Courcelle, bureau 303, Montréal (Québec), Canada H4C 3C5
tél.: (514) 933-9861 téléc.: (514) 933-9546

[htdig-dev] Mirroring Htdig

[Deep76 <dp7...@gm...>]

Hello, I'm interesting in mirroring your  SGI IRIX Freeware
We have dedicated server in Ukraine(UA), Kiev.
But cannot login to cvs
======
Step 2
You'll need to gain anonymous access at the CVS repository.
cvs -d:pserver:ano...@cv...:/cvsroot/htdig login
======
got error -
cvs [login aborted]: connect to
cvs.htdig.sourceforge.net(66.35.250.209):2401 failed: No route to host

[htdig-dev] The bug in URL.cc (htcommon folder)

[Denis S. <sid...@gm...>]

I have found the bug in URL.cc file.

Method: URL::URL(const String &url, const URL &parent)
if parent ULR is like: "http://telur.ru/catalog.php?cat_id=2204"
and url like: "?cat_id=2204&page=4" then
IE and Firefox resolve URL to: "
http://telur.ru/catalog.php?cat_id=2204&page=4", but htdig resolves it to: "
http://telur.ru/?cat_id=2204&page=4"

It is fixed very simple:
instead of:

...
if (_path.last() == '/')
{
        //
        // Parent was a directory.  Easy enough: just append
        // the current ref to it
        //
        _path << ref;
}
...

should be:

...
if (_path.last() == '/' || *rel=='?')
{
        //
        // Parent was a directory.  Easy enough: just append
        // the current ref to it
        //
        _path << ref;
}
...

Hope it helps. I've tested this version. All works fine for me.
Please, let me know if I'm wrong.

Best regards,
Denis Sidorenko,
Russia(Moscow)

[htdig-dev] htmerge memory problem

[Denis S. <sid...@gm...>]

Dear htdig team,

I have installed the latest version of htdig my web server. Then, I run
rundig and create two htdig databases. I tried to merge them with htmerge
utility, it takes ~5 hours :(.

My web server has only 1Gb RAM but htmerge uses 1GB RAM + 2GB SWAP :(
Is it possible to reduce memory usage by htmerge?

I have found the following patch:
http://www.htdig.org/htdig-dev/2000/06/att-0024/01-htmerge.patch
Maybee I should use this one?
http://wwwbs.informatik.htw-dresden.de/internet/htdig/htdig-3.1.6/htmerge/db.cc

Any comments or suggestions are welocome. Thank you!

Best regards,
Denis Sidorenko

[htdig-dev] New Mirror of ht://Dig

[Daniel S. <ko...@br...>]

Hello,

thanks alot for your great work with ht://Dig!!

We like to contribute, so we set up a mirror of the ht://Dig website here:
http://htdig.brainbyte.de/

and the Developer site here:
http://htdig.brainbyte.de/dev/

the Files here:
http://htdig.brainbyte.de/files/  (HTML)

and the Patches here:
http://htdig.brainbyte.de/htdig-patches/ (HTML)

The organisation is Brainbyte:
http://www.brainbyte.de/

The country is Germany!

I would be glad if you add us to the mirror list (like this:

	<tr>
	<td><a href="http://www.brainbyte.de/" target="_top">Brainbyte</a></td>
	<td>Germany</td>
	<td><a href="http://htdig.brainbyte.de/" 
target="_top">htdig.brainbyte.de</a></td>
	<td><a href="http://htdig.brainbyte.de/dev/" 
target="_top">htdig.brainbyte.de</a></td>
	<td><a href="http://htdig.brainbyte.de/files/">(HTTP)</a>
	(FTP)</td>
	<td><a href="http://htdig.brainbyte.de/htdig-patches/">(HTTP)</a>
	(FTP)</td>
	</tr>

)

Many thanks, kind regards

Daniel Schwerd
---
ko...@br...
http://brainbyte.de
Commerce - Content - Consulting