Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

OpenWebSpider / News: Recent posts

OpenWebSpider# v0.1.4 Released

OpenWebSpider now supports MP3s and PDFs

CHANGELOG:
OpenWebSpiderCS v0.1.4
- Mysql/NET Connector upgraded to 5.2.5.0
- Enhanced encodings support
- New feature: Support to META: "robots" (NOINDEX, NOFOLLOW)
- New feature: New configuration file field: crawler_id
- New field "crawler_id" in table "hostlist"
- New table: crawler_act
- New feature: Remote actions over running crawlers [Status, Play, Pause, Kill]
- New file support: PDFs [Using PDFBox and IKVM]
- New table: pdf
- New file support: MP3s [Using UltraID3Lib]
- New table: mp3
- New feature: new command-line argument: --pdf
- New feature: new command-line argument: --mp3... read more

Posted by Stefano Alimonti 2009-05-07

OpenWebSpider# v0.1 Released

OpenWebSpider has been stopped for a long time… but I’ve decided that’s not the time to make it dead and so I worked hard in the last 3-4 weeks and I released a new version of the crawler written from scratch entirely in C#.
For more info: http://www.openwebspider.org/

Posted by Stefano Alimonti 2008-07-29

Released OpenWebSpider v0.6.1 Source Code + Win32 Binary

Files released:
2006-05-07 11:38 15726 openwebspider-0.6.1\macro\ows_macro.c
2005-08-19 20:16 533 openwebspider-0.6.1\macro\macro.dsw
2005-08-19 21:06 4353 openwebspider-0.6.1\macro\macro.dsp
2006-04-19 04:41 34900 openwebspider-0.6.1\macro\libmysql.lib
2006-07-02 11:03 3912 openwebspider-0.6.1\modules\mod_pdf\mod_pdf.c
2006-06-30 10:55 4041 openwebspider-0.6.1\modules\mod_pdf\mod_pdf.vcproj
2006-06-30 11:04 4253 openwebspider-0.6.1\modules\regexFilter\regexFilter.vcproj
2005-11-15 17:26 541 openwebspider-0.6.1\modules\regexFilter\regexFilter.dsw
2005-12-27 13:50 4492 openwebspider-0.6.1\modules\regexFilter\regexFilter.dsp
2006-06-30 11:00 5399 openwebspider-0.6.1\modules\regexFilter\regexFilter.c
2006-03-20 20:27 2018 openwebspider-0.6.1\modules\sampleModReplace.c
2005-12-27 13:34 1670 openwebspider-0.6.1\modules\sampleModFilter.c
2006-03-20 20:09 1648 openwebspider-0.6.1\modules\modHeader.h
2005-12-28 21:22 1265 openwebspider-0.6.1\mysql\typelib.h
2005-12-28 21:22 5841 openwebspider-0.6.1\mysql\raid.h
2005-12-28 21:22 8076 openwebspider-0.6.1\mysql\m_string.h
2005-12-28 21:22 19095 openwebspider-0.6.1\mysql\m_ctype.h
2005-12-28 21:22 35589 openwebspider-0.6.1\mysql\my_sys.h
2005-12-28 21:22 25154 openwebspider-0.6.1\mysql\my_pthread.h
2005-12-28 21:22 1515 openwebspider-0.6.1\mysql\my_list.h
2005-12-28 21:22 41825 openwebspider-0.6.1\mysql\my_global.h
2005-12-28 21:22 3000 openwebspider-0.6.1\mysql\my_getopt.h
2005-12-28 21:22 3678 openwebspider-0.6.1\mysql\my_dbug.h
2005-12-28 21:22 1859 openwebspider-0.6.1\mysql\my_alloc.h
2005-12-28 21:22 790 openwebspider-0.6.1\mysql\mysql_version.h
2005-12-28 21:22 2184 openwebspider-0.6.1\mysql\mysql_time.h
2005-12-28 21:22 1224 openwebspider-0.6.1\mysql\mysql_embed.h
2005-12-28 21:22 16766 openwebspider-0.6.1\mysql\mysql_com.h
2005-12-28 21:22 16114 openwebspider-0.6.1\mysql\mysqld_error.h
2005-12-28 21:22 16059 openwebspider-0.6.1\mysql\mysqld_ername.h
2005-12-28 21:22 32725 openwebspider-0.6.1\mysql\mysql.h
2005-12-28 21:22 2862 openwebspider-0.6.1\mysql\libmysql.def
2005-12-28 21:22 3876 openwebspider-0.6.1\mysql\errmsg.h
2005-12-28 21:22 12923 openwebspider-0.6.1\mysql\config-win.h
2005-12-28 21:22 22387 openwebspider-0.6.1\mysql\config-os2.h
2005-12-28 21:22 3607 openwebspider-0.6.1\mysql\config-netware.h
2006-05-07 14:11 572 openwebspider-0.6.1\PHP\vars.php
2006-05-07 12:49 4808 openwebspider-0.6.1\PHP\searchfunct.php
2005-11-30 19:53 887 openwebspider-0.6.1\PHP\search.php
2005-11-30 19:56 1325 openwebspider-0.6.1\PHP\cache.php
2006-06-30 11:04 6656 openwebspider-0.6.1\Win32_bin\modules\regexFilter.dll
2006-02-08 13:42 536576 openwebspider-0.6.1\Win32_bin\modules\pdftotext.exe
2006-06-30 10:55 7680 openwebspider-0.6.1\Win32_bin\modules\mod_pdf.dll
2006-05-07 14:03 45056 openwebspider-0.6.1\Win32_bin\ows_macro.v0.2.win32.exe
2006-06-30 10:46 94208 openwebspider-0.6.1\Win32_bin\openwebspider.v0.6.1.win32.exe
2006-05-07 14:07 658 openwebspider-0.6.1\Win32_bin\openwebspider.conf
2006-04-19 04:41 34900 openwebspider-0.6.1\Win32_bin\libmysql.lib
2006-04-19 04:41 1449984 openwebspider-0.6.1\Win32_bin\libmysql.dll
2006-07-02 11:18 967 openwebspider-0.6.1\Makefile
2006-07-02 11:16 668 openwebspider-0.6.1\openwebspider.conf
2006-07-02 10:50 237 openwebspider-0.6.1\mod_pdf.conf
2005-10-30 11:48 303 openwebspider-0.6.1\xmalloc.c
2006-06-30 10:36 21924 openwebspider-0.6.1\urlfunct.h
2006-06-30 10:36 11375 openwebspider-0.6.1\thread.h
2006-04-30 17:40 3750 openwebspider-0.6.1\temptable.h
2006-06-30 10:36 8256 openwebspider-0.6.1\strfnct.h
2006-04-30 16:47 6244 openwebspider-0.6.1\sqlfnct.h
2006-06-30 10:36 3839 openwebspider-0.6.1\socket.h
2006-03-07 20:36 18975 openwebspider-0.6.1\snprintf.c
2006-06-30 10:36 18847 openwebspider-0.6.1\server.h
2006-06-30 10:36 4373 openwebspider-0.6.1\search.h
2006-06-30 10:36 3427 openwebspider-0.6.1\robots.h
2005-10-30 11:48 18702 openwebspider-0.6.1\regex.h
2006-02-10 19:33 161839 openwebspider-0.6.1\regex.c
2006-02-18 12:52 2303 openwebspider-0.6.1\rank.h
2006-06-30 10:36 5079 openwebspider-0.6.1\parse_conf.h
2006-06-30 10:36 11204 openwebspider-0.6.1\options.h
2006-05-30 09:44 8108 openwebspider-0.6.1\openwebspider.vcproj
2005-04-09 15:55 541 openwebspider-0.6.1\openwebspider.dsw
2006-04-30 16:48 6372 openwebspider-0.6.1\openwebspider.dsp
2006-06-30 10:44 12941 openwebspider-0.6.1\openwebspider-0.6.c
2006-02-11 14:02 2979 openwebspider-0.6.1\mymutex.h
2006-01-11 20:03 317 openwebspider-0.6.1\mod_regex.conf
2006-06-30 10:36 3940 openwebspider-0.6.1\modules.h
2006-06-30 10:36 12079 openwebspider-0.6.1\misc.h
2006-04-30 16:50 3475 openwebspider-0.6.1\macro.h
2006-06-30 10:36 4690 openwebspider-0.6.1\list.h
2005-01-13 09:46 33890 openwebspider-0.6.1\libmysql.lib
2006-06-30 10:36 14400 openwebspider-0.6.1\indexer.h
2006-06-30 10:36 10742 openwebspider-0.6.1\htmlfnct.h
2005-12-18 16:05 3465 openwebspider-0.6.1\hstlist.h
2005-10-30 11:48 6663 openwebspider-0.6.1\getopt.h
2005-10-30 11:48 40239 openwebspider-0.6.1\getopt.c
2006-06-30 10:36 7508 openwebspider-0.6.1\functions.h
2006-06-30 11:08 3628 openwebspider-0.6.1\Changelog.txt
2005-10-30 11:48 5327 openwebspider-0.6.1\alloca.c
#
# Total Size Files
# 3028826 86

Posted by Stefano Alimonti 2006-07-02

Released OpenWebSpider v0.6 Source Code + Win32 Binary

File released:
openwebspider-0.6\alloca.c 5327
openwebspider-0.6\functions.h 7192
openwebspider-0.6\getopt.c 40239
openwebspider-0.6\getopt.h 6663
openwebspider-0.6\hstlist.h 3465
openwebspider-0.6\htmlfnct.h 10126
openwebspider-0.6\indexer.h 13394
openwebspider-0.6\list.h 4247
openwebspider-0.6\macro\libmysql.lib 34900
openwebspider-0.6\macro\macro.dsp 4353
openwebspider-0.6\macro\macro.dsw 533
openwebspider-0.6\macro\ows_macro.c 15726
openwebspider-0.6\macro.h 3475
openwebspider-0.6\Makefile 765
openwebspider-0.6\misc.h 11566
openwebspider-0.6\modules\modHeader.h 1648
openwebspider-0.6\modules\regexFilter.c 5359
openwebspider-0.6\modules\regexFilter.dsp 4492
openwebspider-0.6\modules\regexFilter.dsw 541
openwebspider-0.6\modules\sampleModFilter.c 1670
openwebspider-0.6\modules\sampleModReplace.c 2018
openwebspider-0.6\modules.h 4120
openwebspider-0.6\mod_regex.conf 317
openwebspider-0.6\mymutex.h 2979
openwebspider-0.6\openwebspider-0.6.c 12480
openwebspider-0.6\openwebspider.conf 658
openwebspider-0.6\openwebspider.dsp 6372
openwebspider-0.6\openwebspider.dsw 541
openwebspider-0.6\options.h 10834
openwebspider-0.6\parse_conf.h 4827
openwebspider-0.6\PHP\cache.php 1325
openwebspider-0.6\PHP\search.php 887
openwebspider-0.6\PHP\searchfunct.php 4808
openwebspider-0.6\PHP\vars.php 572
openwebspider-0.6\rank.h 2303
openwebspider-0.6\regex.c 161839
openwebspider-0.6\regex.h 18702
openwebspider-0.6\robots.h 3256
openwebspider-0.6\search.h 4288
openwebspider-0.6\server.h 17994
openwebspider-0.6\snprintf.c 18975
openwebspider-0.6\socket.h 3575
openwebspider-0.6\sqlfnct.h 6244
openwebspider-0.6\strfnct.h 7929
openwebspider-0.6\temptable.h 3750
openwebspider-0.6\thread.h 11151
openwebspider-0.6\urlfunct.h 20726
openwebspider-0.6\Win32_bin\libmysql.dll 1449984
openwebspider-0.6\Win32_bin\libmysql.lib 34900
openwebspider-0.6\Win32_bin\openwebspider.conf 658
openwebspider-0.6\Win32_bin\openwebspider.v0.6.win32.exe 86016
openwebspider-0.6\Win32_bin\ows_macro.v0.2.win32.exe 45056
openwebspider-0.6\Win32_bin\regexFilter.dll 65536
openwebspider-0.6\xmalloc.c 303
openwebspider-0.6\Changelog.txt 3557 ... read more

Posted by Stefano Alimonti 2006-05-07

Released OpenWebSpider v0.5.2 Source Code + Win32 Binary

File released:
openwebspider-0.5.2\alloca.c 5327
openwebspider-0.5.2\doc\cache.php.txt 1325
openwebspider-0.5.2\doc\Changelog.txt 3145
openwebspider-0.5.2\doc Folder
openwebspider-0.5.2\functions.h 5937
openwebspider-0.5.2\getopt.c 40239
openwebspider-0.5.2\getopt.h 6663
openwebspider-0.5.2\gui.h 5608
openwebspider-0.5.2\hstlist.h 3465
openwebspider-0.5.2\htmlfnct.h 13322
openwebspider-0.5.2\libmysql.lib 33890
openwebspider-0.5.2\list.h 1914
openwebspider-0.5.2\macro Folder
openwebspider-0.5.2\macro\macro.dsp 4353
openwebspider-0.5.2\macro\macro.dsw 533
openwebspider-0.5.2\Win32_bin\macro.exe 45056
openwebspider-0.5.2\Makefile 765
openwebspider-0.5.2\misc.h 22691
openwebspider-0.5.2\mod_regex.conf 317
openwebspider-0.5.2\modules\modHeader.h 1612
openwebspider-0.5.2\modules Folder
openwebspider-0.5.2\modules.h 4120
openwebspider-0.5.2\mymutex.h 2979
openwebspider-0.5.2\doc\openwebspider-0.5-handbook.htm 52012
openwebspider-0.5.2 Folder
openwebspider-0.5.2\openwebspider-0.5.c 10513
openwebspider-0.5.2\openwebspider.conf 512
openwebspider-0.5.2\openwebspider.dsp 6002
openwebspider-0.5.2\openwebspider.dsw 541
openwebspider-0.5.2\Win32_bin\openwebspider.exe 65536
openwebspider-0.5.2\options.h 10059
openwebspider-0.5.2\macro\ows_macro.c 15260
openwebspider-0.5.2\rank.h 2302
openwebspider-0.5.2\regex.c 161839
openwebspider-0.5.2\regex.h 18702
openwebspider-0.5.2\modules\regexFilter.c 5133
openwebspider-0.5.2\modules\regexFilter.def 103
openwebspider-0.5.2\Win32_bin\regexFilter.dll 65536
openwebspider-0.5.2\modules\regexFilter.dsp 4492
openwebspider-0.5.2\modules\regexFilter.dsw 541
openwebspider-0.5.2\robots.h 3406
openwebspider-0.5.2\modules\sampleModFilter.c 1670
openwebspider-0.5.2\doc\search.php.txt 887
openwebspider-0.5.2\doc\searchfunct.php.txt 4811
openwebspider-0.5.2\snprintf.c 18954
openwebspider-0.5.2\socket.h 2915
openwebspider-0.5.2\sqlfnct.h 2191
openwebspider-0.5.2\strfnct.h 7608
openwebspider-0.5.2\temptable.h 3746
openwebspider-0.5.2\thread.h 9884
openwebspider-0.5.2\urlfunct.h 14663
openwebspider-0.5.2\doc\vars.php.txt 573
openwebspider-0.5.2\Win32_bin Folder
openwebspider-0.5.2\xmalloc.c 303

Posted by Stefano Alimonti 2006-02-11

Released OpenWebSpider v0.5.1 Source Code + Win32 Binary

Files released:

openwebspider-0.5.1\alloca.c
openwebspider-0.5.1\doc\cache.php.txt
openwebspider-0.5.1\doc\Changelog.txt
openwebspider-0.5.1\doc\openwebspider-0.5-handbook.htm
openwebspider-0.5.1\doc\search.php.txt
openwebspider-0.5.1\doc\searchfunct.php.txt
openwebspider-0.5.1\doc\vars.php.txt
openwebspider-0.5.1\functions.h
openwebspider-0.5.1\getopt.c
openwebspider-0.5.1\getopt.h
openwebspider-0.5.1\gui.h
openwebspider-0.5.1\hstlist.h
openwebspider-0.5.1\htmlfnct.h
openwebspider-0.5.1\libmysql.lib
openwebspider-0.5.1\list.h
openwebspider-0.5.1\macro\macro.dsp
openwebspider-0.5.1\macro\macro.dsw
openwebspider-0.5.1\macro\ows_macro.c
openwebspider-0.5.1\Makefile
openwebspider-0.5.1\misc.h
openwebspider-0.5.1\modules\modHeader.h
openwebspider-0.5.1\modules\regexFilter.c
openwebspider-0.5.1\modules\regexFilter.def
openwebspider-0.5.1\modules\regexFilter.dsp
openwebspider-0.5.1\modules\regexFilter.dsw
openwebspider-0.5.1\modules\sampleModFilter.c
openwebspider-0.5.1\modules.h
openwebspider-0.5.1\mod_regex.conf
openwebspider-0.5.1\mymutex.h
openwebspider-0.5.1\openwebspider-0.5.c
openwebspider-0.5.1\openwebspider.conf
openwebspider-0.5.1\openwebspider.dsp
openwebspider-0.5.1\openwebspider.dsw
openwebspider-0.5.1\options.h
openwebspider-0.5.1\rank.h
openwebspider-0.5.1\regex.c
openwebspider-0.5.1\regex.h
openwebspider-0.5.1\robots.h
openwebspider-0.5.1\snprintf.c
openwebspider-0.5.1\socket.h
openwebspider-0.5.1\sqlfnct.h
openwebspider-0.5.1\strfnct.h
openwebspider-0.5.1\temptable.h
openwebspider-0.5.1\thread.h
openwebspider-0.5.1\urlfunct.h
openwebspider-0.5.1\Win32_bin\
openwebspider-0.5.1\Win32_bin\macro.exe
openwebspider-0.5.1\Win32_bin\openwebspider.exe
openwebspider-0.5.1\Win32_bin\regexFilter.dll
openwebspider-0.5.1\xmalloc.c

Posted by Stefano Alimonti 2006-01-11

Released OpenWebSpider v0.5 Source Code + Win32 Binary

Files released:

openwebspider-0.5\alloca.c
openwebspider-0.5\doc\cache.php.txt
openwebspider-0.5\doc\Changelog.txt
openwebspider-0.5\doc\openwebspider-0.5-handbook.htm
openwebspider-0.5\doc\search.php.txt
openwebspider-0.5\doc\searchfunct.php.txt
openwebspider-0.5\doc\vars.php.txt
openwebspider-0.5\functions.h
openwebspider-0.5\getopt.c
openwebspider-0.5\getopt.h
openwebspider-0.5\gui.h
openwebspider-0.5\hstlist.h
openwebspider-0.5\htmlfnct.h
openwebspider-0.5\libmysql.lib
openwebspider-0.5\list.h
openwebspider-0.5\macro\libmysql.lib
openwebspider-0.5\macro\macro.dsp
openwebspider-0.5\macro\macro.dsw
openwebspider-0.5\macro\ows_macro.c
openwebspider-0.5\Makefile
openwebspider-0.5\misc.h
openwebspider-0.5\modules\modHeader.h
openwebspider-0.5\modules\modPdf.c
openwebspider-0.5\modules\regexFilter.c
openwebspider-0.5\modules\regexFilter.def
openwebspider-0.5\modules\regexFilter.dsp
openwebspider-0.5\modules\regexFilter.dsw
openwebspider-0.5\modules\sampleModFilter.c
openwebspider-0.5\modules.h
openwebspider-0.5\mod_regex.conf
openwebspider-0.5\mymutex.h
openwebspider-0.5\openwebspider-0.5.c
openwebspider-0.5\openwebspider.conf
openwebspider-0.5\openwebspider.dsp
openwebspider-0.5\openwebspider.dsw
openwebspider-0.5\options.h
openwebspider-0.5\rank.h
openwebspider-0.5\regex.c
openwebspider-0.5\regex.h
openwebspider-0.5\robots.h
openwebspider-0.5\se_PHP\cache_example.php
openwebspider-0.5\se_PHP\search.php
openwebspider-0.5\se_PHP\searchfunct.php
openwebspider-0.5\se_PHP\vars.php
openwebspider-0.5\snprintf.c
openwebspider-0.5\socket.h
openwebspider-0.5\sqlfnct.h
openwebspider-0.5\strfnct.h
openwebspider-0.5\temptable.h
openwebspider-0.5\thread.h
openwebspider-0.5\urlfunct.h
openwebspider-0.5\Win32_bin\openwebspider-0.5.exe
openwebspider-0.5\Win32_bin\ows_macro.exe
openwebspider-0.5\Win32_bin\regexFilter.dll
openwebspider-0.5\xmalloc.c... read more

Posted by Stefano Alimonti 2005-11-30

Released OpenWebSpider v0.4 Source Code

Released OpenWebSpider v0.4 Source Code

OpenWebSpider 0.4 is the first public release of the GUI of OpenWebSpider! Screenshots here: http://www.openwebspider.org/screenshots.php

Files:
109744 GUI\libmysql.a
2749 GUI\Makefile.win
498 GUI\openwebspider.conf
3064 GUI\OpenWebSpider_GUI.dev
766 GUI\OpenWebSpider_GUI.ico
804 GUI\OpenWebSpider_GUI.layout
1276 GUI\OpenWebSpider_GUIApp.cpp
1254 GUI\OpenWebSpider_GUIApp.h
939 GUI\OpenWebSpider_GUIApp.rc
6570 GUI\OpenWebSpider_GUIFrm.cpp
4579 GUI\OpenWebSpider_GUIFrm.h
23644 GUI\OpenWebSpider_GUIFrm.wxform
1506 GUI\OpenWebSpider_GUIFrm_XPM.xpm
646 GUI\OpenWebSpider_GUI_private.h
884 GUI\OpenWebSpider_GUI_private.rc
22971 GUI\WxStaticBitmap1_XPM.xpm
4920 functions.h
6663 getopt.h
5553 gui.h
2975 hstlist.h
11704 htmlfnct.h
1914 list.h
19365 misc.h
2688 mymutex.h
6178 options.h
2284 rank.h
2944 robots.h
2302 socket.h
2102 sqlfnct.h
3354 strfnct.h
3997 temptable.h
8428 thread.h
11428 urlfunct.h
40239 getopt.c
7656 openwebspider-0.4.c
18930 snprintf.c
498 openwebspider.conf
33890 libmysql.lib
6102 openwebspider.dsp
539 Makefile

Posted by Stefano Alimonti 2005-06-20

Released OpenWebSpider v0.4 Binary

Released OpenWebSpider v0.4 Binary

File releases:
wxOpenWebSpider_v0.4_gui.exe
openwebspider_v0.4_win32_nogui.exe
openwebspider.conf
libmysql.dll

wxOpenWebSpider is the first public release of the GUI of OpenWebSpider! Screenshots here: http://www.openwebspider.org/screenshots.php

Posted by Stefano Alimonti 2005-06-10

Released OpenWebSpider v0.3

Released OpenWebSpider v0.3

File releases:
- Source tar.gz (Windows/Linux)
- Source .zip (Windows/Linux)
- Win32 Binary .exe

http://www.openwebspider.org
Shen139

Posted by Stefano Alimonti 2005-04-12

http://www.openwebspider.org/

Project's web site: http://www.openwebspider.org/

Posted by Stefano Alimonti 2005-03-25

New Release

OpenWebSpider v0.2 Released

Posted by Stefano Alimonti 2005-03-22