scloader-cvs Mailing List for Smart Cache Loader
Very configurable web downloader
Brought to you by:
hsn
You can subscribe to this list here.
2003 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(2) |
Oct
|
Nov
|
Dec
|
---|
From: <hs...@us...> - 2003-09-23 20:27:10
|
Update of /cvsroot/scloader/loader/doc In directory sc8-pr-cvs1:/tmp/cvs-serv2673/doc Modified Files: README loader.sgml Log Message: Index: README =================================================================== RCS file: /cvsroot/scloader/loader/doc/README,v retrieving revision 1.1.1.1 retrieving revision 1.2 diff -u -d -r1.1.1.1 -r1.2 --- README 6 Jan 2003 10:46:38 -0000 1.1.1.1 +++ README 23 Sep 2003 20:27:05 -0000 1.2 @@ -1,23 +1,29 @@ Smart Cache Loader Very configurable web grabber with special Smart Cache support - Radim Kolar (hs...@cy...) + Radim Kolar (hsn/at/cybermail.net) - http://home.worldonline.cz/~cz210552/ + http://scloader.sourceforge.net/ GPL license 1. This program may be hard to configure and operate. I am interrested in any feedback. -2. Configuration of this program is quite tricky. Look inside loader.cnf or at -html documentation and try to understand how to configure +2. Configuration of this program is quite tricky. -3. Run it via: java loader [sitename ...] [options ...] [URL ...] +3. Look inside loader.cnf and html documentation and try to understand +how to configure and use + +4. Basic usage: java loader [sitename ...] [options ...] [URL ...] main options: depth=X <scan depth> threads=X <download threads> upd=XXX <update flags> log=XXX <log flags> +5. In cause of troubles join mailing list + +6. Use http://scache.sf.net/ for user friendly interface to this program + Special URL formats: ==================== @@ -27,7 +33,7 @@ LIMITATIONS: ============ - Content-type and size handling is not implemented. + Content-type and size handling is not yet implemented. Have fun. Index: loader.sgml =================================================================== RCS file: /cvsroot/scloader/loader/doc/loader.sgml,v retrieving revision 1.1.1.1 retrieving revision 1.2 diff -u -d -r1.1.1.1 -r1.2 --- loader.sgml 24 May 2003 14:05:08 -0000 1.1.1.1 +++ loader.sgml 23 Sep 2003 20:27:05 -0000 1.2 @@ -6,7 +6,7 @@ <author> <name>Radim Kolar</name> - <email>hs...@cy...</email> + <email>hsn/at/cybermail.net</email> </author> <version>0.25</version> @@ -57,10 +57,6 @@ files or consult sources for more informations. <p> This manual was created using <package>debiandoc-sgml</package> package, which allows to generate many output formats from one source. -<p> -English is not my native language and if you see any errors, just -ignore it or <url id="mailto:hs...@cy...?Subject=loader_manual" name="mail me">. -</p> <sect id="about"> <heading>About this program</heading> <p>I have tried many programs which copies web to local disk, but none |
From: <hs...@us...> - 2003-09-23 20:27:10
|
Update of /cvsroot/scloader/loader/samples In directory sc8-pr-cvs1:/tmp/cvs-serv2673/samples Modified Files: loader.cnf Log Message: Index: loader.cnf =================================================================== RCS file: /cvsroot/scloader/loader/samples/loader.cnf,v retrieving revision 1.1.1.1 retrieving revision 1.2 diff -u -d -r1.1.1.1 -r1.2 --- loader.cnf 12 Sep 2003 18:00:05 -0000 1.1.1.1 +++ loader.cnf 23 Sep 2003 20:27:05 -0000 1.2 @@ -1,20 +1,21 @@ #sample config file for loader # http://scloader.sourceforge.net/ # -#http_proxy proxy-server port, use Smart Cache if possible :) +#http_proxy proxy-server port, +# use Smart Cache if possible :) http://scache.sf.net/ +# http_proxy 127.0.0.1 8080 #how many files download at once? threads 2 -#What to do with downloaded files? - -#do not store it +#What to do with downloaded files? Define localstore type +#1) do not store it #localstore null -#use Smart Cache data directory, enter path to SC main config. +#2) use Smart Cache data directory, enter path to SC main config. localstore smartcache ../scache045/scache.cnf -#or write them to specified directory (like wget does) +#3) or write them to specified directory (like wget does) #localstore directory data case_sensitive_matching 1 @@ -30,7 +31,8 @@ #if none specified, default server options are initialized to system defaults #Server options -#TODO: implement changepolicy (????) +#TODO: implement changepolicy when jumping to other location (????) +# #passive,nofetch - don't process it when this config file is invoked #active,autostart - allways start fetching when this config file is invoked #nodefaults - don't process ANY settings in 'default' server after settings @@ -57,7 +59,7 @@ DefaultMask target=site src=IMG upd=norefresh act=noparse log=server q=0.5 #Locations -#URLMASK regexp(s) included from HTML TAGs +#URLMASK regexp(s) included from HTML TAGs #ACTIONS and OPTIONS # @@ -110,7 +112,7 @@ # size=unknown - only if we don't know size of object beeing downloaded # size=any - don't care about it -# poradi zpracovavani EXT,URLMASK,CONTENTMASK +# order of proccessing 1) EXT 2)URLMASK 3)CONTENTMASK #URLmasks are processed BEFORE OBJECT'S LOADING STARTS # MASKS xx,yy,zz = xx OR yy or ZZ |