Re: [proxyTools-users] first time user of proxyTools (was:Re: Help for an absolutely new user)

[wayne <wa...@ny...>]

> From: Mat Brown <mat...@ya...>
> Subject: Re: [proxyTools-users] first time user of proxyTools (was:Re: Help for an absolutely new user)
> To: pro...@li...
> Cc: wayne <wa...@ny...>

No need to copy me. I get all the list posts anyway ...

> Dear Wayne
 
[...]

> >> >2) double-click localProxy.pl, and wait for the extra Perl modules 
> >> >   to download.
> >> 
> >> Done
> >> 
> >> After this a GUI comes up
> >> In that GUI there is box for configuration
> >
> >Good. Didn't it try to download any modules?
> >
> 
> It must have. 
> I din't notice. May be I was busy checking the site :-) 

Heh, that must make it truly user friendly. 
There are a lot of modules it needs which don't come with the core 
Perl.
I see you got it running below, so it must have got them.

> >> >6) configure the browser as described in the log window (proxy at 
> >> >   localhost:10080)
> >> >7) browse until it learns (complicated web pages are best). 
> >> >
> >> 
> >> The POP up window lists many proxies as "not" responding and so on
> >
> >That's because you have an old copy from the zip (September?).
> >
> 
> Yes. I have the one dated 04th Sep 02. That seems to be the oldest
> build for "non-Technical" users. As far as I understand the CVS is for
> the technically competent ? I have no idea of CVS. Am I missing
> something here. 

Or the 'youngest build' :-)
After all, the other releases are older.
'Latest' we could agree on.

I don't know how to advise on the CVS stuff. It requires no extra 
knowledge, but you're more likely to see bugs, crashes etc. 
As long as you're willing to keep a working version somewhere as 
a backup, you can take advantage of it (the fixes, and all the new 
features as they happen). 
You get to be able to tell me about bugs more often.
You get to call yourself a beta tester :-)

The biggest problem seems to be for people to be able to use WinCVS 
properly :-(

Truth is, it's so simple, I'll probably add code to do it for the 
user one day. A button to press which gets at least the latest 
proxy database, and probably the new code too.

> >> Further questions
> >> ------------------
> >> 1) Local proxy is accepting connections from **just** my own
> firewall
> >
> >So that others can't use your LP proxy. Security mainly.
> >If you want to share your LP with friends, I'll tell you how to set 
> >that in a configuration file.
> >
> 
> Ok. 
> I wish to use it myself / test it myself. 
> I do *not* wish to share it. 

That's why it defaults to *not* allowing access from elsewhere.

> Pl. forget this question for the moment
> 
> >> !! I am unable to use other working proxies. At least I haven't
> found
> >> out how to merge the results from find proxy into local proxy :-)
> >
> >It's a long sequence:
> >The idea is that hosts.xml (the big database everyone gets from 
> >hosts.zip) is updated by me (or you if you must), and the config file 
> >you use is personalized with your own proxies by you. In both cases, 
> >the only tool for this is mergeHosts.pl (or by hand if you're daring).
> >
> 
> Didn't understand the above much. But still proceeding

You did it exactly correctly below.
Your modesty exceeds your lack of understanding :-)

> >If you copy config-UAE-dialup.xml to config-mat.xml and restart the 
> >both GUI and backend LP, you can select that config for your own use.
> >
> 
> The way I started  ..
> - click on Local proxy.pl file and wait for the GUI to come up. 
> - choose UAE-Dial up in the config box 
> and 
> - click "start Services". 

That will do, the copy is nice because you know that you can CVS, or 
unzip a new release in the same directory and it will not get 
overwritten. Your secret proxies are safe.

> I did **not** copy config-UAE-dialup.xml to config-mat.xml 
> 
> What is config-mat.xml ? 

A config file you named after you :-)
The file name format is important though, so you get to see it 
in the GUI config list. config-<something>.xml

> what is it named as in the config box inside the GUI 

mat

> For e.g. for a lay user like me config-UAE-dialup.xml appears as
> UAE-dial up in the config box within the GUI. Similarly what does
> config-mat.xml appear as

mat

> >It will never get overwritten by any new versions of the tools that 
> >you get.
> >
> 
> didn't understand this either

See above.

> >The input to mergeHosts.pl must be in a standard format, and that is 
> >produced by statProxy.pl (the proxyTools generalized proxy tester).
> >
> >If your proxy list is in listOfProxies.txt, the whole sequence goes 
> >like this:
> >perl statProxy.pl -t all:-18 -l listOfProxies.txt > results.txt
> >(leave off the >results.txt if you want to see results on the screen).
> >
> 
> I had some 600 + proxies in a file ! The above command is taking quite
> some time with that file !!. Though I'm occassionally checking that it
> hasn't gone for a hang, it has taken some hours !!

The new version (CVS) is quicker, but it is designed to evaluate 
the proxy capabilities properly. Most scanners just check if they 
can connect. SP has 18 tests to do on each proxy as specified above.

To just get a list of working proxies, try -t 0 instead 
of -t all:-18

> How often does one run this ? 

On a file like that, once a year (in fact, I often throw such files 
away after the initial run).
Therafter, you run it on your config file where you have merged the 
good ones from that list.

> Proxies come and go often. Proxies live now are dead later in the day
> and come back next day and so on. If this checking is taking time how
> do I check often ? 

It runs in the background, so no problem.
I often split the files into smaller ones and check a new one each 
day.
But your checking time is better spent on the ones that localProxy 
is selecting for it's build. You can get those by copy and paste 
from the GUI layer 0 windows, or you can click 'Save running 
configuration' and then test the ones localProxy found to be bad 
(after browsing for some time) in localInfo-saved.xml, or after 
shutting down LP, you can test the 'last' configuration in 
config-last.xml, or ...

These approaches are better because LP does most of the checking 
for you, as you browse. Remember it can be fed a heap of bad proxies 
and still browse ok. It's designed for it.

Also it's safer for the average user to *not* test huge bunches of 
proxies.
China, KSA etc. are not known for their friendliness to users when 
complaints are received from proxy owners.

And one more excuse :-)
When I was adding the parallel testing code in the latest version, 
I thought about doing each test in parallel too, but decided the 
connection (alive or dead) test was enough. Any more than that 
means hitting a web page with 600 hits in a few seconds from the 
same address for exactly the same url - they tend to notice such 
things :-)
LP needs (well, uses) the speed of the proxy and that can only be 
found by downloading a fixed reference page (currently www.panix.com).
And that's just for test 0!

> >Then:
> >perl mergeHosts.pl results.txt config-mat.xml
> >will update your config appropriately (disabling proxies as necessary,
> 
> >adding new ones, updating existing ones etc.) based on speed, 
> >connectability, reliability, capabilities.
> 
> I suppose the sequence is 
> 
> 1. copy config-UAE-dialup.xml to config-mat.xml 
> 2. perl statProxy.pl -t all:-18 -l listOfProxies.txt > results.txt
> ( 1 and 2 may be independant of each other. I didn't mean a chronology
> too ) 

That's ok, but after you copy the config don't copy it each time or 
you end up with only the latest test results in it.
MergeHosts is designed to take account of previous tests, since 
proxies come and go all the time.
The proxy speed and reliability are running averages of the last 
few tests. Even if it appears to be dead on the current test, 
mergeHosts will not disable it unless the averaged speed or 
reliablity drops below a low threshold.

> 3. perl mergeHosts.pl results.txt config-mat.xml
> 4. Restart both GUI and backend LP
> 
> 5. What config should I select at this point ?
> 
> BTW what do you mean by back end LP

You restarted it in 4 above :-)

> For a lay user is it the same as "Start Services" within the GUI ?

Yes.
It's the second command window you see. the first one comes with 
the GUI startup, and the second with the 'Start services' click.
That's the guy that does all the hard work.

> >Once in a while you can run statProxy on the config file itself, and 
> >use mergeHosts to update the config again. 
> >
> 
> ********
> Sorry didn't understand that again

Simply run SP with the config file as input. SP is good at selecting 
proxy-like strings from any file, and then testing them.

> Could you please give me a list of commands ?

A full test:
perl statProxy.pl -t all:-18 -l config-mat.xml
Or if yu don't have much time, and want to get alive/dead and speed 
results just use -t 0

> >That ensures that all the 
> >ones in there (even if they came from somewhere other than your 
> >current proxylist file), are updated (and eventually disabled if bad).
> >You can do this as often as you like.
> >
> >> 2) Please tell me what are complicated web pages ?
> >Ones with lots of banners, thumbnails, popups, or other components.
> >They all exercize LP.
> >
> 
> 
> Ok
> 
> Will send you the results.tx file by seperate mail

The log files are instructive. My comments on them tell everyone 
how to see LP is working (or not!).
About the only reason to do that is if your proxies are secret.
Your own address probably should be munged (it sometimes appears 
more than once in the log, where LP thinks you might have been 
disconnected).
If that's the reason, ok, but don't think the list doesn't want 
the long logs. I think it should. ProxyTools problems/info is what 
the list is for.

> Best Regards
> 
> Mat_Brown_Jr

--
wa...@ny...
http://proxytools.sourceforge.net/