From: Fabian K. <fk...@fa...> - 2009-02-22 18:06:55
|
Lee <le...@gm...> wrote: > On 2/22/09, Fabian Keil <fk...@fa...> wrote: > > Lee <le...@gm...> wrote: > > > >> Is it expected behavior to not run all of the regression tests? > > > > Yes. Tests that are known not to apply are skipped. > > How hard would it be to put out a "skipping <whatever>: known not to > apply" message? Trivial, after all it's written in Perl. > I don't use the regression test very often because my action files > cause lots of errors. So when I do run the regression test on a > vanilla build and it comes back with less than 100% success I don't > know if that's normal/expected or indicating a problem. Adding an > explanation of why something was skipped would make it clear there's > no problem.. Did you investigate the reason for these errors? This can happen if custom action files overrule earlier sections that are regression tested, but as it doesn't happen here, solving this problem has been low on my list so far. If you provide some more details about which tests fail and why, I'll look into possible workarounds. > >> $ current/tools/privoxy-regression-test.pl --privoxy-address > >> 127.0.0.1:8118 > >> 2009/02/22 08:54:11: Asking Privoxy for the number of action files > >> available ... > >> 2009/02/22 08:54:11: Gathering regression tests from 4 action file(s) > >> delivered by Privoxy. > >> 2009/02/22 08:54:12: Executing regression tests ... > >> 2009/02/22 08:55:57: Executed 426 regression tests. Skipped 25. 426 > >> successes, 0 failures. > > > > This however is the result of an incorrect dependency check. > > It should be fixed in CVS now. > > It's better > $ current/tools/privoxy-regression-test.pl --privoxy-address 127.0.0.1:8118 > 2009/02/22 11:29:55: Asking Privoxy for the number of action files available ... > 2009/02/22 11:29:56: Gathering regression tests from 4 action file(s) > delivered by Privoxy. > 2009/02/22 11:29:56: Executing regression tests ... > 2009/02/22 11:31:30: Executed 448 regression tests. Skipped 3. 448 > successes, 0 failures. Looks reasonable for a default installation with FEATURE_KEEP_ALIVE enabled and the action editor disabled. > >> And is there an easy way to tell which tests were skipped? > > > > Not with Privoxy-Regression-Test itself, but I agree that > > it should be. At the moment the easiest way is to check > > the regression-tests.action file: > > > > # Some dependencies Privoxy-Regression-Test should know about: > > # > > # Level 9 needs = config line user-manual\s+(\.?\.?/|[A-Za-z]:) > > # Level 12 needs = config line enable-edit-actions\s+1 > > # Level 13 needs = feature status FEATURE_CONNECTION_KEEP_ALIVE Yes > > # Level 14 needs = feature status FEATURE_CONNECTION_KEEP_ALIVE No > > > > The Level 9 line used to be: > > > > # Level 9 needs = config line user-manual\s+(/|[A-Za-z]:) > > > > so tests with level 9 were skipped in case of relative paths. > > Are the levels cumulative? As in do I have to set > "enable-edit-actions" for the level 12 checks to work if I want to do > the level 13 connection keep-alive checks? Test levels are independent of each other and calling them levels wasn't the wisest decision ever. "Test categories" is probably more fitting. > >> I tried " --max-level 9" thinking it might be the doc tests that were > >> skipped & the process hung. <ctrl>C didn't do anything.. I had to go > >> into task manager & kill perl.exe to get control back to the cygwin > >> terminal window. Dunno if the regression test follows all the links > >> in the user manual or not, but the user manual has a link to the > >> developer manual & that isn't included in the release package :( I > >> need to fix that, but Privoxy does come back with a 404: Privoxy > >> Configuration page not found so [it seems to me] that shouldn't cause > >> a hang. > > > > The regression tests don't follow any links and > > simply request URLs that are explicitly given. > > > > I can't reproduce the hang on FreeBSD, so it may be a cygwin+perl > > issue. I'll look into this in the next days. > > I don't know if it was the missing doc/developer-manual/ or your > change to the action file, but it doesn't hang now: > $ current/tools/privoxy-regression-test.pl --privoxy-address > 127.0.0.1:8118 --max-level 9 > 2009/02/22 11:39:39: Asking Privoxy for the number of action files available ... > 2009/02/22 11:39:39: Gathering regression tests from 4 action file(s) > delivered by Privoxy. > 2009/02/22 11:39:40: Executing regression tests ... > 2009/02/22 11:41:13: Executed 445 regression tests. Skipped 6. 445 > successes, 0 failures. The developer manual isn't requested (as it isn't delivered by Privoxy anyway), so I assume changing the action file masked the problem. > > Where in the user manual is the link that results in a 404? > > It doesn't sound like a Windows-only problem to me. > > The line just before section 2.3 Keeping your Installation Up-to-Date > links to the developer manual (For more detailed instructions on how > to build Redhat RPMs, Windows self-extracting installers, building on > platforms with special requirements etc, please consult the developer > manual.) That's a a bug. The link shouldn't be relative as Privoxy currently only delivers the user manual. > The problem was having "user-manual ./doc/user-manual/" in the config > file and not including doc/developer-manual/ in the distribution > package. It works fine now :) The link should still lead to the 404 page. > Sorry for the long response time - I'm doing this on Vista and the > default file names & permissions are killing me. cd'ing or copying > files to /cygdrive/c/Program Files (x86)/Privoxy in cygwin is a royal > pain & a 'runas /user:administrator cmd" doesn't let me fix file > permission problems. I have to log in as the administrator to change > file permissions :( No problem, take your time. Fabian |