Re: [Simpletest-support] test failure in alpha7 package

[Marcus B. <ma...@la...>]

Hi...

Jason Sweat wrote:
> Hi Marcus,
> 
> I downloaded the php_simple_test_1.0alpha7 tarball from sf.net, unpacked it and
> ran the tests/all_tests.php script.
> 
> All tests
> Fail: Boundary
> tests->live_test.php->testoflivecookies->testtimedcookieexpiry->Not expecting
> cookie [short_cookie]
> 51/51 test cases complete: 961 passes and 1 fails.
> 
> The unit_tests script runs fine.
> 
> I saw the archive email from 8-15 for this list stating: Please use only the
> unit_tests.php script until Alpha7. But I thought I would report it since I am
> using Alpha7 :)

That's correct. I had to update the sample test pages whilst I was 
rolling out Alpha7, thus invalidating Alpha6's tests.

The test is actually quite straightforward...

function testTimedCookieExpiry() {
     $this->get('http://www.lastcraft.com/test/set_cookies.php');
     $this->restartSession(time() + 101);
     $this->assertNoCookie("session_cookie");
     $this->assertNoCookie("short_cookie");
     $this->assertCookie("day_cookie", "C");
}

It reads the cookies from the test page and then jumps 101 seconds into 
the future (closing and reopening the browser) to see which have 
expired. The test page starts with...

<?php
     setcookie("session_cookie", "A");
     setcookie("short_cookie", "B", time() + 100, "/");
     setcookie("day_cookie", "C", time() + 24 * 3600, "/");
     setcookie("path_cookie", "D", time() + 100, "/test/path/");
?>

Of course I have allowed a one second margin of difference between the 
server and client+transport times! Being close to the server (30ms ping) 
on all the machines I tested with sunk me on this one.

Ok first of all I need to allow 15 seconds at least for the wire time. 
If you just hack the test to...

$this->restartSession(time() + 115);

...that should fix it. Is your client box using an internet clock signal 
(such as NTP or NITS)? I hope the lastcraft server is or I am in big 
trouble. If you are not and it still fails then the test suite is 
probably correctly reporting a failure in the test environment. Of 
course it's not the job of the tester tests to report this, but...

How about this for a plan?

I'll up the short cookie plus margin to two days rather than 101 secs 
(in case people have forgotten to set the TZ variable) and the day 
cookie will become a month cookie. I'll add a seperate tighter test with 
an explicit description of the timing issues with a fifteen second wire 
time to replace the above. I won't be able to change the tests until 
just before Beta1.

This should normally work and drop a meaningful hint about clocks and 
cookies and testing if people haven't set up their machines with this in 
mind.

Meanwhile I'll confirm that the time is correct at lastcraft.

> 
> Regards,
> 
> Jason
> 
> P.S. the readme lists this list's email address as 
> sim...@so... when it is actually  sim...@li...
> 

Ouch. That probably means that we are the only members :(. I'll fix that 
straight away in CVS and enter the above as bug reports.

yours, Marcus
-- 
Marcus Baker, ma...@la..., no...@ap...