simpletest-support — Help, advice, bugs and workarounds.

RE: [Simpletest-support] Testing forms

[Matt G. <mgr...@ne...>]

Actually, I solved my own problem.  By dumping most of the class variables I
was able to notice that my form appeared only in the open_forms array but
not in closed_forms where the form routines draw from.  Sure enough, someone
had put the </form> tag inside conditional logic.  Once I fixed that,
everything worked fine.  I've been able to make some useful tests by posting
to the action pages and setting all of the variables manually.  I'm pretty
sure now that the actual form test routines are not going to be helpful
anyway due to the amount of javascript involved in our pages.  Perhaps this
is something we want to look at for refactoring but there's just not the
time now :-\

Regardless, thanks for putting out this fine piece of software.  It's been
very useful, on the whole.  I'll let you know if there's any feature that
comes up that would save us (and probably others) a lot of time.

Matt

-----Original Message-----
From: Marcus Baker [mailto:ma...@la...]
Sent: Tuesday, February 17, 2004 7:00 PM
To: mgr...@ne...; simpletest-support
Subject: Re: [Simpletest-support] Testing forms


Hi...

Matt Griffin wrote:
> I'm new to SimpleTest so bear with me.

So am I ;).

 > However, I can't find
> any documentation for these (the link to for mtesting is dead on the
> website) and I'm having some difficulties.

I am actually writing the documentation for this now. It only actually
rolled out in the last Beta, was buggy and has just been heavily
refactored for Beta4.

 > When I call clickSubmit($label)
> I simply remain on the same page.  My thought is that the submit button is
> not being found or addressed properly but I cannot determine why.

Can you mail me the page source?

 > Also, am
> I correct in assuming that the browser object is ignorant of all
javascript?

Absolutely. In fact the HTML parsing is currently pretty limited as
well. This is the main area of effort right now, but even the version
1.0 release will not have JavaScript support. There is a possibility
that a later version will use the HTML Tidy module for the HTML parsing.
With luck someone will write a PHP extension for the Rhino module (the
JavaScript intepreter) and then a solution would at least be possible.
Realistically that would be a year away at least.

The current schedule is Beta4 in a couple of days (no frames support),
version 1.0 in about 6 weeks. Beta4 has all of the basic HTML widgets,
but no browser history list (no back button). This and a few other
niceties should make it into 1.0, but no further core functionality has
been mapped out. Do you have any requests?

I am afraid you are very much in the early adopter camp here :(. You can
probably use the Java HTTPUnit tool to accomplish what you want,
although it's not so easy to use.

>
> Matt Griffin
>

yours, Marcus

p.s. I have CC'd this to you so that you can just reply with the HTML
source file.
--
Marcus Baker, ma...@la..., no...@ap...

Re: [Simpletest-support] Testing forms

[Marcus B. <ma...@la...>]

Hi...

Matt Griffin wrote:
> I'm new to SimpleTest so bear with me.

So am I ;).

 > However, I can't find
> any documentation for these (the link to for mtesting is dead on the
> website) and I'm having some difficulties.

I am actually writing the documentation for this now. It only actually 
rolled out in the last Beta, was buggy and has just been heavily 
refactored for Beta4.

 > When I call clickSubmit($label)
> I simply remain on the same page.  My thought is that the submit button is
> not being found or addressed properly but I cannot determine why.

Can you mail me the page source?

 > Also, am
> I correct in assuming that the browser object is ignorant of all javascript?

Absolutely. In fact the HTML parsing is currently pretty limited as 
well. This is the main area of effort right now, but even the version 
1.0 release will not have JavaScript support. There is a possibility 
that a later version will use the HTML Tidy module for the HTML parsing. 
With luck someone will write a PHP extension for the Rhino module (the 
JavaScript intepreter) and then a solution would at least be possible. 
Realistically that would be a year away at least.

The current schedule is Beta4 in a couple of days (no frames support), 
version 1.0 in about 6 weeks. Beta4 has all of the basic HTML widgets, 
but no browser history list (no back button). This and a few other 
niceties should make it into 1.0, but no further core functionality has 
been mapped out. Do you have any requests?

I am afraid you are very much in the early adopter camp here :(. You can 
probably use the Java HTTPUnit tool to accomplish what you want, 
although it's not so easy to use.

> 
> Matt Griffin
> 

yours, Marcus

p.s. I have CC'd this to you so that you can just reply with the HTML 
source file.
-- 
Marcus Baker, ma...@la..., no...@ap...

[Simpletest-support] Testing forms

[Matt G. <mgr...@ne...>]

Hi all,

I'm new to SimpleTest so bear with me.  I've had great luck testing objects
and dynamic pages (by posting directly to them) and now I'm trying to test
forms using the setField() and clickSubmit() methods.  However, I can't find
any documentation for these (the link to for mtesting is dead on the
website) and I'm having some difficulties.  When I call clickSubmit($label)
I simply remain on the same page.  My thought is that the submit button is
not being found or addressed properly but I cannot determine why.  Also, am
I correct in assuming that the browser object is ignorant of all javascript?

Matt Griffin

Re: [Simpletest-support] Testing for errors

[Marcus B. <ma...@la...>]

Hi...

Jeff Moore wrote:
> Should I put a swallowErrors() call after my assertError() to prevent 
> the exceptions?

Yes, that will cure it. As Jason suggests that may swallow other errors 
unless you have a discriminatory enough match, so it really depends on 
the specific example. The alternative is to have a return statement 
after the trigger_error(). If someones app. uses WACT inside a custom 
error handler, you would need to document the behaviour anyway, so that 
extra return statement may ultimately be a simplification.

I don't see an easy way out of this until we finally get exceptions.

> 
> If so, should I put a swallowErrors() call at the end of every test case 
> that calls assertError?

If you really need to do that as a blanket case. This example seems 
unusual to me.

> 
> Thanks,
> 
> Jeff

yours, Marcus
-- 
Marcus Baker, ma...@la..., no...@ap...

Re: [Simpletest-support] Testing for errors

[Jason S. <jsw...@ya...>]

--- Jeff Moore <je...@pr...> wrote:
> The problem is that condition2 is dependant upon condition1.  If  
> condition1 is true, condition2 will always be true as well.  There is  
> no way to re-arrange the test cases to avoid getting two errors.  It  
> would be possible to re-arrange the code, but that doesn't seem like  
> the right thing.

how about just:
  $this->assertError(error_signature1);
  $this->assertError(errop_signature2);

?


__________________________________
Do you Yahoo!?
Yahoo! Finance: Get your refund fast by filing online.
http://taxes.yahoo.com/filing.html

Re: [Simpletest-support] Testing for errors

[Jeff M. <je...@pr...>]

On Feb 14, 2004, at 11:03 PM, Jason Sweat wrote:

> --- Jeff Moore <je...@pr...> wrote:
>> Hello,
>>
>> I was looking for some advise regarding testing for errors.  If I have
>> some code that I want to test of the form:
>>
>> if (condition1) {
>> 	trigger_error()
>> }
>>
>> if (condition2) {
>> 	trigger_error()
>> }
>>
>> And I have the following test case
>>
>> // trigger error condition 1
>> $this->assertError();
>
> Marcus has conditioned me to use the assertErrorPattern() method.
>
> http://simpletest.sourceforge.net/api/SimpleTest/UnitTester/ 
> UnitTestCase.html#assertErrorPattern

About 99.5% of the errors that I want to test for were triggered by the  
WACT RaiseError() function.  Since the RaiseError signature involves  
only error codes and data, it should be possible to test for the items  
directly.  No regular expressions required.

For example, the call:

RaiseError('compiler', 'VALUENOTFOUND', array(
     'file' => $file,
     'section' => $section,
     'key' => $key));

could be directly tested for with a statement similar to

$this->expectError(BuildWactError('compiler', 'VALUENOTFOUND', array(
     'file' => '/filename.ini', 'section' => 'compiler', 'key' =>  
'path')));


> I would say your best bet would be to set up the test to meet each or  
> none, one
> or both of the conditions (perhaps four separate methods in the test  
> case?),
> and then explicitly test for the expected errors.

The problem is that condition2 is dependant upon condition1.  If  
condition1 is true, condition2 will always be true as well.  There is  
no way to re-arrange the test cases to avoid getting two errors.  It  
would be possible to re-arrange the code, but that doesn't seem like  
the right thing.

(if you are interested, the WACT test case that triggered this  
discussion is
cases/template/tags/core/include.test.php->CoreIncludeTagTestCase- 
 >testmissingfileattribute

>
>> Should I put a swallowErrors() call after my assertError() to prevent
>> the exceptions?
>>
>> If so, should I put a swallowErrors() call at the end of every test
>> case that calls assertError?
>
> I would avoid use of swallowErrors at all, I have found this got rid  
> of error
> that crept in unexpectedly on me. YMMV

Hmmm. :(

Re: [Simpletest-support] Testing for errors

[Jason S. <jsw...@ya...>]

--- Jeff Moore <je...@pr...> wrote:
> Hello,
> 
> I was looking for some advise regarding testing for errors.  If I have 
> some code that I want to test of the form:
> 
> if (condition1) {
> 	trigger_error()
> }
> 
> if (condition2) {
> 	trigger_error()
> }
> 
> And I have the following test case
> 
> // trigger error condition 1
> $this->assertError();

Marcus has conditioned me to use the assertErrorPattern() method.

http://simpletest.sourceforge.net/api/SimpleTest/UnitTester/UnitTestCase.html#assertErrorPattern

I would say your best bet would be to set up the test to meet each or none, one
or both of the conditions (perhaps four separate methods in the test case?),
and then explicitly test for the expected errors.

> Should I put a swallowErrors() call after my assertError() to prevent 
> the exceptions?
> 
> If so, should I put a swallowErrors() call at the end of every test 
> case that calls assertError?

I would avoid use of swallowErrors at all, I have found this got rid of error
that crept in unexpectedly on me. YMMV

Regards,

Jason

__________________________________
Do you Yahoo!?
Yahoo! Finance: Get your refund fast by filing online.
http://taxes.yahoo.com/filing.html

[Simpletest-support] Testing for errors

[Jeff M. <je...@pr...>]

Hello,

I was looking for some advise regarding testing for errors.  If I have 
some code that I want to test of the form:

if (condition1) {
	trigger_error()
}

if (condition2) {
	trigger_error()
}

And I have the following test case

// trigger error condition 1
$this->assertError();

When simpletest records the errors, it allows execution to continue, 
where in the program, normally execution would not continue past the 
first trigger_error statement.  This causes two errors to be triggered 
(when condition 2 is also true).  The second error occurs only because 
the first error was ignored.

Now, my test passes because an error did occur, but simpletest records 
the second error as an exception and shows a red bar.  I'd like to have 
a green bar.  :)

Should I put a swallowErrors() call after my assertError() to prevent 
the exceptions?

If so, should I put a swallowErrors() call at the end of every test 
case that calls assertError?

Thanks,

Jeff

Re: [Simpletest-support] error suppression operator

[Jeff M. <je...@pr...>]

On Feb 13, 2004, at 10:16 PM, Marcus Baker wrote:

> Jeff Moore wrote:
>> I'm having some trouble writing tests for some code that uses the 
>> error suppression operator (@).
>
> Fixed in CVS. Will roll out in Beta4. I basically added pretty much 
> the code you suggested and a stack of tests.

Excellent.  Thank you very much.

Re: [Simpletest-support] error suppression operator

[Marcus B. <ma...@la...>]

Hi...

Jeff Moore wrote:
> I'm having some trouble writing tests for some code that uses the error 
> suppression operator (@).

Fixed in CVS. Will roll out in Beta4. I basically added pretty much the 
code you suggested and a stack of tests.

Thanks for that.

yours, Marcus
-- 
Marcus Baker, ma...@la..., no...@ap...

Re: [Simpletest-support] error suppression operator

[Marcus B. <ma...@la...>]

Hi...

Jeff Moore wrote:
> I'm having some trouble writing tests for some code that uses the error 
> suppression operator (@).

Unfortunately SimpleTest still very much reflects my personal biases. I 
tend not to use the @ operator. I know of no way to distinguish a 
warning that is suppressed from one that isn't assuming that your ini 
file reports all errors.

> Adding the following to simpleTestErrorHandler clears up the problem and 
> the test passes.
> 
>         if ( ( $severity & error_reporting() ) != $severity ) {
>             return; // Ignore this error because of error suppression
>         }

Ok, I should probably add this or something similar to the core code. 
Obviously this won't happen until the next release (Beta4). I'd like to 
investigate the error_reporting() function further before I decide if 
that is OK?

> 
> Any suggestions on how to proceed?

If your fix is sufficient then use it for now. Hopefully on the next 
release of Wact (I assume) you can specify SimpleTest Beta4 and things 
should be less severe. Other actions, decorating the runner or 
subclassing the reporter, would simply involve you in more work than 
really necessary.

I am afraid that is all that I can suggest right now until I investigate 
things a bit more :(.

> 
> Jeff

yours, Marcus
-- 
Marcus Baker, ma...@la..., no...@ap...

Re: [Simpletest-support] error suppression operator

[Jason S. <jsw...@ya...>]

--- Jeff Moore <je...@pr...> wrote:
> Hello,
> 
> I'm having some trouble writing tests for some code that uses the error 
> suppression operator (@).
> 
> My problem is that simpletest records ALL errors, even ones that the 
> code explicitly suppresses using the error suppression operator.

IIRC, this is a problem with _ALL_ custom error handlers.  All errors are
passed, even with the suppresion operator in place.  I recall this came up on
the smarty list a while back, but I do not remember the resolution.  Might try
to google and see if you can find anything regarding error suppression and
custom error handlers.



> 
> I am getting cascades of unimportant errors that cause my test to 
> indicate failure when I don't consider it to have failed.
> 
> Adding the following to simpleTestErrorHandler clears up the problem 
> and the test passes.
> 
>          if ( ( $severity & error_reporting() ) != $severity ) {
>              return; // Ignore this error because of error suppression
>          }
> 
> Any suggestions on how to proceed?
> 
> Jeff
> 
> 
> 
> -------------------------------------------------------
> SF.Net is sponsored by: Speed Start Your Linux Apps Now.
> Build and deploy apps & Web services for Linux with
> a free DVD software kit from IBM. Click Now!
> http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
> _______________________________________________
> Simpletest-support mailing list
> Sim...@li...
> https://lists.sourceforge.net/lists/listinfo/simpletest-support


__________________________________
Do you Yahoo!?
Yahoo! Finance: Get your refund fast by filing online.
http://taxes.yahoo.com/filing.html

[Simpletest-support] error suppression operator

[Jeff M. <je...@pr...>]

Hello,

I'm having some trouble writing tests for some code that uses the error 
suppression operator (@).

My problem is that simpletest records ALL errors, even ones that the 
code explicitly suppresses using the error suppression operator.

I am getting cascades of unimportant errors that cause my test to 
indicate failure when I don't consider it to have failed.

Adding the following to simpleTestErrorHandler clears up the problem 
and the test passes.

         if ( ( $severity & error_reporting() ) != $severity ) {
             return; // Ignore this error because of error suppression
         }

Any suggestions on how to proceed?

Jeff

[Simpletest-support] expectCookie() removed in Beta4

[Marcus B. <ma...@la...>]

Hi...

This method is a refactoring victim. The same functionality can be had 
with setCookie() and assertCookie() anyway, so it's loss is marginal. I 
simply have no way of supporting it in a deprecated state under the new 
structure so it had to go without warning. Sorry for any inconvenience 
caused.

yours, Marcus
-- 
Marcus Baker, ma...@la..., no...@ap...

Re: [Simpletest-support] SimpleMock::ExpectArgumentsAt()

[Jason S. <jsw...@ya...>]

--- Marcus Baker <ma...@la...> wrote:
> Hi...
> 
> Jason Sweat wrote:
> > Well, I think we should at least detect if an array was supplied, and fail
> the
> > test if something other than an array was provided rather than letting the
> test
> > have a php error for invalid argument to the foreach.
> 
> Oh yeah. That would be a good idea wouldn't it :-/

Okay, added:
        function testBadArgParameter() {
            $mock = &new MockDummy($this);
            $mock->expectArguments("aMethod", "foo");
            $this->assertErrorPattern('/\$args.*not an array/i');
            $mock->aMethod();
            $mock->tally();
       }
to TestOfMockExpectations Unit Test Case

__________________________________
Do you Yahoo!?
Yahoo! Finance: Get your refund fast by filing online.
http://taxes.yahoo.com/filing.html

Re: [Simpletest-support] SimpleMock::ExpectArgumentsAt()

[Marcus B. <ma...@la...>]

Hi...

Jason Sweat wrote:
> Well, I think we should at least detect if an array was supplied, and fail the
> test if something other than an array was provided rather than letting the test
> have a php error for invalid argument to the foreach.

Oh yeah. That would be a good idea wouldn't it :-/

> 
> I'll look into possible fixing that tonight.

Cheers.

yours, Marcus
-- 
Marcus Baker, ma...@la..., no...@ap...

Re: [Simpletest-support] SimpleMock::ExpectArgumentsAt()

[Jason S. <jsw...@ya...>]

--- Marcus Baker <ma...@la...> wrote:
> Hi...
> 
> Jason Sweat wrote:
> > Would it be reasonable to automatically wrap the third argument in an
> array, if
> > it is not an array, to avoid this error?
> 
> I thought about this, but could not come to a final conclusion and so 
> went with simplicity. The problem with overloaded arguments is that you 
> have more stuff to explain in the documentation and more things to 
> figure out in use. At least this way it always behaves the same way...
> 
> $mock->expectArguments('method', $login->getPermissions());
> 
> What if the method sometimes returns false and sometimes an array? Not a 
> likely scenario, but you kind of see what I mean. If I were going to go 
> that way I should have used the func_get_args() to have them flat. 
> Perhaps I should have done that :(.

Well, I think we should at least detect if an array was supplied, and fail the
test if something other than an array was provided rather than letting the test
have a php error for invalid argument to the foreach.

I'll look into possible fixing that tonight.

__________________________________
Do you Yahoo!?
Yahoo! Finance: Get your refund fast by filing online.
http://taxes.yahoo.com/filing.html

Re: [Simpletest-support] SimpleMock::ExpectArgumentsAt()

[Marcus B. <ma...@la...>]

Hi...

Jason Sweat wrote:
> Would it be reasonable to automatically wrap the third argument in an array, if
> it is not an array, to avoid this error?

I thought about this, but could not come to a final conclusion and so 
went with simplicity. The problem with overloaded arguments is that you 
have more stuff to explain in the documentation and more things to 
figure out in use. At least this way it always behaves the same way...

$mock->expectArguments('method', $login->getPermissions());

What if the method sometimes returns false and sometimes an array? Not a 
likely scenario, but you kind of see what I mean. If I were going to go 
that way I should have used the func_get_args() to have them flat. 
Perhaps I should have done that :(.

yours, Marcus
-- 
Marcus Baker, ma...@la..., no...@ap...

[Simpletest-support] SimpleMock::ExpectArgumentsAt()

[Jason S. <jsw...@ya...>]

Exception: testcheckinputform->Unexpected PHP error [Invalid argument supplied
for foreach()] severity [E_WARNING] in
[/home/httpd/phplib/simpletest_1.0beta3/mock_objects.php] line [173]

Would it be reasonable to automatically wrap the third argument in an array, if
it is not an array, to avoid this error?

__________________________________
Do you Yahoo!?
Yahoo! Finance: Get your refund fast by filing online.
http://taxes.yahoo.com/filing.html

Re: [Simpletest-support] use WantedPatterExpectation for aguments in Expect* ?

[Marcus B. <ma...@la...>]

Hi...

Jason Sweat wrote:
> Basically wondering if you can use regex for arguments.  I have some functions
> that wrap a bunch of junk around a relativly simple SQL query before it hit's
> my mock, and I would just like to verify the original chunk of SQL is present
> in the first argument.

Very much so...

$connection = &new MockConnection($this);
$connection->expectOnce(array(
         new WantedPatternExpectation('/name = [\']Fred[\']/')));

Basically any parameter that is not an expectation class will have an 
identical expectation wrapped around it, otherwise the expectation class 
is used instead.

yours, Marcus
-- 
Marcus Baker, ma...@la..., no...@ap...

[Simpletest-support] use WantedPatterExpectation for aguments in Expect* ?

[Jason S. <jsw...@ya...>]

Basically wondering if you can use regex for arguments.  I have some functions
that wrap a bunch of junk around a relativly simple SQL query before it hit's
my mock, and I would just like to verify the original chunk of SQL is present
in the first argument.

__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free web site building tool. Try it!
http://webhosting.yahoo.com/ps/sb/

[Simpletest-support] useful additional assertion?

[Jason S. <jsw...@ya...>]

in my write test -> fail -> write code -> pass cycles, the first thing that
happens where I write a new method is that the test dies with a php fatal error
because the method does not exist.  Using this assertion I can write a test
that verifies the method exists, see it fail, add the method to the class I am
testing, see the test pass and then move on to real relevant tests of the
method.

        /**
         *      Confirms that a particular method is defined for a class.
         *      @param  string  $class          the class to test
         *      @param  string  $method         the name of the method to test
         *      @param  string  $message        optional - Message to display.
         *      @access public
         */
        function AssertMethodExists($class, $method, $message=false) {
                if (!class_exists($class)) {
                        $this->Fail("'$class' is not a class");
                }
                if (false === $message) {
                        $message = "$class::$method() does not exist.";
                }
                if (is_callable(array($class, $method))) {
                        $this->Pass($message);
                } else {
                        $this->Fail($message);
                }
        }

or perhaps I am just being pedantic?

__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free web site building tool. Try it!
http://webhosting.yahoo.com/ps/sb/

[Simpletest-support] [Fwd: Line numbers for failed tests]

[Zbynek W. <zw...@us...>]

This somehow did not get through. The line numbers would be really helpfull.
Zbynek

-------- Original Message --------
Subject: 	Line numbers for failed tests
Date: 	Thu, 29 Jan 2004 23:55:49 +0100
From: 	Zbynek Winkler <zw...@us...>
To: 	sim...@li....



Hello. Are you considering adding line numbers to the report of failed 
tests?
Zbynek

-- 
http://zw.matfyz.cz/     http://robotika.cz/
Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic

[Simpletest-support] Line numbers for failed tests

[Zbynek W. <zw...@us...>]

Hello. Are you considering adding line numbers to the report of failed 
tests?
Zbynek

-- 
http://zw.matfyz.cz/     http://robotika.cz/
Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic

[Simpletest-support] Beta3 release

[Marcus B. <ma...@la...>]

Hi...

Not had as much time as I would have liked to get new releases out. This 
is not an essential release, feature wise, but is if you are extending 
the tool or thinking of extending the tool. For this various hooks have 
been added to a cleaner runner architecture. It is possible to do dry 
runs of the test suite now and to run tests on remote servers. I have 
also added XML output and also a parser to convert it back into test events.

yours, Marcus
-- 
Marcus Baker, ma...@la..., no...@ap...