simpletest-support — Help, advice, bugs and workarounds.

[Simpletest-support] Web Testing with SimpleTest

[Gabor S. <sz...@gm...>]

Thanks for the answers so far.

I published another article, now starting some WebTesting.

The newsletter is here:
http://mail.szabgab.com/mailman/listinfo/test-automation-tips

The blog entry is here:

http://szabgab.com/blog/2009/03/1237128967.html

Your comments are welcome!

Gabor


-- 
Gabor Szabo                     http://szabgab.com/blog.html
Test Automation Tips        http://szabgab.com/test_automation_tips.html

Re: [Simpletest-support] helper asserts assertGet assertClick

[Marcus B. <ma...@wo...>]

Hi...

Gabor Szabo wrote:
>     	$this->assertTrue($this->get($url));
> 	$this->assertTrue($this->click('Add'));
> 
> are there helper method such as assertGet and assertClick that will replace
> the need for the above or are there plans to add those?

You don't really need this, as simply asserting text on the next page in
the first test should be enough. These rarely fail, and if they do the
problem is not with your code, but with the infrastructure. Write a
separate test for infrastructure (a short one), run that first and then
assume everything works until proven otherwise.

As for changing this in the future, failures will throw exceptions. This
way, if you want to emulate a failure...

$this->expectException('NotClickable');
$this->click('Not shown on this page');

This is all stuff for the PHP only rewrite that will probably take all year.

> 
> Gabor

yours, Marcus

[Simpletest-support] helper asserts assertGet assertClick

[Gabor S. <sz...@gm...>]

Now I need to write

    	$this->assertTrue($this->get($url));
	$this->assertTrue($this->click('Add'));

are there helper method such as assertGet and assertClick that will replace
the need for the above or are there plans to add those?

Gabor

Re: [Simpletest-support] TextReporter and HTMLReporter

[Marcus B. <ma...@wo...>]

Hi...

Gabor Szabo wrote:
> Anyway I think this is the reason way I could not subclass the SimpleReporter
> only either the HTMLReporer or the TestReporter.

Aha...yes! When I get to "R" I'll fix that.

> 
> Gabor

yours, Marcus

Re: [Simpletest-support] submitting a form

[Marcus B. <ma...@wo...>]

Hi...

Gabor Szabo wrote:
> Once I 'get' a page, how can I fill the fields in a form?

$this->setField('life', 42);
$this->setField('pi', 3.14);
$this->click('Submit');

> Gabor
> 

yours, Marcus

Re: [Simpletest-support] assertField but in which form?

[Marcus B. <ma...@wo...>]

Hi...

Gabor Szabo wrote:
> So if I want to check if a field called xyz exists and has a value of 42.
> The success won't let me know in which form was that input field.

This is a known issue and is a performance problem with a PHP4
compatible version. The PHP5 version can look for the tidy extension to
speed things up, and so can do a lot more parsing.

A future version of SimpleTest will recognise a method called within()
used in two ways...

$this->within('#my-form')->assertField('life', 42);
$this->within('#my-form')->assertField('pi', 3.14);
$this->assertTitle('stuff');

...or...

$this->within('#my-form')
$this->assertField('life', 42);
$this->assertField('pi', 3.14);
$this->withinAll();     // or $this->within('*');
$this->assertTitle('stuff');

These would be equivalent. As will be this...

$this->assertField(new CssSelector('#life'), 42);
$this->within('#my-form')->assertField('#life', 42);
$this->assertField('#my-form #life', 42);

You may have to wait for this. I haven't written the CSS parser yet!

> 
> regards
>    Gabor

yours, Marcus

Re: [Simpletest-support] TextReporter and HTMLReporter

[Gabor S. <sz...@gm...>]

On Sun, Mar 15, 2009 at 1:27 PM, Marcus Baker <ma...@wo...> wrote:
> Hi...
>
> Gabor Szabo wrote:
>>
>> I managed to extend both TextReporter and HTMLReporter to show the passes
>> as the example on http://simpletest.org/en/reporter_documentation.html shows
>> but The TextReporer extension was only taking effect when I ran on the command
>> line while the HTML::Reporter extension worked only in the browser.
>>
>> Is this a feature a bug or an incorrect usage on my side?
>
> It is supposed to work like that. Right now there is no way to override
> it to have the TextReporter in the browser. I didn't think it would be
> necessary. Do you need this?

Maybe for debugging purposes but I am not sure.
Anyway I think this is the reason way I could not subclass the SimpleReporter
only either the HTMLReporer or the TestReporter.

Gabor

Re: [Simpletest-support] TextReporter and HTMLReporter

[Marcus B. <ma...@wo...>]

Hi...

Gabor Szabo wrote:
> So I wondered when would I need to write
> 
> <?php
> require_once('simpletest/autorun.php');
> SimpleTest :: prefer(new TextReporter());
> require_once('../classes/writer.php');
> 
> class FileTestCase extends UnitTestCase {
>     ...
> }
> ?>

You don't anymore. I'm in the process of correcting all the docs prior
to the SimpleTest 1.1 release. I'm working through in alphabetical order
and I'm up to "M". It will take another couple of weeks at current rate.

> I tried to add that code and access the page via the browser but I still
> got the HTML Report.

The autorun mechanism has three modes (one triggered by the --xml flag).
I couldn't see a need to have text mode in a browser or vice versa, so
that's not currently supported.

The prefer() system (?!) works on subclasses of HtmlRporter,
TextReporter or XmlReporter and sniffs the perefered class to determine
which on it subclasses. Right now it's a bit of a beta quality hack.
Later it will look for interfaces as well.

> 
> I managed to extend both TextReporter and HTMLReporter to show the passes
> as the example on http://simpletest.org/en/reporter_documentation.html shows
> but The TextReporer extension was only taking effect when I ran on the command
> line while the HTML::Reporter extension worked only in the browser.
> 
> Is this a feature a bug or an incorrect usage on my side?

It is supposed to work like that. Right now there is no way to override
it to have the TextReporter in the browser. I didn't think it would be
necessary. Do you need this?

> 
> Gabor

yours, Marcus

[Simpletest-support] submitting a form

[Gabor S. <sz...@gm...>]

Last question for today, I hope.

Once I 'get' a page, how can I fill the fields in a form?

I could not find a way so I decided I'll use ->get again for now
and pass the parameters but I'd prefer to fill the form values if that's
possible.

Gabor

-- 
Gabor Szabo                     http://szabgab.com/blog.html
Test Automation Tips        http://szabgab.com/test_automation_tips.html

[Simpletest-support] assertField but in which form?

[Gabor S. <sz...@gm...>]

Looking at the WebTestCase I see the assertField but it does not
distinguish between two forms on the same page.

So if I want to check if a field called xyz exists and has a value of 42.
The success won't let me know in which form was that input field.

regards
   Gabor
-- 
Gabor Szabo                     http://szabgab.com/blog.html
Test Automation Tips        http://szabgab.com/test_automation_tips.html

[Simpletest-support] TextReporter and HTMLReporter

[Gabor S. <sz...@gm...>]

I was trying to understand how the switching between TextReport and
HTMLReport work
and I could not get it work so far.

When I access test.php in the browser I get the nice report with the
red line, it is clear it sent me
HTML.
When I run the   $ php test.php I get a textual report. So autorun
recognized that I am running it on the
command line and automatically switched to the TextReporter. This is nice.

So I wondered when would I need to write

<?php
require_once('simpletest/autorun.php');
SimpleTest :: prefer(new TextReporter());
require_once('../classes/writer.php');

class FileTestCase extends UnitTestCase {
    ...
}
?>


mentioned at the bottom of
http://simpletest.org/en/unit_test_documentation.html  ?

I tried to add that code and access the page via the browser but I still
got the HTML Report.

I managed to extend both TextReporter and HTMLReporter to show the passes
as the example on http://simpletest.org/en/reporter_documentation.html shows
but The TextReporer extension was only taking effect when I ran on the command
line while the HTML::Reporter extension worked only in the browser.

Is this a feature a bug or an incorrect usage on my side?

Gabor

Re: [Simpletest-support] Documentation wishlist

[tarjei <ta...@nu...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Marcus Baker wrote:
> Hi...
> 
> tarjei wrote:
>> Hi, I know that pleas for better docs often fail to impress, but there
>> is one thing I would love to have: A simple table over all the assert
>> interfaces in simpletest and also another table with the different
>> methods for using mocks.
> 
> OK.
> 
> How about a tail end summary page? We can place the instructions on 
> how to contribute there as well.
> 
That would fit perfectly.

regards,
Tarjei


>> Kind regards,
>> Tarjei
> 
> yours, Marcus
> 
> ------------------------------------------------------------------------------
> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> _______________________________________________
> Simpletest-support mailing list
> Sim...@li...
> https://lists.sourceforge.net/lists/listinfo/simpletest-support

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJuPFjYVRKCnSvzfIRAplGAKCocu6vA4oOo/mSPVMvBhJMj8JcBwCfflOA
dcYnzC2zGqTwy3H/d7vKJfg=
=XPBp
-----END PGP SIGNATURE-----

Re: [Simpletest-support] Documentation wishlist

[Marcus B. <ma...@wo...>]

Hi...

tarjei wrote:
> Hi, I know that pleas for better docs often fail to impress, but there
> is one thing I would love to have: A simple table over all the assert
> interfaces in simpletest and also another table with the different
> methods for using mocks.

OK.

How about a tail end summary page? We can place the instructions on 
how to contribute there as well.

> Kind regards,
> Tarjei

yours, Marcus

[Simpletest-support] Documentation wishlist

[tarjei <ta...@nu...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi, I know that pleas for better docs often fail to impress, but there
is one thing I would love to have: A simple table over all the assert
interfaces in simpletest and also another table with the different
methods for using mocks.

The docs today are great to get started with, but when I want to dig out
 a spesific mock method I often end up reading the source.

Kind regards,
Tarjei
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJuCypYVRKCnSvzfIRAsv0AJ4pCFjPTv3Cy3U+UIEG2j6Jk3yUmgCgl4ue
LAd5IDbROMEXgtq5DwVSKgA=
=6TAR
-----END PGP SIGNATURE-----

Re: [Simpletest-support] What happens if I quit testing too soon? Can I declare a plan?

[Marcus B. <ma...@wo...>]

Hi...

Gabor Szabo wrote:
> Is there a way to declare up front that I'll have 100 calls to some
> assert() and ensure
> that I get failure report if I don't actually deliver the 100 pass-es?

No, but you wouldn't normally have 100 asserts in a single test. Usually
you have lot's of small test methods. You can run those individually...

php my_tests.php --test=testJustThisOnePlease

...or in the browser...

http://localhost/my_tests.php?t=testJustThisOnePlease

> 
> Gabor

yours, Marcus

Re: [Simpletest-support] Testing PHP code with SimpleTest

[Marcus B. <ma...@wo...>]

Hi...

Gabor Szabo wrote:
> I hope you'll keep the old version around as I am quite sure there are tons of
> legacy applications that would need testing and that won't upgrade PHP.

We will. We also maintain a lot of backward compatibility across versions.

> Hmm, what hole are you talking about?

The error message doesn't quite read, because just before the "at [...]"
it should have a message.

> For what I understood assertTrue() as all the other assertMethods() will
> accept an *optional* extra parameter as a messages. Isn't that the case?

It should be, but still isn't. The docs are wrong, although it's
certainly not mission critical :).

> Sounds cool but let me first try to write what I understand and then you
> write the expert version. If that's ok with you.

Absolutely fine.

> Again, I think you should also keep the docs for the older versions as many
> people won't be able to use the new one. At least not till they write tons
> of tests so they can risk the upgrade.

It should be a drop in upgrade. the upgrade path would be (a) upgrade to
PHP5 (using SimpleTest 1.0.1 to keep their tests running) (b) Upgrade to
SimpleTest 1.1, which should just drop in. (c) Turn on E_STRICT and then
fix all the PHP4'isms in their code base.

> I am now trying to subclass the reporting class, but I don't understand how
> can I tell autorun to use my ShowPasses class (copied from the docs).

This is all pretty new, and wasn't doc'd at the time of the 1.0.1
release as it was still considered beta. It will be for the 1.0.2
version (that will come out shortly after the beta of 1.1).

<?php
require_once('simpletest/autorun.php');
SimpleTest::prefer(new MyReporter());

class MyTests extends TestSuite { ...}
?>

> Oh and the docs say to require
> simple_test.php while I think it should be simpletest.php

That's just a screw up :(. I'll add it to my doc fixes.

> 
> Gabor

yours, Marcus

[Simpletest-support] What happens if I quit testing too soon? Can I declare a plan?

[Gabor S. <sz...@gm...>]

Let's say you have 100 calls in the testAdd function.

Number 3 fails.
While you are debugging your code you don't want to rerun all the 100
every time.

What do you do?

You might add a return() just after the 3rd call to avoid executing the rest
of the tests.

Once you fix the code you *should* remove the return() call but as everything
is green now and you are in a hurry you can forget.
>From that time on only 3 tests will run and you effectively lost the
other 97 tests.

Is there a way to declare up front that I'll have 100 calls to some
assert() and ensure
that I get failure report if I don't actually deliver the 100 pass-es?

Gabor
http://szabgab.com/test_automation_tips.html

Re: [Simpletest-support] Testing PHP code with SimpleTest

[Gabor S. <sz...@gm...>]

On Wed, Mar 11, 2009 at 2:11 PM, Marcus Baker <ma...@wo...> wrote:
> Hi...
>
> Gabor Szabo wrote:
>> I'd be glad to receive your comments about it.
>
> 1) You got all the autorun stuff in, which is cool! You probably
> need to point out that the 1.0.1 version of SimpleTest does not work
> with the E_STRICT setting. It does work with PHP 4.2.2 to PHP 5.3
> though. You would need the 1.1 version which is PHP 5.0.3+ to PHP 6
> to be compatible with E_STRICT.

I hope you'll keep the old version around as I am quite sure there are tons of
legacy applications that would need testing and that won't upgrade PHP.


> 2) It's not explained in the docs, but assertTrue() needs a message
> as a second parameter, hence the hole where one should be. It's a
> legacy restriction which will disappear now that you've reminded me
> of it :(.
>
> Otherwise cool!

Hmm, what hole are you talking about?
For what I understood assertTrue() as all the other assertMethods() will
accept an *optional* extra parameter as a messages. Isn't that the case?


> Need a guest article on the web tester (which is the biggest feature
> of SimpleTest)?

Sounds cool but let me first try to write what I understand and then you
write the expert version. If that's ok with you.


>> I plan to add more of these but I am sure I'll also have
>> questions as I am not yet familiar with SimpleTest.
>
> Fair enough. I'm right in the middle of updating the docs for a
> version 1.1 release. Hopefully it will be out by the time you do
> your next post.

Again, I think you should also keep the docs for the older versions as many
people won't be able to use the new one. At least not till they write tons
of tests so they can risk the upgrade.

I am now trying to subclass the reporting class, but I don't understand how
can I tell autorun to use my ShowPasses class (copied from the docs).


Oh and the docs say to require
simple_test.php while I think it should be simpletest.php

Gabor

Re: [Simpletest-support] Testing PHP code with SimpleTest

[Marcus B. <ma...@wo...>]

Hi...

Gabor Szabo wrote:
> I'd be glad to receive your comments about it.

1) You got all the autorun stuff in, which is cool! You probably 
need to point out that the 1.0.1 version of SimpleTest does not work 
with the E_STRICT setting. It does work with PHP 4.2.2 to PHP 5.3 
though. You would need the 1.1 version which is PHP 5.0.3+ to PHP 6 
to be compatible with E_STRICT.

2) It's not explained in the docs, but assertTrue() needs a message 
as a second parameter, hence the hole where one should be. It's a 
legacy restriction which will disappear now that you've reminded me 
of it :(.

Otherwise cool!

Need a guest article on the web tester (which is the biggest feature 
of SimpleTest)?

> 
> I plan to add more of these but I am sure I'll also have
> questions as I am not yet familiar with SimpleTest.

Fair enough. I'm right in the middle of updating the docs for a 
version 1.1 release. Hopefully it will be out by the time you do 
your next post.

> 
> regards
>   Gabor

yours, Marcus

[Simpletest-support] Testing PHP code with SimpleTest

[Gabor S. <sz...@gm...>]

hi,

I have been teaching test automation using Perl for several years now.
Recently I decided to add a few chapters on testing PHP applications.
As I am writing my slides I am also including part of the examples in
a newsletter called Test Automation Tips.

    http://szabgab.com/test_automation_tips.html

The messages are also posted on my blog
http://szabgab.com/blog.html

The the first entry in the PHP SimpleTest related post is here:

   http://szabgab.com/blog/2009/03/1236753971.html

I'd be glad to receive your comments about it.

I plan to add more of these but I am sure I'll also have
questions as I am not yet familiar with SimpleTest.

regards
  Gabor

[Simpletest-support] Testing 1.0.2beta: Getting reflection errors

[tarjei <ta...@nu...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi, I tried to upgrade to 1.0.2beta and found some errors.

The following testcase fails:
<?php
set_include_path(
    dirname(__FILE__).'/..'.
    PATH_SEPARATOR.
    get_include_path());
require_once 'simpletest/unit_tester.php';
require_once 'simpletest/reporter.php';
require_once 'simpletest/mock_objects.php';

require_once 'PHPTAL.php';
class TestCase extends UnitTestCase {

    function setUp() {

    }

    function test_mock_phptal() {
        Mock::Generate('PHPTAL', 'MockView');
    }

}

    $test = new TestSuite();
    $test->add(new TestCase());
    $reporter = new TextReporter();
    $test->run($reporter);


?>

With this output:

TestCase
Exception 1!
Unexpected exception of type [ReflectionException] with message [Class
PHPTAL_TranslationService does not exist] in
[/home/tarjei/Viddi/Viddi/php5/vendor/simpletest/reflection_php5.php
line 342]
	in test_mock_phptal
	in TestCase
FAILURES!!!
Test cases run: 1/1, Passes: 0, Failures: 0, Exceptions: 1

You can fetch PHPTAL from http://phptal.motion-twin.com/

What is wrong here?

Also reported as:
https://sourceforge.net/tracker/index.php?func=detail&aid=2660515&group_id=76550&atid=547455
Kind regards,
Tarjei
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJrjMrYVRKCnSvzfIRAkG6AJ4zaF2bXTfnx7/S5OKMKn26lbdj3gCeOp29
BolzCWZmF2+hJatFziRSK4c=
=GFe9
-----END PGP SIGNATURE-----

Re: [Simpletest-support] Code coverge instructions

[Douglas H. <do...@hu...>]

Marcus Baker <marcus@...> writes:
> Douglas Hubler wrote:
> > but we can reuse this for simpletest internal documentation
> Cool. Can I just clone the HTML, restyling (via XSLT) as necessary?

sure or harvesting tidbits

Re: [Simpletest-support] Code coverge instructions

[Marcus B. <ma...@wo...>]

Hi...

Douglas Hubler wrote:
> but we can reuse this for simpletest internal documentation

Cool. Can I just clone the HTML, restyling (via XSLT) as necessary?

yours, Marcus

[Simpletest-support] Code coverge instructions

[Douglas H. <do...@hu...>]

I posted this instructions for generating code coverage for a project
called Drupal

http://acquia.com/blog/calculating-test-coverage

but we can reuse this for simpletest internal documentation

Re: [Simpletest-support] SimpleTest 1.1beta

[Perrick P. <pe...@no...>]

Are you ready now the UK conference is over ??

Yours,
	Perrick


Marcus Baker wrote:
> Hi...
> 
> Perrick Penet wrote:
>> It means we can now roll out ST 1.1beta ! Marcus, you have my green light.
> 
> I have to beg internet connections at the moment until I get broadband
> hooked up. I'll wade into the docs offline, and then do a release.
> 
>> 	Perrick
> 
> yours, Marcus
> 
> p.s. You coming to the conference on the 27th? http://phpconference.co.uk
> 
> yours, Marcus
> 
> ------------------------------------------------------------------------------
> Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
> software. With Adobe AIR, Ajax developers can use existing skills and code to
> build responsive, highly engaging applications that combine the power of local
> resources and data with the reach of the web. Download the Adobe AIR SDK and
> Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
> _______________________________________________
> Simpletest-support mailing list
> Sim...@li...
> https://lists.sourceforge.net/lists/listinfo/simpletest-support
> 
>