simpletest-support — Help, advice, bugs and workarounds.

Re: [Simpletest-support] SimpleTest 1.1beta

[Marcus B. <ma...@la...>]

Hi...

Douglas Hubler wrote:
> Please let's put code coverage in.

It probably will be (in the extensions folder). The main thing is for me
to finish the current round of doc updates. The docs are in a bit of a
half and half state right now.

yours, Marcus
-- 
Marcus Baker
ma...@la...

Re: [Simpletest-support] SimpleTest 1.1beta

[Douglas H. <dh...@gm...>]

Please let's put code coverage in.

Re: [Simpletest-support] adding HTTP PUT to web tester

[Mark R. <co...@gm...>]

I must have been mixed up then... for some reason I was certain that
PUT was defined in the 1.1 RFC.

If that's the case, then we *can* use the new PUT and entity body
support. Sorry for the confusion.

Re: [Simpletest-support] adding HTTP PUT to web tester

[Marcus B. <ma...@wo...>]

Hi...

Mark Kimsal wrote:
> If we really want to be only HTTP/1.0, then we should look at removing the use 
> of "Host" header too.  Then simpletest would be truly http/1.0 and work on 
> almost no Web servers.

We don't want to be HTTP 1.0 for the sake of it, and don't aim to be 
terribly strict. We aren't testing web servers, but PHP code. Our 
aim is to get through the web server with the least amount of 
effort. Full HTTP 1.1 support with multiple connections and chunked 
encoding is quite a bit of work.

yours, Marcus

Re: [Simpletest-support] Question on backwards compatibility and testing web output

[Perrick P. <pe...@no...>]

Hi,

Sorry about missing the heat this week-end (I was getting engaged ;-)

> I made a half hearted effort to centralise the paths at least, but never
> got as far as adding the switch. This would be really useful and it
> would be great if you could add it :).

I remember asking for this some time ago. Right now, there's a section 
in acceptance_test.php that I need to edit when testing live on my own 
dev machine :

     static function samples() {
         return 'http://www.lastcraft.com/test/';
     }

Now how do we want to add the switch ? Via a configuration *.ini file 
somewhere for dev. users to user or something else. Maybe we can just 
change the setting based on the VERSION file.


> Even better if it could default to simpletest.org/acceptance/VERSION
> where "VERSION" is the version file contents in the tarball. A procedure
> for adding acceptance tests that all developers could use would clear a
> few roadblocks. Talk to Perrick as I think he already had some ideas.

For the documentation, I'd be ready to roll out the proposed directories 
on the server. That would be easy enough !

Yours,
	Perrick

Re: [Simpletest-support] adding HTTP PUT to web tester

[Mark K. <ma...@me...>]

On Monday 22 December 08, Marcus Baker wrote:
> Hi...
>
> Mark Rickerby wrote:
> > Hi, just a small comment on:
> > http://sourceforge.net/tracker2/?func=detail&aid=1578925&group_id=76550&a
> >tid=547457
>
> Cool.
>
> > The reason why I didn't follow up on this originally and fold this
> > code into trunk is because PUT support is an aspect of HTTP 1.1, which
> > involves a few more things than just this one feature, and I didn't
> > want to make the client pretend that it was 1.1, when it really
> > wasn't.
>
> Is there no PUT in HTTP 1.0? Can you do a 1.0 compliant PUT?
>
> Making SimpleTest HTTP 1.1 compliant is a big undertaking and doesn't
> really seem necessary right now. The growth of REST and non HTML
> resources will probably change this, but it's certainly not a feature
> for the 1.1 version of SimpleTest.

There is most definitely PUT in HTTP/1.0

http://ftp.ics.uci.edu/pub/ietf/http/rfc1945.html#PUT

HTTP/1.1 really just made Host mandatory, addressed some caching issues, added 
pipe-lining and chunked transfer encodings.  I think it added keep-alive as 
well.

If we really want to be only HTTP/1.0, then we should look at removing the use 
of "Host" header too.  Then simpletest would be truly http/1.0 and work on 
almost no Web servers.

> yours, Marcus
>
> ---------------------------------------------------------------------------
>--- _______________________________________________
> Simpletest-support mailing list
> Sim...@li...
> https://lists.sourceforge.net/lists/listinfo/simpletest-support

Re: [Simpletest-support] adding HTTP PUT to web tester

[Marcus B. <ma...@wo...>]

Hi...

Mark Rickerby wrote:
> Hi, just a small comment on:
> http://sourceforge.net/tracker2/?func=detail&aid=1578925&group_id=76550&atid=547457

Cool.

> 
> The reason why I didn't follow up on this originally and fold this
> code into trunk is because PUT support is an aspect of HTTP 1.1, which
> involves a few more things than just this one feature, and I didn't
> want to make the client pretend that it was 1.1, when it really
> wasn't.

Is there no PUT in HTTP 1.0? Can you do a 1.0 compliant PUT?

Making SimpleTest HTTP 1.1 compliant is a big undertaking and doesn't
really seem necessary right now. The growth of REST and non HTML
resources will probably change this, but it's certainly not a feature
for the 1.1 version of SimpleTest.

yours, Marcus

[Simpletest-support] adding HTTP PUT to web tester

[Mark R. <ma...@co...>]

Hi, just a small comment on:
http://sourceforge.net/tracker2/?func=detail&aid=1578925&group_id=76550&atid=547457

The reason why I didn't follow up on this originally and fold this
code into trunk is because PUT support is an aspect of HTTP 1.1, which
involves a few more things than just this one feature, and I didn't
want to make the client pretend that it was 1.1, when it really
wasn't.

Re: [Simpletest-support] SimpleTest 1.1beta

[Marcus B. <ma...@wo...>]

Hi...

Edward Z. Yang wrote:
> Aaand, done. Do we have our release manager around?

That's me. The docs have to be fixed either way. Basically the job
involves changing all the instructions to use autorun and replacing
deprecated stuff with newer stuff. Some of the examples have to be fixed
too. It's tedious, but undemanding work. I was doing it on the tube.

> 
> Cheers,
> Edward
> 

yours, Marcus

[Simpletest-support] Autorun class regex hack

[Edward Z. Y. <edw...@th...>]

Hello Pavel,

I've been doing some magic to turn test data files (very common when
testing complex algorithms) into first-class test suites. Something that
came up was that test suites I generated with eval() did not get slurped
by autorun. I took a look at autorun.php and this was because autorun
ran a regex on the original file to see whether or not a newly declared
class was found.

It seems to me that this check is unnecessary. If aggregate tests are
declared with addFile() (I don't see why they wouldn't be), deeper test
files are included after autorun has calculated suitable candidates.
Furthermore, we could implement aggregate tests simply by *including*
the other test files (no need to create all those classes).

But the main reason is because I find it bothersome to work around. :-)
I've removed it for now, but please revert me if you decide that it is
still a good idea.

Cheers,
Edward

P.S. I've CC'ed simpletest-support.

Re: [Simpletest-support] SimpleTest 1.1beta

[Edward Z. Y. <edw...@th...>]

Marcus Baker wrote:
> We should also issue a 1.0.2 release as well. [snip]

Aaand, done. Do we have our release manager around?

Cheers,
Edward

Re: [Simpletest-support] SimpleTest 1.1beta

[Edward Z. Y. <edw...@th...>]

Marcus Baker wrote:
> Agree totally. I have the doc sort out gradually working it's way
> through my queue (currently pegged for the Xmas holiday). It's really
> the docs that would cause all the problems.

Yeah, sounds like it might be. If there are any specific tasks I can
help on, please say so.

> We should also issue a 1.0.2 release as well. I had to introduce a
> couple of extra methods, assertSame() and assertCopy(), due to problems
> with the ampersands in assertReference() when porting test suites
> (especially PHP4) from 1.0.1 to 1.1 versions. [snip]

I'm going to branch the 1.0.1 tag into a 1.0 branch, and then add the
appropriate methods. However, since 1.1 breaks a lot of
"not-supposed-to-be-used" flex points, I expect people are going to have
to fix things on 1.1 sans E_STRICT anyway.

Cheers,
Edward

Re: [Simpletest-support] SimpleTest 1.1beta

[Marcus B. <ma...@wo...>]

Hi...

Edward Z. Yang wrote:
> What do you all think?

Agree totally. I have the doc sort out gradually working it's way
through my queue (currently pegged for the Xmas holiday). It's really
the docs that would cause all the problems.

We should also issue a 1.0.2 release as well. I had to introduce a
couple of extra methods, assertSame() and assertCopy(), due to problems
with the ampersands in assertReference() when porting test suites
(especially PHP4) from 1.0.1 to 1.1 versions. Ideally the 1.1 is a drop
in replacement for 1.0.x so that they can immediately turn on E_STRICT
straight afterwoods. Otherwise they are rewriting tests while trying to
convert their own code to E_STRICT. Not an ideal situation.

The 1.0.2 release is trivial (uses the same docs) and just means adding
the new methods. That way they can go back to 1.0.2, rewrite the tests
without assertReference(), then go forward to 1.1 again. Then, any
E_STRICT errors are theirs alone.

> 
> Cheers,
> Edward

yours, Marcus

Re: [Simpletest-support] "Deprecated" functions

[Edward Z. Y. <edw...@th...>]

Marcus Baker wrote:
> Which is why they are still there :).

Heh. :-)

> They are "deprecated" not as a note to us, but to others outside the
> core that extend the tool. Eventually they will be refactored out of
> existance.

It occurs to me, then, that it might be a good idea to throw some sort
of userland error so those people who don't actually source-dive will
get a warning about this behavior (of course, that means our code that
uses it will also throw errors; not sure what do to do here--maybe an
optional parameter that turns off the error? An isomorphic function that
doesn't have the error?)

> If we move over to one failure per test (as per most of the xUnit
> systems) then an explicit pass() is pointless. A pass is just the
> absence of failures in a test method. We are a long way from removing it
> of course, but we have to signal it's demise early, as it's so entrenched.

Ok, so this is a long term goal.

> Does that make sense?

Yep. Does that mean we should consider the "Remove deprecated functions"
task done for now?

Cheers,
Edward

Re: [Simpletest-support] "Deprecated" functions

[Marcus B. <ma...@wo...>]

Hi...

Edward Z. Yang wrote:
> I commented them out and got fatal errors on the test suite.

Which is why they are still there :).

They are "deprecated" not as a note to us, but to others outside the
core that extend the tool. Eventually they will be refactored out of
existance.

> SimpleTestCase->pass() being deprecated is especially surprising, since
> this is the only we can render passes in our test suite, which can be
> useful when implementing a --verbose mode to check for hanging, etc.

If we move over to one failure per test (as per most of the xUnit
systems) then an explicit pass() is pointless. A pass is just the
absence of failures in a test method. We are a long way from removing it
of course, but we have to signal it's demise early, as it's so entrenched.

Does that make sense?

> 
> Cheers,
> Edward

yours, Marcus

[Simpletest-support] SimpleTest 1.1beta

[Edward Z. Y. <edw...@th...>]

As per the TODO list, we are just about done as soon as we get some
closure on the undefined property bug (which seems to be extremely
edge-case-y and easily worked around) and the deprecated function
removal (which I have addressed in another email). Therefore, I would
like to get the ball rolling on a (long overdue) SimpleTest 1.1beta
release. What do you all think?

Cheers,
Edward

[Simpletest-support] "Deprecated" functions

[Edward Z. Y. <edw...@th...>]

Since one of the items on the TODO list is "remove deprecated
functions", I did a quick grep and found that the following functions
were marked deprecated:

SimpleTestCase->pass()
SimpleTest::setMockBaseClass()
SimpleTest::getMockBaseClass()

I commented them out and got fatal errors on the test suite. It seems to
me that the functionality these methods provide is necessary, and we'll
need suitable alternatives if we want to get rid of them.
SimpleTestCase->pass() being deprecated is especially surprising, since
this is the only we can render passes in our test suite, which can be
useful when implementing a --verbose mode to check for hanging, etc.

Cheers,
Edward

Re: [Simpletest-support] Removing trailing ?>

[Marcus B. <ma...@wo...>]

Hi...

Edward Z. Yang wrote:
> Does anyone have any objections to removing trailing ?>s?

No. As long as they are taken out everywhere.

yours, Marcus

Re: [Simpletest-support] Question on backwards compatibility and testing web output

[Marcus B. <ma...@wo...>]

Hi...

Edward Z. Yang wrote:
> While we're on that note, right now the acceptance tests result in
> requests that travel across the network. This is great for "unpack and
> run anywhere" mentality, but not so great if you're trying to locally
> modify the acceptance tests, etc. So I've been wondering if we should
> introduce some way of setting the location manually, like through an
> environment variable, or something.

I made a half hearted effort to centralise the paths at least, but never
got as far as adding the switch. This would be really useful and it
would be great if you could add it :).

Even better if it could default to simpletest.org/acceptance/VERSION
where "VERSION" is the version file contents in the tarball. A procedure
for adding acceptance tests that all developers could use would clear a
few roadblocks. Talk to Perrick as I think he already had some ideas.

> 
> Cheers,
> Edward

yours, Marcus

[Simpletest-support] Removing trailing ?>

[Edward Z. Y. <edw...@th...>]

Does anyone have any objections to removing trailing ?>s?

Re: [Simpletest-support] Question on backwards compatibility and testing web output

[Edward Z. Y. <edw...@th...>]

Edward Z. Yang wrote:
> Latest SVN checkout shows acceptance tests pointing to lastcraft.com. In
> fact, two of the acceptance tests are broken. :-/

Scrap that, I just fixed them. :-)

While we're on that note, right now the acceptance tests result in
requests that travel across the network. This is great for "unpack and
run anywhere" mentality, but not so great if you're trying to locally
modify the acceptance tests, etc. So I've been wondering if we should
introduce some way of setting the location manually, like through an
environment variable, or something.

Cheers,
Edward

Re: [Simpletest-support] Question on backwards compatibility and testing web output

[Edward Z. Y. <edw...@th...>]

Marcus Baker wrote:
> Did we switch the acceptance tests to point at simpletest.org? I 
> can't remember. I still get test traffic to Lastcraft, but this may 
> be older versions.

Latest SVN checkout shows acceptance tests pointing to lastcraft.com. In
fact, two of the acceptance tests are broken. :-/

Cheers,
Edward

Re: [Simpletest-support] Question on backwards compatibility and testing web output

[Marcus B. <ma...@wo...>]

Hi...

Perrick Penet wrote:
>> Also, I have a test case that will show that no data has been  
> That would be to me ;-)

Did we switch the acceptance tests to point at simpletest.org? I 
can't remember. I still get test traffic to Lastcraft, but this may 
be older versions.

> 
> Yours,
> 	Perrick

yours, Marcus

Re: [Simpletest-support] Question on backwards compatibility and testing web output

[Perrick P. <pe...@no...>]

> Also, I have a test case that will show that no data has been  
> submitted, but it requires a simple POST variable.  Who do I need to  
> talk to do get some code added to simpletest.org so I can use that for  
> the test?

That would be to me ;-)

Yours,
	Perrick

Re: [Simpletest-support] Question on backwards compatibility and testing web output

[Travis S. <dev...@do...>]

On Dec 17, 2008, at 4:22 PM, Marcus Baker wrote:

>> Also, I have a test case that will show that no data has been
>> submitted, but it requires a simple POST variable.  Who do I need to
>> talk to do get some code added to simpletest.org so I can use that  
>> for
>> the test?
>
> I don't understand the problem?

The test case is for the buggy implementation.  It's easiest to test  
by creating a POST to a webserver and echo'ing out the data posted.   
When commas are used, no data is displayed; when semi-colons are used  
the data is displayed.

-T