Menu
▾
▴
#468 Test case fails even though all test steps are valid
A test case with multiple test steps fails in soapui-maven-plugin v3.0 although all test steps assertions are valid. Same test case is valid when run in soapUI eclipse plugin v2.5.1. Two of the test steps have a single script assertion each: "assert messageExchange.responseHeaders["#status#"].contains("HTTP/1.1 404 Not Found")" - for these test steps a response has status 404, empty content, and only headers are returned. soapui-maven-plugin prints in build log that these two assertions have been found to be valid but test case fails. If I disable these two test steps, and run soapui-maven-plugin again, test case becomes valid too. Thus, I believe that SoapUITestCaseRunner in soapui-maven-plugin v3.0 has problems with determining status of test cases with test steps that have script assertions.
I believe this is not only issue in soapui-maven-plugin v3.0, it depends on soapui libraries v3.0.1, and I guess fault in them is the root cause of the issue. That is why I'm setting category to be general.
JRE is 1.6 update 15. Platform is Windows XP SP3.
Hello,
Could you please attach the full output of running the tests? Thanks!
Regards,
Dain
eviware.com
To be more precise, test step with single script assertion fails even though assertion is (green and) marked as valid.
A WS service handles request based on QName of root element in soap body. If request is soap envelope with empty soap body service should return HTTP 404 error and test step verifies this expected behavior using script assertion ( assert messageExchange.responseHeaders["#status#"].contains("HTTP/1.1 404 Not Found") ) as I couldn't find any other assertion type able to verify response headers. In raw response output I can clearly see the headers and the script assertion is green but "Assertions (1)" label, and test step label are red. When using soapui maven plugin and running the same soapui project, build fails and soapui outputs the error:
"java.lang.Exception: TestCase [XY TestCase] failed without assertions"
even though test step with script assertion produced following info:
"[SoapUITestCaseRunner] Assertion [Script Assertion] has status VALID"
Seems soapui doesn't treat script assertion as assertion, it doesn't get counted as one and that somehow influences the test outcome.
This doesn't work with soapui 3.0 and 3.0.1, but if I recall well it worked ok with soapui 2.5. Currently I have such test steps on disabled. As soon as the step is disabled, build is successful, but lacks this check.
I've tried to reproduce this, but haven't been able to. Would it be possible for you to send us your project? You can send it to support@eviware.com .
Regards,
Dain
eviware.com
I'm suspecting that this occurs only when web service response is 404 error but with empty content (content-length: 0). This happens if web service is a web application project without registered 404 page and without application server serving it's default one - like on jetty (6.1.19) server. Trying to create appropriate project that would show the bug.
Example project which demonstrates the bug
Just attached example project which demonstrates the bug. It's a Maven2 Java project.
Bug was successfully reproduced on following platform:
Apache Maven 2.2.1
Java version: 1.6.0_16
Default locale: en_US, platform encoding: Cp1252
OS name: "windows xp" version: "5.1" arch: "x86" Family: "windows"
Thanks for your help in tracking this bug down, I've now confirmed and reproduced the bug, and I have committed a fix for it. The upcoming nightly build of soapUI 3.0.x should no longer have this problem. If you clear out your maven2 repository, the maven-plugin should also be fixed tomorrow. Please confirm that the fix works for you.
Regards,
Dain
eviware.com
This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 14 days (the time period specified by
the administrator of this Tracker).