From: rajdeep_dua <nu...@jb...> - 2005-03-17 12:56:24
|
i was able to run a simple jms api test client with this Properties props = new Properties(); | props.put("java.naming.factory.initial","org.jboss.messaging.util.StandaloneInitialContextFactory"); | ctx = new InitialContext(props); please let me know if my writing test cases for jms compliance is ok i am using test package -- org.jboss.test.messaging.jmsapi View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3870539#3870539 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3870539 |
From: <ovi...@jb...> - 2005-03-17 17:46:40
|
Thank you for your involvement. Yes, we are at the development stage when a JMS API client can be written, or at least, we're very close to that. Please look at the JMS Facade thread http://jboss.org/index.html?module=bb&op=viewtopic&t=61556, you'll see where we are. However, that client would have a very limited functionality. A lot of JMS API calls just throw NotYetImplementedException, so I am sure that at this stage, the client API won't pass any compliance test. The client you used, even if it is functional, it is deprecated, so I would suggest not to start testing anything on it, since we'll drop that code anyway. Having someone who specializes compliance tests, or other kind of testing for that matter, is essential, so your help is more than welcome. I hope we'll have something that will start passing compliance tests in several weeks. Until then, I would like you to start thinking at a more conprehensive testing and benchmarking harness that will grow toghether with the project. This is what I have in mind: We'll have four kinds of tests: - unit tests: written by the developers, at the same time or before writing the code itself. "Old" XP principle. They are essential, since they'll catch regression bugs and we'll allow us to refactor more freely. - compliance tests: your stuff. Ideally, I don't want developers involved with them, they should be performed by someone who is not closely involved with the code (or not yet), just because his/her only concern is to see how solid is the implementation from the JMS point of view. - integration test: test how well our JMS provider integrates with the AS; mainly the QA team's job. - stress tests and benchmarks: I hope to have a benchmarking framework that will allow us to closely follow key metrics for our implementation. As a very quick example, I am thinking of something like: how does our implementation behave when one hundred producers concurrently send 10KB messages? What are the key metrics to follow? How do these metrics evolve from a release to another? Does our implementation scale? Are you interested in getting involved with that? How did you think of doing compliance tests? SpecJ? View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3870617#3870617 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3870617 |
From: <ad...@jb...> - 2005-03-17 19:28:06
|
I'd recommend reworking what is already in the testsuite to work over both JBossMQ and JBoss Messaging: http://cvs.sourceforge.net/viewcvs.py/jboss/jbosstest/src/main/org/objectweb/jtests/ However, that is only jms1.0.2b level tests. Some of the JBossMQ tests are also pretty generic where they don't use internal apis for things like creating/clearing message queues, etc. http://cvs.sourceforge.net/viewcvs.py/jboss/jbosstest/src/main/org/jboss/test/jbossmq/ Since JBoss Inc. is a j2ee licensee we have access to the JMS 1.1. TCK, but only our employees can run it. It is still a good idea to have a thorough testsuite for the developers who are not JBoss Inc employees or don't work for another j2ee licensee. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3870650#3870650 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3870650 |
From: rajdeep_dua <nu...@jb...> - 2005-03-18 04:18:34
|
Adrian objectwebs test cases you are talking about - are they same as the joram test suite. i had intergrated it with pramat's jms server 3 years back. i think they have upgraded the test suite for 1.1 version also. we can integrated those test cases. My first test client failed yesterday connection.getClientID() ClientID is always the tricky part throws OperationNotsupportedException. will post the exact stact trace. For performance test there is a grinder framework available Sonic also has a test harness which is downloadable. It covers many permutations/combinations outlined above.I can start looking into that also. please let me know where should i start posting testcases? View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3870701#3870701 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3870701 |
From: rajdeep_dua <nu...@jb...> - 2005-03-18 04:20:39
|
For check on compliance i am following the specification itself. each api call has various senarios where things can bomb or atleast there is a risk! View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3870702#3870702 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3870702 |
From: <ovi...@jb...> - 2005-03-18 19:24:38
|
Try to build your compliance tests starting from what Adrian suggested http://cvs.sourceforge.net/viewcvs.py/jboss/jbosstest/src/main/org/objectweb/jtests/ and http://cvs.sourceforge.net/viewcvs.py/jboss/jbosstest/src/main/org/jboss/test/jbossmq/ No need to get into JBoss-specific API testing just yet. We'll do that later. If you could come up with a comprehensive JMS 1.1 test suite, that would be great. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3870779#3870779 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3870779 |
From: rajdeep_dua <nu...@jb...> - 2005-03-21 08:23:17
|
As a first step towards compliance with 1.1, i have integrated the latest testsuite from Joram which is for 1.1 compliance This i tested with JBoss 4.0 release code since the latest jms server is still not ready. There are total of 186 tests in this test suite, 2 failed and there were errors in 2. I had to modify org.objectweb.jtests.providers.admin.JBossMQAdmin Added te following methods | public void createConnectionFactory(String name) { | createQueueConnectionFactory(name); | } | | public void deleteConnectionFactory(String name) { | deleteQueueConnectionFactory(name); | } | Here is the test summary report commands, ant tests-objectweb-jms followed by ant tests-report-text in the testsuite module(note : i replaced the org.objectweb.* with the newer version from their CVS) JBoss daily test results SUMMARY Number of tests run: 186 -------------------------------------------- Successful tests: 180 Errors: 2 Failures: 4 -------------------------------------------- [time of test: 2005-03-21.08-16 GMT] [java.version: 1.4.2] [java.vendor: Sun Microsystems Inc.] [java.vm.version: 1.4.2-b28] [java.vm.name: Java HotSpot(TM) Client VM] [java.vm.info: mixed mode] [os.name: Windows XP] [os.arch: x86] [os.version: 5.1] Useful resources: - http://jboss.sourceforge.net/junit-results/32/2005-03-21.08-16 for the junit report of this test. NOTE: If there are any errors shown above - this mail is only highlighting them - it is NOT indicating that they are being looked at by anyone. It is assumed that whoever makes change(s) to jboss that break the test will be fixing the test or jboss, as appropriate! -------------------------------------------- DETAILS OF ERRORS Suite: org.objectweb.jtests.jms.conform.message.properties.JMSXPropertyTest Test: testSupportsJMSXDeliveryCount Type: failure Exception: junit.framework.AssertionFailedError Message: JMSXDeliveryCount property is not supported --------------------------------- Suite: org.objectweb.jtests.jms.conform.message.properties.JMSXPropertyTest Test: testJMSXDeliveryCount Type: failure Exception: junit.framework.AssertionFailedError Message: Message property 'JMSXDeliveryCount' not set. --------------------------------- Suite: org.objectweb.jtests.jms.conform.selector.SelectorSyntaxTest Test: testInvalidIdentifiersStart Type: error Exception: org.jboss.mq.selectors.TokenMgrError Message: Lexical error at line 1, column 1. Encountered: "%" (37), after : "" --------------------------------- Suite: org.objectweb.jtests.jms.conform.selector.SelectorTest Test: unknown Type: error Exception: junit.framework.AssertionFailedError Message: Timeout occurred --------------------------------- Suite: org.objectweb.jtests.jms.conform.session.QueueSessionTest Test: testRollbackRececeivedMessage Type: failure Exception: junit.framework.ComparisonFailure Message: expected:<...RollbackRececeivedMessage> but was:<...JMSDeliveryModeInSelector:1> --------------------------------- Suite: org.objectweb.jtests.jms.conform.session.UnifiedSessionTest Test: testCreateDurableConnectionConsumerOnQueueConnection Type: failure Exception: junit.framework.AssertionFailedError Message: Should throw a javax.jms.IllegalStateException --------------------------------- Please let me know how this can be checked in. I think we might have to modify JBossMQAdmin if the MBean APIs etc are changing for the serverless impl. Guys let me know your thoughts on this. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3870924#3870924 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3870924 |
From: <ad...@jb...> - 2005-03-21 17:06:31
|
"rajdeep_dua" wrote : | I think we might have to modify JBossMQAdmin if the MBean APIs etc are changing for the serverless impl. | | Guys let me know your thoughts on this. | The idea would be to make the admin class configurable by a system property. Then the tests could be run multiple times: 1) over JBossMQ 2) over serverless jms 3) over JBoss Messaging + JMS Facade 4) over JBoss + A.N.Other jms provider View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3871001#3871001 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3871001 |
From: <ad...@jb...> - 2005-03-21 17:08:24
|
rajdeep, if you send me your sourceforge id/email address we can see about getting you cvs r/w to integrate this work into the testsuite. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3871002#3871002 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3871002 |
From: rajdeep_dua <nu...@jb...> - 2005-03-22 12:53:09
|
Adrian, My sourceforge id is: rajdeep_dua My mail id is dua...@ya... View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3871138#3871138 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3871138 |
From: <ovi...@jb...> - 2005-03-24 02:58:06
|
Rajdeep, Please let me know when you start working on the test suite. I still don't recommend to apply it to the Mess (see http://www.jboss.org/index.html?module=bb&op=viewtopic&t=61556), but it's great if you get it ready to run on JBossMQ and you create the hooks for the Mess and other JMS provider. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3871433#3871433 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3871433 |
From: rajdeep_dua <nu...@jb...> - 2005-03-28 05:06:13
|
Hi Ovidiu, Started working on the test suite last week end. (Had some issues with compiling the jmx module,hence the delay) Here is the brief of the test suite structure Location ${jboss-head}/jms/tests Package : org.jboss.test.messaging.jmsapi org.jboss.test.messaging.jmsapi.connection org.jboss.test.messaging.jmsapi.flow org.jboss.test.messaging.jmsapi.message ... Actors All TestCases extend org.jboss.test.messaging.MessagingTestCase | Provider specfic Subclasses of Abstract class org.jboss.test.messaging.util.AdminUtility : responsible for getting the initial context, binding /unbinding of the administered objects. | | Property file decides which subclass of this AdminUtility gets picked up.Sub class instantiated using reflection to keep the code independent of teh actual impl | | e.g | org.jboss.test.messaging.util.provider.JBossMessAdminUtility extends | org.jboss.test.messaging.util.AdminUtility | | JBossMessAdminUtility uses ServerWrapper internally(for the time being) | | org.jboss.test.messaging.util.provider.JBossMQAdminUtility : for JBossMQ.. | | | | Let me know if this is ok,. | {Will come out with the design proposal for the perofrmance test framework in the other thread.} View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3871734#3871734 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3871734 |
From: <ovi...@jb...> - 2005-03-28 20:04:15
|
Looks fine. Go ahead. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3871809#3871809 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3871809 |
From: <ovi...@jb...> - 2005-03-29 00:41:24
|
An observation related to naming: org.jboss.test.messaging.util is intendend to contain tests for Messaging utility classes, not utilities. Please use org.jboss.test.messaging.tools for that. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3871830#3871830 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3871830 |
From: <ovi...@jb...> - 2005-03-30 05:52:38
|
The existent JMS facade unit tests (org.jboss.test.jms) test so far only colocated configurations. It would be nice if you could create similar tests that use two VMs. See http://jira.jboss.org/jira/browse/JBMESSAGING-45 View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3871990#3871990 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3871990 |
From: rajdeep_dua <nu...@jb...> - 2005-03-30 17:23:51
|
sure, give me sometime to clear jgroup concepts. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3872069#3872069 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3872069 |
From: <ovi...@jb...> - 2005-03-31 17:02:33
|
I didn't exactly think about JGroups here (maybe I should have). I would have started with something really simple, just two VMs, Runtime.exec() or something. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3872224#3872224 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3872224 |
From: rajdeep_dua <nu...@jb...> - 2005-04-01 05:31:17
|
It will be a combination of the two 1. exec will spawn the processes 2. jgroups will choreagraph the events between these processes and the results this mechanism i am planning to use for the performance tests also. (remoting does the similar things!) and they have the infrastructure in place. Only glitch is all the classes like DistributedTestCase,MultipleTestCase are in org.jboss.remoting : Should i keep using them or should i make a replica for jms/or somebody at jboss abstracts them to a common module, where multiple modules can use them? View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3872301#3872301 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3872301 |
From: <ovi...@jb...> - 2005-04-01 05:45:36
|
Sounds good. We shouldn't use classes out of org.jboss.remoting.test. This would introduce a dangerous dependency. We should nicely ask Tom Elrod for copyright and physically copy the classes we need in our own project. I am sure you will come with a lot of new ideas and modify them in the future. If you'll be able to abstract a commod distributed testing framework, that's nice and useful, but it's not something we want to do right now. Create a JIRA tasks for exactly this thing, so we won't forget, and move on. Probably Ryan already has plans to do something similar. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3872302#3872302 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3872302 |
From: rajdeep_dua <nu...@jb...> - 2005-04-08 10:14:45
|
Ovidiu, Spent last week on analysis of jgroups and the distributed testing framework developed by Tom. i developed the test clients with producer/consumers in separate VMs and notifications through jgroups. With 3 VMs its very slow and its not so predictable(may be i am doing somethign wrong).I was wondering what will happen when its more that 3 VMs? so i am thinking of using RMI for the synchronization instead. Will update on that in a day or two. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3873194#3873194 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3873194 |
From: <ovi...@jb...> - 2005-04-08 19:21:30
|
It shouldn't be slow. Something it's wrong. JGroups scales well (when used correctly) and the breaking point is certainly NOT 3 members. I wouldn't go the RMI path, I would try to figure out what's wrong with your approach. Remeber that we eventually want to evolve this in a distributed test framework. If you send me the code and explain what you are trying to do, I could take a look ... View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3873278#3873278 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3873278 |
From: rajdeep_dua <nu...@jb...> - 2005-04-15 07:04:23
|
The distributed test case framework for JMS using jgroups has been implemented(Based on remoting DTF source base) Scenario covered 1.One producer 2.One synchronous consumer Its based on the distributed test framework developed by Tom for remoting org/jboss/dtf/** I copied it into jboss-head/jms/tests directory. Result Testsuite: org.jboss.test.messaging.jms.dist.DistributedProducerSynchConsumerTestCase Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 35.441 sec ------------- Standard Output --------------- ------------------------------------------------------- GMS: address is localhost:1821 ------------------------------------------------------- ------------- ---------------- --------------- Testcase: testInvokers took 30.113 sec FAILED expected: but was:<ID:3j001-4n2l02-e7k0983a-1-e7k09gkq-3> junit.framework.ComparisonFailure: expected: but was:<ID:3j001-4n2l02-e7k0983a-1-e7k09gkq-3> at org.jboss.test.messaging.jms.dist.DistributedProducerSynchConsumerTestCase.testInvokers(DistributedProducerSynchConsumerTestCase.java:125) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at org.jboss.dtf.DistributedTestCase.run(DistributedTestCase.java:509) The Test Fails, producer is setting the msg id to null, (similar scenario in a local call passes) With last weeks' CVS co this was passing, when i did the update yesterday the test started failing. Issues faced 1.Had to change the {}/jboss-head/jms/src/resources/jgroups-core.jar with the one from thirdparty module Problem with RPCDispatcher of jgroups-core, not able to make remote calls to methods of super class if they are not overridden in the sub class. 2.Added following property to jgroups props in org.jboss.dtf.DistributedTestCase to make jgroups work on my m/c bind_addr=127.0.0.1; please remove this if tests give an error. Design The test case has the following java files org.jboss.test.messaging.jms.dist.DistributedProducerSynchConsumerTestCase | org.jboss.test.messaging.jms.dist.ProducerClientTest | org.jboss.test.messaging.jms.dist.SynchConsumerClientTest | DistributedProducerSynchConsumerTestCase is the main TestCase (extends org.jboss.dtf.DistributedTestCase). It configures the Topics/Queues using ServerManagement utility. It launches Producers and Consumers in their own VMs (each producer and consumer extend org.jboss.dtf.DistributedTestCase.) during setUp. All the VMs wait for each other to come up and then the actual execution of the test cases happen.(each VM must know what are the total number of parties coming up) In this test case producer and consumers update DistributedProducerSynchConsumerTestCase for the messageID sent and received , the assert call happens in DistributedProducerSynchConsumerTestCase. New Features added to org.jboss.dtf.DistributedTestCase Call back method : public void sendMessage(String type,String message) | { | MethodCall call = new MethodCall("receiveMessage", | new Object[]{type,message}, new Class[]{String.class,String.class}); | disp.callRemoteMethods(null, call, GroupRequest.GET_NONE, 0); | log.debug("sent Message :" + call); | } | | public void receiveMessage(String type,String message) | { | log.debug("Received Msg:" + type + "::" + message); | } | for sending receiving the messageIDs between VMs. Logging initializer : public Logger setLogging(String level, String fileName, Class classObj) | { | ... | } | Each VM has its own log file named named after fully qualified class name and an ID given as a command line agreement. Changes to build.xml in jboss-head/jms/tests Added the following lines in | <path id="library.classpath"> | | <!--<pathelement path="${module.root}/../src/resources/jgroups-core.jar"/>--> | <pathelement path="${module.root}/../src/resources/jgroups.jar"/> | <pathelement path="${module.root}/../../j2ee/output/lib/jboss-j2ee.jar"/> | | Added new component | <path id="jms.classes"> | <pathelement path="./../output/etc"/> | <pathelement path="./../output/classes"/> | <pathelement path="${module.root}/../../aspects/output/lib/jboss-aspect-library.jar"/> | <pathelement path="${module.root}/../../common/output/lib/jboss-common.jar"/> | <pathelement path="${module.root}/../../remoting/output/lib/jboss-remoting.jar"/> | <pathelement path="${module.root}/../../j2se/output/lib/jboss-j2se.jar"/> | <pathelement path="${module.root}/../../naming/output/lib/jnp-client.jar"/> | <pathelement path="${module.root}/../../thirdparty/javassist/lib/javassist.jar"/> | <pathelement path="${module.root}/../../thirdparty/oswego-concurrent/lib/concurrent.jar"/> | <pathelement path="${module.root}/../../thirdparty/apache-commons/lib/commons-httpclient.jar"/> | <pathelement path="${module.root}/../../thirdparty/trove/lib/trove.jar"/> | </path> | | <path id="dependentmodule.classpath"> | <path refid="jboss.common.classpath"/> | <path refid="jboss.jms.classpath"/> | <path refid="jboss.j2ee.classpath"/> | <path refid="jboss.j2se.classpath"/> | <path refid="jms.classes"/> | <path refid="jboss.remoting.classpath"/> | <path refid="jboss.aop.classpath"/> | </path> | | <target name="tests-jar" | depends="compile-test-classes" | description="Creates the jar file with all the tests"> | | <mkdir dir="${build.tests.lib}"/> | | <!-- Build the tests jar --> | <jar jarfile="${build.tests.lib}/${build.tests.archive}"> | <fileset dir="${build.tests.classes}"> | <include name="org/jboss/test/messaging/**"/> | <include name="org/jboss/dtf/**"/> | <include name="org/jboss/dummy/**"/> | </fileset> | </jar> | </target> | Changes made to org.jboss.jms.ServerManagement Added the following methods | public synchronized static void startRemoteServer(Hashtable ht) throws Exception | | public synchronized static void stopRemoteServer() throws Exception | | public static InitialContext getRemoteInitialContext() throws Exception | Ovidiu,Earlier tests i had written were based on my design in this forum. Now those utility classes are doing what ServerManagement does. ServerManagement has to be abstracted into an interface so that our test cases run with any jms impl. View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3874050#3874050 Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3874050 |