-- :NOTE:
This message was copied from the thread "Checking for consistency between systems" to here. The feature requested here is to allow different versions of the database driver to be used in the same SQLUnit test.
---- Requested by boucherb 8< ---------------------------------
As far as the class loader issue is concerned, it goes even deeper than url handling I'm afraid.
Consider this:
Say I have HSQLDB 1.7.1 jar and HSQLDB 1.7.2 jar.
Both drivers are org.hsqldb.jdbcDriver and both repond to "jdbc:hsqldb:..."
Even if one were to use javax.sql.DataSource, bypassing the java.sql.DriverManager, each data source would have to be loaded by a distinct class loader. Otherwise, when asking for an org.hsqldb.jdbcDriver instance, whichever jar was first on the common class loader's class path would be the one providing the org.hsqldb.jdbcDriver class (and the rest of the org.hsqldb.* classes along with it).
If I wanted to test one release of hsqldb against another, distinct class loaders for the driver classes would have to be the way to go. This probably applies to many JDBC implementations as the package naming and class naming usually does not change across releases, especially point releases.
In fact, one can't even go the route of using the hsqldb network protocol connection url and set up a 1.7.1 server and 1.7.2 server, as the network protocol has changed, I think in an incompatible way, so the 1.7.1 driver cannot talk to a 1.7.2 server and a 1.7.2 driver cannot talk to a 1.7.1 server.
Even if a single driver could talk both protocols, one generally wants to test the driver as much as the underlying engine. Without distinct class loaders, one would still be using a single driver and set of backing classes to talk with both servers and therefore only the behaviour of the servers would be diffed, not the shared driver point release.
Anyway, maybe getting as heavy as supporting distinct class loaders for drivers is not something to worry about at this time.
---- rest of message snipped >8 --------------------------
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Looks like URLClassLoader() would be the way to go here, as you pointed out. It looks like its doable, but I would like to run a test on it just to make sure. Here is my plan:
In ConnectionHandler:
instead of:
Class.forName(driverClass);
we can use:
Class.forName(driverClass, true|false, new URLClassLoader(
new String[] {jarFileUrl});
or some thing similar.
I will do this test and let you know the results.
-sujit
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I am going to download the HSQLDB 1.7.1 and 1.7.2 driver files, for a small test. If it is as easy as it looks from the javadocs, we could push this functionality out with Simon's changes since we will be handling multiple connections in the same suite anyway. If not, we can tackle this after Simon's features are taken care of. What differences should I be testing for, to make sure I got the correct one? I am looking for a sql snippet or stored procedure, just to check that the connection(s) I get goes through the correct driver.
Thanks
Sujit
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2003-11-13
In general, you can use the JDBC DatabaseMetaData implementation of any driver, like this:
DatabaseMetaData dbmd = conn.getMetaData();
String driverName = dbmd.getDriverName();
String driverVersion = dbmd.getDriverVersion();
int eriverMajorVersion = dbmd.getDriverMajorVersion();
etc // look in the java.sql.DatabaseMetaData java docs.
as of JDBC3, you can query the database engine version of the connection independenty. I suppose this comes from the fact that there are now a plethora of third-party middleware drivers, so a single driver may acutally be hooked up to one of several different BDMS backends.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
OK, so I created a JUnit test and looks like its not possible. The URLClassLoader does bring back the class from a JAR file specified in a file:// URL, but DriverManager does not recognize anything that it cannot find in the CLASSPATH that the JVM was instantiated in. I tried deregistering all the drivers and registering only the Driver class that came back from the URLClassLoader, but no luck.
So I thought that the way to go (as you had suggested previously in an email) would be to make SQLUnit have the functionality to look up a DataSource object on a remote JNDI server. That way you could run two remote JNDI servers, one with the HSQLDB 1.71 jar and the other with HSQLDB 1.72 jar, and make SQLUnit point to them to pick up the different connections. Something like this:
I have used JBossNS since its free, but you could also use BEA Weblogic or IBM Websphere to do this, since they both support remote standalone JNDI servers.
The code is already in CVS to parse this. I realize that this implementation is not particularly lightweight, but given that (a) not too many people would be trying to do what you are trying to do, and (b) people who use the JNDI form to get connections will already have access to a corporate JNDI server which can function in standalone mode, this approach is probably workable. Indeed, afaik, this seems to be the only approach to getting connections from different drivers which support the same URL signature in the same JVM.
I hope this will work for you. Please let me know if you have some ideas that are potentially less heavyweight than the one I chose.
I will be releasing this change and Simon's multi-threaded functinality change later today in release 2.8.
-sujit
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2003-12-08
True.
Looking at most of the JDBC-enabled GUI tools that allow dynamically selecting the driver class paths at runtime, what's been done is to provide a Work-Alike replacement for DriverManager that works around the "problem" with java.sql.DriverManager.
BTW
I haven't died or abandoned my efforts here. I've just been heavily tied up with trying to get the HSQLDB JMX extentions out the door for the 1.7.2 RC1
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Your email yesterday got me thinking about this, and I had never thought of building another DriverManager. I do remember now your mentioning it earlier, but I must have missed it in my previous go-round.
So I wrote a SQLUnitDriverManager with 3 getConnection() methods, whose arguments are identical to the DriverManager.getConnection() methods, except that you must pass in a String driver class name for your JDBC driver (com.mysql.jdbc.Driver for example) and the location of the JAR file as a URL (file:///tmp/mysql-connector-java.jar for example). I tried extending DriverManager but its constructor is private so SQLUnitDriverManager is its own (incomplete) version of the DriverManager class. Internally SQLUnitDriverManager uses Driver.connect() to retrieve the Connection object.
The JUnit test works and I have incorporated it into the ConnectionHandler, will be checking in to CVS in a few hours. An additional element <jarfile-url> needs to be specified under the <connection> element in order to use SQLUnitDriverManager to retrieve a Connection from a JAR file not in the CLASSPATH but whose URL is specified by the contents of <jarfile-url>.
So anyway, to make a long story short, we can do your HSQLDB test using JDBC. I have left the JNDI stuff in, since it is a nice-to-have, and Sun seems to be nudging everybody away from JDBC with its "DataSource is better" argument anyway.
And, thanks a lot for your suggestion, it pointed me in the right direction (you all but wrote the code for me :-)).
-sujit
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2003-12-13
Thanks Sujit.
Now if I could only finish writing some code for myself.
Have been fighting with MC4J 1.2b today, providing a pre-built "dashboard" xml document for the upcoming HSQLDB JMX addon package.
Anyway, hopefully I'll be able to send something off to Fred by Sunday and revisit this concern in the following week.
The "real" job has been ramping up quite a bit in the last week as well, so...:)
Cheers,
Campbell
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
>>Have been fighting with MC4J 1.2b today, providing a
>>pre-built "dashboard" xml document for the upcoming
>>HSQLDB JMX addon package.
Sounds cool. Maybe I will tap your expertise in adding some JMX instrumentation to a server program I am working on at work. Can you point me to some websites, etc?
>>Anyway, hopefully I'll be able to send something off to Fred
>> by Sunday and revisit this concern in the following week.
Thank you, take your time, I am not doing much with SQLUnit at the moment; I am trying to wrap up another stalled project on sourceforge, lets see where it goes.
-sujit
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2003-12-21
Sure...I'd be glad to pass on what little expertise I've gained in the Java Management Extentions area.
IN addition to the Sun Reference Implementation, hear are some interesting links:
mx4j.sourceforge.net (open JSR 3 and JSR 160 implementation)
mc4j.sourceforge.net (NetBeans platform based JMX console)
www.xmojo.org (derived from AdventNet's JMX implementation)
I think the fairly recent JSR 160 (JMX remoting standard) is quite exiciting, since it will allow writing clients without worry over supporting the many different home-rolled remoting implementations and having to include jars for each.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2003-12-21
Akkk! It's really late here and my spleling has gone down the tubes.
should have been: here are some interesting links...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
-- :NOTE:
This message was copied from the thread "Checking for consistency between systems" to here. The feature requested here is to allow different versions of the database driver to be used in the same SQLUnit test.
---- Requested by boucherb 8< ---------------------------------
As far as the class loader issue is concerned, it goes even deeper than url handling I'm afraid.
Consider this:
Say I have HSQLDB 1.7.1 jar and HSQLDB 1.7.2 jar.
Both drivers are org.hsqldb.jdbcDriver and both repond to "jdbc:hsqldb:..."
Even if one were to use javax.sql.DataSource, bypassing the java.sql.DriverManager, each data source would have to be loaded by a distinct class loader. Otherwise, when asking for an org.hsqldb.jdbcDriver instance, whichever jar was first on the common class loader's class path would be the one providing the org.hsqldb.jdbcDriver class (and the rest of the org.hsqldb.* classes along with it).
If I wanted to test one release of hsqldb against another, distinct class loaders for the driver classes would have to be the way to go. This probably applies to many JDBC implementations as the package naming and class naming usually does not change across releases, especially point releases.
In fact, one can't even go the route of using the hsqldb network protocol connection url and set up a 1.7.1 server and 1.7.2 server, as the network protocol has changed, I think in an incompatible way, so the 1.7.1 driver cannot talk to a 1.7.2 server and a 1.7.2 driver cannot talk to a 1.7.1 server.
Even if a single driver could talk both protocols, one generally wants to test the driver as much as the underlying engine. Without distinct class loaders, one would still be using a single driver and set of backing classes to talk with both servers and therefore only the behaviour of the servers would be diffed, not the shared driver point release.
Anyway, maybe getting as heavy as supporting distinct class loaders for drivers is not something to worry about at this time.
---- rest of message snipped >8 --------------------------
Looks like URLClassLoader() would be the way to go here, as you pointed out. It looks like its doable, but I would like to run a test on it just to make sure. Here is my plan:
In ConnectionHandler:
instead of:
Class.forName(driverClass);
we can use:
Class.forName(driverClass, true|false, new URLClassLoader(
new String[] {jarFileUrl});
or some thing similar.
I will do this test and let you know the results.
-sujit
Hi Campbell,
I am going to download the HSQLDB 1.7.1 and 1.7.2 driver files, for a small test. If it is as easy as it looks from the javadocs, we could push this functionality out with Simon's changes since we will be handling multiple connections in the same suite anyway. If not, we can tackle this after Simon's features are taken care of. What differences should I be testing for, to make sure I got the correct one? I am looking for a sql snippet or stored procedure, just to check that the connection(s) I get goes through the correct driver.
Thanks
Sujit
In general, you can use the JDBC DatabaseMetaData implementation of any driver, like this:
DatabaseMetaData dbmd = conn.getMetaData();
String driverName = dbmd.getDriverName();
String driverVersion = dbmd.getDriverVersion();
int eriverMajorVersion = dbmd.getDriverMajorVersion();
etc // look in the java.sql.DatabaseMetaData java docs.
as of JDBC3, you can query the database engine version of the connection independenty. I suppose this comes from the fact that there are now a plethora of third-party middleware drivers, so a single driver may acutally be hooked up to one of several different BDMS backends.
Thank you, I will create a straight JUnit test to see if the classloader issue pans out and let you know.
-sujit
OK, so I created a JUnit test and looks like its not possible. The URLClassLoader does bring back the class from a JAR file specified in a file:// URL, but DriverManager does not recognize anything that it cannot find in the CLASSPATH that the JVM was instantiated in. I tried deregistering all the drivers and registering only the Driver class that came back from the URLClassLoader, but no luck.
So I thought that the way to go (as you had suggested previously in an email) would be to make SQLUnit have the functionality to look up a DataSource object on a remote JNDI server. That way you could run two remote JNDI servers, one with the HSQLDB 1.71 jar and the other with HSQLDB 1.72 jar, and make SQLUnit point to them to pick up the different connections. Something like this:
<connection connection-id="1">
...<datasource>jdbc/testdb_1_71</datasource>
...<jndi>
......<arg name="java.naming.factory.initial" value="org.jnp.interfaces.NamingContextFactory" />
......<arg name="java.naming.provider.url" value="jnp://machine1:1099" />
......<arg name="java.naming.factory.url.pkgs" value="org.jboss.naming" />
...</jndi>
</connection>
<connection connection-id="2">
...<datasource>jdbc/testdb_1_72</datasource>
...<jndi>
......<arg name="java.naming.factory.initial" value="org.jnp.interfaces.NamingContextFactory" />
......<arg name="java.naming.provider.url" value="jnp://machine2:1099" />
......<arg name="java.naming.factory.url.pkgs" value="org.jboss.naming" />
...</jndi>
</connection>
I have used JBossNS since its free, but you could also use BEA Weblogic or IBM Websphere to do this, since they both support remote standalone JNDI servers.
The code is already in CVS to parse this. I realize that this implementation is not particularly lightweight, but given that (a) not too many people would be trying to do what you are trying to do, and (b) people who use the JNDI form to get connections will already have access to a corporate JNDI server which can function in standalone mode, this approach is probably workable. Indeed, afaik, this seems to be the only approach to getting connections from different drivers which support the same URL signature in the same JVM.
I hope this will work for you. Please let me know if you have some ideas that are potentially less heavyweight than the one I chose.
I will be releasing this change and Simon's multi-threaded functinality change later today in release 2.8.
-sujit
True.
Looking at most of the JDBC-enabled GUI tools that allow dynamically selecting the driver class paths at runtime, what's been done is to provide a Work-Alike replacement for DriverManager that works around the "problem" with java.sql.DriverManager.
BTW
I haven't died or abandoned my efforts here. I've just been heavily tied up with trying to get the HSQLDB JMX extentions out the door for the 1.7.2 RC1
Hi Campbell,
Your email yesterday got me thinking about this, and I had never thought of building another DriverManager. I do remember now your mentioning it earlier, but I must have missed it in my previous go-round.
So I wrote a SQLUnitDriverManager with 3 getConnection() methods, whose arguments are identical to the DriverManager.getConnection() methods, except that you must pass in a String driver class name for your JDBC driver (com.mysql.jdbc.Driver for example) and the location of the JAR file as a URL (file:///tmp/mysql-connector-java.jar for example). I tried extending DriverManager but its constructor is private so SQLUnitDriverManager is its own (incomplete) version of the DriverManager class. Internally SQLUnitDriverManager uses Driver.connect() to retrieve the Connection object.
The JUnit test works and I have incorporated it into the ConnectionHandler, will be checking in to CVS in a few hours. An additional element <jarfile-url> needs to be specified under the <connection> element in order to use SQLUnitDriverManager to retrieve a Connection from a JAR file not in the CLASSPATH but whose URL is specified by the contents of <jarfile-url>.
So anyway, to make a long story short, we can do your HSQLDB test using JDBC. I have left the JNDI stuff in, since it is a nice-to-have, and Sun seems to be nudging everybody away from JDBC with its "DataSource is better" argument anyway.
And, thanks a lot for your suggestion, it pointed me in the right direction (you all but wrote the code for me :-)).
-sujit
Thanks Sujit.
Now if I could only finish writing some code for myself.
Have been fighting with MC4J 1.2b today, providing a pre-built "dashboard" xml document for the upcoming HSQLDB JMX addon package.
Anyway, hopefully I'll be able to send something off to Fred by Sunday and revisit this concern in the following week.
The "real" job has been ramping up quite a bit in the last week as well, so...:)
Cheers,
Campbell
Hi Campbell,
>>Have been fighting with MC4J 1.2b today, providing a
>>pre-built "dashboard" xml document for the upcoming
>>HSQLDB JMX addon package.
Sounds cool. Maybe I will tap your expertise in adding some JMX instrumentation to a server program I am working on at work. Can you point me to some websites, etc?
>>Anyway, hopefully I'll be able to send something off to Fred
>> by Sunday and revisit this concern in the following week.
Thank you, take your time, I am not doing much with SQLUnit at the moment; I am trying to wrap up another stalled project on sourceforge, lets see where it goes.
-sujit
Sure...I'd be glad to pass on what little expertise I've gained in the Java Management Extentions area.
IN addition to the Sun Reference Implementation, hear are some interesting links:
mx4j.sourceforge.net (open JSR 3 and JSR 160 implementation)
mc4j.sourceforge.net (NetBeans platform based JMX console)
jboss.org/developers/projects/jboss/jbossmx (JBOSS JMX implementation)
www.xmojo.org (derived from AdventNet's JMX implementation)
I think the fairly recent JSR 160 (JMX remoting standard) is quite exiciting, since it will allow writing clients without worry over supporting the many different home-rolled remoting implementations and having to include jars for each.
Akkk! It's really late here and my spleling has gone down the tubes.
should have been: here are some interesting links...