Hi,
Our installation has similar number of elements (~15000) and yet the update seems to go on forever. The status never changes. I have set a variable CATALINA_HOME=$HOME/jakarta-tomcat-5.0.28 in my .profile file and followed all the instructions as mentioned in the web page.
Is there any particular change to be done for Derby ?
Finally, it would be lot nicer if you could ship a job that does the parsing and loading of the Derby database as it is a one-time activity. Plus, a job can be setup to be run from selected IDs only thereby limiting the number of persons allowed to submit.
Regards,
Nags.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I like your suggestion about using a job for running the updates. That would make it a lot easier to schedule. I am also going to make a change so that the update is a singleton.
Alan
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have got to ask this. In the unstring.properties file, the following variables have been mentioned :
Default if commented out or removed:
proglib=//DD:PROG
copylib=//DD:COPY
includelib=//DD:INCLD
I did not quite understand how a JCL DD reference figures here.
Therefore, I had given the PDS name that holds the program and copybooks. (Our DB2 includes are in COPYBOOK PDS too; that should be ok ?) I had tried providing the name both with and without quotes.
I tried an update by allocating the PDS names to the said DD names using TSO ALLOC. I get a NPE here too - with slight difference in messages as compared from above.
I made the mistake of assuming that the pds has members in it when looping through it and it throws a null pointer if it can't find members or the PDS. I will have to fix it so that it gives a more meaningfully message if the PDS is empty or not found.
Coboldoc is using JZOS to read the PDS members. JZOS is a free toolkit that IBM ships with its SDK that allows java programs running in unix system services to read non-hfs files such as PDS datasets. The naming convention it uses for non-hfs files start with "//".
Try putting a // in front of the pds names. For example, if your programs are in MY.COBOL.PDS then set
proglib=//MY.COBOL.PDS
You can also pass in the datasets from the jcl used to start Tomcat. JZOS also uses a naming convention where it looks for the datasets defined in the jcl that started the jvm if the name referenced by the java code starts with "//DD:". For example, if you set unstring.properties to (or let it default to):
Then you can specify the datasets in the Tomcat jcl by adding PROG, COPY, and INCLD DD statements:
//*
//* Datasets used by applications running under Tomcat
//*
//PROG DD DSN=MY.COBOL.PDS,DISP=SHR
//COPY DD DSN=MY.COPYLIB.PDS,DISP=SHR
//INCLD DD DSN=MY.INCLUDE.PDS,DISP=SHR
The shop that I work at puts DB2 includes in a separate PDS (I don't know why) so I had to treat them separately because JZOS doesn't handle concatenation. I think it should be fine if you point the includelib to the copylib PDS.
Let me know if you have any problems.
Alan
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I _think_ I know the problem. I just noticed that our source PDSes are having DSORG as PO-E i.e. PDS-Extended. Do you think this could be the reason ? I have used JZOS; well, fiddled with it really. I am yet to use it for some heavy duty work.
I have initiated a copy to a regular PDS. It will take sometime until I re-initiate the update with the name of the PDS than PDS-E.
Will report soon.
Regards,
Nags.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
And, I had a nagging feeling that the PDS name must be enclosed in quotes, else it will not consider it a fully qualified DSN and so prefixes the user ID as the HLQ. (See Chapter 10 in C/C++ Programming Guide). I put the quotes in and still, the parser had problems with certain PDS members. So, I coded them in the JCL and got it working.
WONDERFUL job, Alan !
I have been the dovetail to ibm path too. Infact, I came across Coboldoc from your post in dW which you have pasted above !
BTW, if you are considering the JCL option, I have a suggestion. Apart from the JCL that implements your parsing logic, it would be great, if you published a load-only JCL too. That is, you publish the DDL of the tables in Derby and show the template JCL which users can use it to load into Derby. With this option, you can leave the extracting/parsing logic part to users who will handle their installation's particular idiosyncrasies !
The only setup for Derby should be having the derby.jar in the classpath and you can specify a database.home in unstring.properties. The database.home can be an empty directory when you first start. Coboldoc uses derby as an embedded database and it creates the tables when you first use it.
Hi Alan,
I checked the unstring.log file. There is an error about PdsDirectory usage. I could verify that everything is in place except for this :
"..the directory containing your unstring.properties file is in your servers classpath and that the jar containing PdsDirectory is in your servers classpath
java.lang.reflect.InvocationTargetException..."
I have the unstring.properties file in $CATALINA_HOME/common/classes. Also, I have not changed the jzos.file.factory.class and jzos.pds.directory.class settings as the default values that it will take are valid.
What am I still missing ?
Regards,
Nags.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If you have ibmjzos.jar and it is in your classpath then try changing unstring.properties to reference the ibm version:
jzos.file.factory.class=com.ibm.jzos.FileFactory
jzos.pds.directory.class=com.ibm.jzos.PdsDirectory
Since I don't have access to the ibmjzos.jar I could not test it. However, the javadoc api seems to be the same as the dovetail version for the methods that I use so it probably works the same.
Hi,
Our installation has similar number of elements (~15000) and yet the update seems to go on forever. The status never changes. I have set a variable CATALINA_HOME=$HOME/jakarta-tomcat-5.0.28 in my .profile file and followed all the instructions as mentioned in the web page.
Is there any particular change to be done for Derby ?
Finally, it would be lot nicer if you could ship a job that does the parsing and loading of the Derby database as it is a one-time activity. Plus, a job can be setup to be run from selected IDs only thereby limiting the number of persons allowed to submit.
Regards,
Nags.
Are there any exceptions in the logs? I added a page to the project website that describes how to view the logs:
http://unstring.sourceforge.net/wiki/index.php/Log_files
I like your suggestion about using a job for running the updates. That would make it a lot easier to schedule. I am also going to make a change so that the update is a singleton.
Alan
Hi Alan,
Thanks for the link to the log files. I would check them.
If you can share with me the parsing logic you used, I can contribute some towards the JCL. :)
Regards,
Nags
Hi Alan,
I checked the logs and see NPE messages :
java.lang.NullPointerException
.at org.unstring.io.Directory.listHFS(Directory.java:61)
.at org.unstring.io.Directory.<init>(Directory.java:45)
.at org.unstring.actions.UpdateAction.execute(UpdateAction.java:69)
..and more
I have got to ask this. In the unstring.properties file, the following variables have been mentioned :
Default if commented out or removed:
proglib=//DD:PROG
copylib=//DD:COPY
includelib=//DD:INCLD
I did not quite understand how a JCL DD reference figures here.
Therefore, I had given the PDS name that holds the program and copybooks. (Our DB2 includes are in COPYBOOK PDS too; that should be ok ?) I had tried providing the name both with and without quotes.
I tried an update by allocating the PDS names to the said DD names using TSO ALLOC. I get a NPE here too - with slight difference in messages as compared from above.
java.lang.NullPointerException
.at org.unstring.actions.UpdateAction.execute(UpdateAction.java:85)
...and more.
Regards,
Nags.
Hi Nags,
I made the mistake of assuming that the pds has members in it when looping through it and it throws a null pointer if it can't find members or the PDS. I will have to fix it so that it gives a more meaningfully message if the PDS is empty or not found.
Coboldoc is using JZOS to read the PDS members. JZOS is a free toolkit that IBM ships with its SDK that allows java programs running in unix system services to read non-hfs files such as PDS datasets. The naming convention it uses for non-hfs files start with "//".
Try putting a // in front of the pds names. For example, if your programs are in MY.COBOL.PDS then set
proglib=//MY.COBOL.PDS
You can also pass in the datasets from the jcl used to start Tomcat. JZOS also uses a naming convention where it looks for the datasets defined in the jcl that started the jvm if the name referenced by the java code starts with "//DD:". For example, if you set unstring.properties to (or let it default to):
proglib=//DD:PROG
copylib=//DD:COPY
includelib=//DD:INCLD
Then you can specify the datasets in the Tomcat jcl by adding PROG, COPY, and INCLD DD statements:
//*
//* Datasets used by applications running under Tomcat
//*
//PROG DD DSN=MY.COBOL.PDS,DISP=SHR
//COPY DD DSN=MY.COPYLIB.PDS,DISP=SHR
//INCLD DD DSN=MY.INCLUDE.PDS,DISP=SHR
The shop that I work at puts DB2 includes in a separate PDS (I don't know why) so I had to treat them separately because JZOS doesn't handle concatenation. I think it should be fine if you point the includelib to the copylib PDS.
Let me know if you have any problems.
Alan
Hi Allan,
With the value given as //MY.COBOL.PDS, the Reading Directories part shoots upto 100% in no time. But, I still have these messages :
java.lang.NullPointerException
.at org.unstring.actions.UpdateAction.execute(UpdateAction.java:85)
...and more.
I _think_ I know the problem. I just noticed that our source PDSes are having DSORG as PO-E i.e. PDS-Extended. Do you think this could be the reason ? I have used JZOS; well, fiddled with it really. I am yet to use it for some heavy duty work.
I have initiated a copy to a regular PDS. It will take sometime until I re-initiate the update with the name of the PDS than PDS-E.
Will report soon.
Regards,
Nags.
Hi Alan,
No, even with a regular PDS too, it did not work.
BTW, I have this message in derby.log :
Database Class Loader started - derby.database.classpath=''
Is there any setup to be done for Derby ?
Regards,
Nags.
Hi Alan,
At last, SUCCESS !
"Alternatively, you can download the jzos125.zip from http://www.alphaworks.ibm.com/tech/zosjavabatchtk/download, extract the jzos.jar, and put it in Tomcat common/lib directory."
Now, THAT I missed ! Sorry.
And, I had a nagging feeling that the PDS name must be enclosed in quotes, else it will not consider it a fully qualified DSN and so prefixes the user ID as the HLQ. (See Chapter 10 in C/C++ Programming Guide). I put the quotes in and still, the parser had problems with certain PDS members. So, I coded them in the JCL and got it working.
WONDERFUL job, Alan !
I have been the dovetail to ibm path too. Infact, I came across Coboldoc from your post in dW which you have pasted above !
BTW, if you are considering the JCL option, I have a suggestion. Apart from the JCL that implements your parsing logic, it would be great, if you published a load-only JCL too. That is, you publish the DDL of the tables in Derby and show the template JCL which users can use it to load into Derby. With this option, you can leave the extracting/parsing logic part to users who will handle their installation's particular idiosyncrasies !
And, if you are a WebSphere MQ on z/OS and a DFSORT man, here is a little something from a fellow mainframe developer, ;-)
http://www-1.ibm.com/support/docview.wss?uid=swg24015179
Regards,
Nags.
The only setup for Derby should be having the derby.jar in the classpath and you can specify a database.home in unstring.properties. The database.home can be an empty directory when you first start. Coboldoc uses derby as an embedded database and it creates the tables when you first use it.
It is probably failing on line 85 of UpdateAction.java because the copybooks map is empty (see
http://unstring.svn.sourceforge.net/viewvc/unstring/tags/1.0.6/src/org/unstring/actions/UpdateAction.java?revision=55&view=markup\). The map is coming from Directory.getDirMap() (see http://unstring.svn.sourceforge.net/viewvc/unstring/tags/1.0.6/src/org/unstring/io/Directory.java?revision=55&view=markup\). The map is populated in Directory.listPDS(). Either the PdsDirectory class is returning an empty collection or there is a problem invoking the class. If the problem is with invoking the class then there should be something in the log4j log. Are there any messages in the log4j log?
Hi Alan,
I checked the unstring.log file. There is an error about PdsDirectory usage. I could verify that everything is in place except for this :
"..the directory containing your unstring.properties file is in your servers classpath and that the jar containing PdsDirectory is in your servers classpath
java.lang.reflect.InvocationTargetException..."
I have the unstring.properties file in $CATALINA_HOME/common/classes. Also, I have not changed the jzos.file.factory.class and jzos.pds.directory.class settings as the default values that it will take are valid.
What am I still missing ?
Regards,
Nags.
IBM has taken the jzos classes and renamed the packages from com.dovetail.jzos to com.ibm.jzos. The company I work at doesn't have the com.ibm.jzos version of jzos because they have an older version of z/os. I had to download the jzos.jar from http://www.alphaworks.ibm.com/tech/zosjavabatchtk/download (I used the jzos.jar that is in jzos125.zip). I asked IBM if I can include jzos.jar with the Coboldoc distribution and they said no ( http://www.ibm.com/developerworks/forums/dw_thread.jsp?message=13924503&cat=28&thread=153371&treeDisplayType=threadmode1&forum=801#13924503 ).
If you have ibmjzos.jar and it is in your classpath then try changing unstring.properties to reference the ibm version:
jzos.file.factory.class=com.ibm.jzos.FileFactory
jzos.pds.directory.class=com.ibm.jzos.PdsDirectory
Since I don't have access to the ibmjzos.jar I could not test it. However, the javadoc api seems to be the same as the dovetail version for the methods that I use so it probably works the same.
Alternatively, you can download the jzos125.zip from http://www.alphaworks.ibm.com/tech/zosjavabatchtk/download
, extract the jzos.jar, and put it in Tomcat common/lib directory.
Thanks for your patience Nags.