Thanks for spending time with tool. Yes, DB2 treats catalogue as database itself. Good that You are able to connect and analyze.
1.) There are multiple ways you can delete regular expression - you can go to Data Quality --> Standardization Regex ( create a key value pair of file if not already there - key can be regex and values can be replace value ) -choose the column and file - it will do. You can save it back to DB
Other way is you open the table in edit mode then from menu choose regex search replace ( attached is screen shot)
Other you can use tools --> open file --> columns --> Standardization regex
Preferred way is, first take a dump , use file regex rather than in Situ because it will change database. In String analysis , you can do analysis of regex match etc which will appear from popupmenu.txt
2.) If analysis is done in sql then Sql can be saved and reused in future - Sql Interface --> Option -> Save Query then you can open query in future. The other option is you can a business rule and schedule it main frame --> Business Rules --> Create Rule ( but it is still sql based.)
But analysis which is not SQL is only interactive for now and NOT persisted for. But we can build a customer workflow where analysis can be saved in workflow and can be used later. Talk to your organization, if they will be interested we can custom build. Everything else is free and open sourced.
Feel free to contact us if you need any help. Feel free to share you feedback ( even it is not best:) though we try to hard to make a good and useful tool.
regards,
Vivek
---- On Wed, 26 Sep 2018 21:27:29 +0530 , < . @ .ca> wrote ----
Hi Vivek,
I found the problem with the SQL interface – the semicolon at the end of the statement caused the error. Meantime, the schema name or catalog is required too.
I will appreciate it very much if you can shed some light on the following two questions:
Is there a way to delete some existing regular expressions from the table?
Can I save some analyses (settings/parameters) and run later against the same database?
Thank you,
/
From: ,
Sent: September 26, 2018 10:54 AM
To: 'Support' support@arrahtech.com
Subject: RE: osDQ tool questions
Thank you, Vivek.
I played with the tool a bit more and was able to access the database tables after I put both Catalog and Schema (catalog.schema) in the Catalog input field.
Now I encountered another problem. I am unable to query database using the SQL interface. I tried without and with catalog name as follows:
DB name: DB
Table name: TABLE
DB Catalog: BD.SCHEMA
table only
Cat.table
Database&table
By the way, is there a way to delete some existing regular expressions from the table? Can I save some analyses (settings/parameters) and run later against the same database?
Regards,
/
---- On Mon, 24 Sep 2018 20:13:39 +0530 , < . @ .ca> wrote ----
Thank you for your quick response.
I put in the Catalog and Schema connection parameters, got the same results.
Also I tried the SQL Interface and got error on Check SQL: DB2 SQL Error: SQLCODE=-104, SQLSTATE=42601, SQLERRMC=;; up by TRVL_CRDN_TCSN;END-OF-STATEMENT, DRIVER=4.15.82.
Anything I should check that “Error: Report Table Not Filled”?
Regards,
/
From: Support [mailto:support@arrahtech.com]
Sent: September 21, 2018 1:11 AM
To: , < . @ .ca>
Subject: Re: osDQ tool questions
Thanks for using aggregate profiler. Unfortunately user guide is not updated. It will be done in coming versions.
Following error occured when it is not able to access tables for profiling.
You know different Databases treat Catalog, Schema in different way. Aggregate profiler makes "cat.table" to make qualified calls. You need to put Catalog and Schema (if empty will take default) values in connection parameter to make qualifies calls.
In tools->SqlInterface - there is query dialog. You can type sql there and see if they return values like 'select a from cat.table'
---- On Thu, 20 Sep 2018 21:00:06 +0530 , < . @ .ca> wrote ----
Hi,
I am working on a DQ proof of concept project and just got the Aggregate Profiler v6.2.6 installed. But the user guide included in the download was v6.1.8 which doesn’t match the v6.2.6 GUI. Could you please help me on the user guide for v6.2.6?
My test data is currently stored in a DB2 database. When I tried to access the data, I got the following console message:
at org.arrah.framework.profile.FirstInformation.getPatternValues(FirstInformation.java:122)
at org.arrah.gui.swing.FirPanel$MyChangeListener.propertyChange(FirPanel.java:290)
Error: Could not Get Distribution Information
……
I checked the Limitation Information under Metadata Information menu and found the Maximum Row Size (for DB2) is 32677. Is there a way to increase this row size? We have millions records in the database.
Thank you very much for your hard work on developing and supporting the tool.
Regards,
/
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
---- On Thu, 27 Sep 2018 17:36:39 +0530 Support support@arrahtech.com wrote ----
Thanks for spending time with tool. Yes, DB2 treats catalogue as database itself. Good that You are able to connect and analyze.
1.) There are multiple ways you can delete regular expression - you can go to Data Quality --> Standardization Regex ( create a key value pair of file if not already there - key can be regex and values can be replace value ) -choose the column and file - it will do. You can save it back to DB
Other way is you open the table in edit mode then from menu choose regex search replace ( attached is screen shot)
Other you can use tools --> open file --> columns --> Standardization regex
Preferred way is, first take a dump , use file regex rather than in Situ because it will change database. In String analysis , you can do analysis of regex match etc which will appear from popupmenu.txt
2.) If analysis is done in sql then Sql can be saved and reused in future - Sql Interface --> Option -> Save Query then you can open query in future. The other option is you can a business rule and schedule it main frame --> Business Rules --> Create Rule ( but it is still sql based.)
But analysis which is not SQL is only interactive for now and NOT persisted for. But we can build a customer workflow where analysis can be saved in workflow and can be used later. Talk to your organization, if they will be interested we can custom build. Everything else is free and open sourced.
Feel free to contact us if you need any help. Feel free to share you feedback ( even it is not best:) though we try to hard to make a good and useful tool.
regards,
Vivek
---- On Wed, 26 Sep 2018 21:27:29 +0530 , < . @ .ca> wrote ----
Hi Vivek,
I found the problem with the SQL interface – the semicolon at the end of the statement caused the error. Meantime, the schema name or catalog is required too.
I will appreciate it very much if you can shed some light on the following two questions:
Is there a way to delete some existing regular expressions from the table?
Can I save some analyses (settings/parameters) and run later against the same database?
Thank you,
/
From: ,
Sent: September 26, 2018 10:54 AM
To: 'Support' support@arrahtech.com
Subject: RE: osDQ tool questions
Thank you, Vivek.
I played with the tool a bit more and was able to access the database tables after I put both Catalog and Schema (catalog.schema) in the Catalog input field.
Now I encountered another problem. I am unable to query database using the SQL interface. I tried without and with catalog name as follows:
DB name: DB
Table name: TABLE
DB Catalog: BD.SCHEMA
table only
Cat.table
Database&table
By the way, is there a way to delete some existing regular expressions from the table? Can I save some analyses (settings/parameters) and run later against the same database?
Regards,
/
---- On Mon, 24 Sep 2018 20:13:39 +0530 , < . @ .ca> wrote ----
Thank you for your quick response.
I put in the Catalog and Schema connection parameters, got the same results.
Also I tried the SQL Interface and got error on Check SQL: DB2 SQL Error: SQLCODE=-104, SQLSTATE=42601, SQLERRMC=;; up by TRVL_CRDN_TCSN;END-OF-STATEMENT, DRIVER=4.15.82.
Anything I should check that “Error: Report Table Not Filled”?
Regards,
/
From: Support [mailto:support@arrahtech.com]
Sent: September 21, 2018 1:11 AM
To: , < . @ .ca>
Subject: Re: osDQ tool questions
Thanks for using aggregate profiler. Unfortunately user guide is not updated. It will be done in coming versions.
Following error occured when it is not able to access tables for profiling.
You know different Databases treat Catalog, Schema in different way. Aggregate profiler makes "cat.table" to make qualified calls. You need to put Catalog and Schema (if empty will take default) values in connection parameter to make qualifies calls.
In tools->SqlInterface - there is query dialog. You can type sql there and see if they return values like 'select a from cat.table'
Feel free to contact me if you have any questions. You can look at community discussion page - https://sourceforge.net/p/dataquality/discussion/
regards,
---- On Thu, 20 Sep 2018 21:00:06 +0530 , < . @ .ca> wrote ----
Hi,
I am working on a DQ proof of concept project and just got the Aggregate Profiler v6.2.6 installed. But the user guide included in the download was v6.1.8 which doesn’t match the v6.2.6 GUI. Could you please help me on the user guide for v6.2.6?
My test data is currently stored in a DB2 database. When I tried to access the data, I got the following console message:
DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=PXS132.TIUC150, DRIVER=4.15.82
Error: Report Table Not Filled
And the Java exception like the following:
C:\TEMP\ProfilerV6.2.6\ProfilerV6.2.6>java -Xmx4096M -Xms4096M -classpath osdq-desktop-6.2.6-SNAPSHOT.jar;lib\rowset.jar;lib\jdbcjar*;lib\hivejar*;lib\hive2jar*;lib* org.arrah.gui.swing.Profiler
Warning:DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=PXS132.TIUC157, DRIVER=4.15.82
java.lang.Exception:
Warning: Could not Get Pattern Information
at org.arrah.framework.profile.FirstInformation.getPatternValues(FirstInformation.java:122) at org.arrah.gui.swing.FirPanel$MyChangeListener.propertyChange(FirPanel.java:290)Error: Could not Get Distribution Information
……
I checked the Limitation Information under Metadata Information menu and found the Maximum Row Size (for DB2) is 32677. Is there a way to increase this row size? We have millions records in the database.
Thank you very much for your hard work on developing and supporting the tool.
Regards,
/