CloverETL / News: Recent posts

New CloverETL 4.0.6

Bugfix release CloverETL 4.0.6

Bugfixes - Engine:

  • Fixed: SQL queries ending with semicolon cause graphs failures
Posted by Jaroslav Urban 2015-06-23

New CloverETL 4.0.5

Bugfix release CloverETL 4.0.5

Bugfixes - Engine:

  • Fixed: CloverDataWriter may report invalid checksum on append
  • Fixed: ExecuteMapReduce ignores the username from the connection
  • Fixed: Different behavior of DBInputTable with declared variable in TSQL between versions
  • Fixed: DBOutputTable prevents final commit when Commit=MAX_INT
  • Fixed: HTTPConnector returns WARN message even when successful status code is received... read more
Posted by Jaroslav Urban 2015-06-11

New CloverETL 4.1.0-M1

Milestone release of CloverETL 4.1.0

New Features & Improvements:

  • Optional ports in subgraphs
  • New Components (CrossJoin, GenericComponent, RecordToKeyValues, KeyValuesToRecord)
  • Amazon S3 support in file operations
  • Metadata extraction from Clover data files
  • JSONReader/XMLReader input->output mapping

Fixes:

  • FastSort may lose records for extremely small run size
  • XMLWriter: 'Write null element' does not work for elements with attributes
  • CTL2: switch does not allow negative number in cases
  • UniversalDataReader: "Character decoding error" results in graph failure in lenient and controlled mode... read more
Posted by Jan Sedlacek 2015-05-13

New CloverETL 4.0.4

Bugfix release CloverETL 4.0.4

Bugfixes - Engine:

  • Fixed: FastSort may lose records for extremely small run size
  • Fixed: XMLExtract - " character is escaped to &quot (missing semicolon)
  • Fixed: Readers fail with "max number of records" when reading from input port
  • Fixed: File operations: wildcard resolution in sandbox root fails
  • Fixed: Subgraph component does not finish when running in a loop
  • Fixed: Proper reporting of graph configuration errors avoiding duplicates
  • Fixed: Exported custom component attribute reports nonexistent warning... read more
Posted by Jaroslav Urban 2015-03-11

New CloverETL 3.5.5

Bugfix release CloverETL 3.5.5

  • Improved handle of "running" jobs of the lost cluster node

Bugfixes:

  • FastSort may lose records for extremely small run size
  • URL dialog should accept SFTP connection with certificate-based authentication
  • MultiFileReader: checkConfig warning for port protocol

Compatibility:

  • Writers do not decode escape sequences in sandbox URLs
Posted by Jaroslav Urban 2015-02-26

New CloverETL 4.0.3

Bugfix release CloverETL 4.0.3

Bugfixes:

  • Fixed: Server fails to start on Weblogic12 with message 'Cannot initialize engine'
  • Fixed: Too many files in the sandbox make file event listener fail
  • Fixed: CloverDataWriter produces invalid clover data format when appending
  • Fixed: Persistence of A1digest for HTTPDigestAuthentication should be configurable
  • Fixed: View data on component using SFTP URL throws exception
  • Fixed: SandboxPermission should be deleted with UserGroup
  • Fixed: Log4j error message when starting Server
  • Fixed: Exceptions in server's command line when selecting Group assignment
Posted by Jaroslav Urban 2015-01-27

New CloverETL 3.5.4

Bugfix release CloverETL 3.5.4

  • Improved support for multipart entities in HttpConnector

Bugfixes:

  • Fixed: multipart entities mapping does not work
  • Fixed: DBOutputTable fails on syntax error when using multiple quotes
  • Minor bugfixes
Posted by Jan Sedlacek 2015-01-21

New CloverETL 4.0.2

Bugfix release CloverETL 4.0.2

  • Changed priority of edge-based explicit metadata propagation

Bugfixes:

  • Fixed: Tracking on edges with no record
  • Minor bugfixes
Posted by Jan Sedlacek 2014-12-18

New CloverETL 3.5.3

Bugfix release CloverETL 3.5.3

Bugfixes:

  • Fixed: SortWithinGroup was losing records
  • Fixed: In SQL query everything after 'returning' string is cut off
  • Fixed: UniversalDataWriter was producing wrong delimiters if used with autofilled fields
  • Fixed: XMLWriter fails with "Unexpected null value"
  • Fixed: Fixed-length metadata trim string fields
  • Fixed: UniversalDataWriter: fixed-length fields exclusion issue
  • Fixed: "Create directories" does not work in UniversalDataWriter when using Dictionary as file URL
Posted by Jan Sedlacek 2014-12-11

New CloverETL 4.0.0

Production release CloverETL 4.0.0 - contains all features of its 2 milestones and also:

Improvements:

  • Explicit metadata propagation
  • Secure paramers support in local projects
  • TableauWriter component
  • Open a running or finished job in Designer
  • Updated CloverDataReader and CloverDataWriter components
  • Loop support in ETL graphs, "Token logging" attribute of Loop
  • and more ...

Fixes: ... read more

Posted by Jaroslav Urban 2014-11-13

New CloverETL 3.5.2

Bugfix release CloverETL 3.5.2

Improvements:

  • DBOutputTable supports RETURNING for PostgreSQL

Bugfixes:

  • Fixed: "where" in sql comment when used in Dynamic metadata query
  • Fixed: SystemExecute was closing error file descriptor during execution
  • Fixed: DBLookupTable had incorrectly synchronized activeLookups variable

Compatibility:

  • Fixed: dash ("-") was passing through isNumber function as a number
Posted by Jaroslav Urban 2014-08-25

Milestone CloverETL 4.0.0-M2 released

2nd milestone release of CloverETL 4.0.0

New Features & Improvements:

  • new Runtime in Designer
  • subgraphs can be developed, used and executed in Designer
  • subgraphs can be configured as a component
  • improved subgraph editor
  • extended model of graph parameters (public parameters, CTL2 in parameter value, etc)
  • Designer is bundled with Eclipse 4.3
  • Server supports Tomcat7

Fixes:

  • Fixed when sanboxes.home was not accessible during initial deploy, Server didn't start after restart
  • Fixed asterisk in Samba URL causing unexpected behavior
  • Fixed AddressDoctor configuration dialogs missing some options
  • Fixed updates to properties not reflected unless confirmed by clicking somewhere else (on OSX)
  • Fixed job type icon sometimes not displayed in Server Console... read more
Posted by Jaroslav Urban 2014-08-01

New CloverETL 3.5.1

Bugfix release CloverETL 3.5.1

Improvements:

  • When parsing deprecated format of parameters, ignore those with invalid names

Bugfixes:

  • SimpleGather performance regression
  • CTL code in parameter value does not work when used as a DB password
  • Vertica meta-function execution fails
  • XMLDataWriter behaves inconsistently with long strings
  • Debug files are always created when running graph via RunGraph
  • DBExecute does not report on error port
  • Pivot produces wrong data for string key
  • CTL2: function removeNonPrintable() incorrectly removes also all non-ASCII characters (unicode)
Posted by Pavel Najvar 2014-05-09

Milestone CloverETL 4.0.0-M1 released

Milestone release of CloverETL 4.0.0

New Features & Improvements:

  • Subgraphs
  • Automatic metadata propagation
  • Java 1.7 support (java 1.6 is not compatible any more)
  • Update of used libraries
  • Updated list of application containers supported by server

Fixes:

  • Consistency of string CTL2 functions
  • Improved logging of UniversalDataReader
  • Email Reader: CC value propagation to following records... read more
Posted by Pavel Najvar 2014-04-14

New CloverETL 3.5.0

Production release CloverETL 3.5.0 - contains all features of its 2 milestones and also:

  • Hadoop connectivity in Community Edition
  • Validator improvements
  • Server UI polishing
  • Vertica support
  • Encrypted Server configuration
Posted by Pavel Najvar 2014-01-23

New CloverETL 3.4.3

Bugfix release CloverETL 3.4.3

Bugfixes:

  • Component status and number of records on the edge are not shown after joiner that has input edges in different phases
  • Insufficient error message while reading CSV files with header across two rows
  • Insufficient and unclear error message while reading CSV files with incorrect encoding
  • HTTPConnector changes EOL delimiters in response content
  • Null value being saved into db table field of time type
  • Hadoop engine plugin is registered twice in the GUI
  • CTL - Clicking the ".in." item is inserted invalid value
  • Usage of a sequence in TE is falsely reported as invalid
  • Not possible to move components on top of each other
  • java.lang.IndexOutOfBoundsException: while opening SpreadsheetDataReader GUI
  • "Inc/Des phase by" feature does not work properly
  • Help in the Regex tester of TransformerEditor overlaps the input
  • Message "No data to display" is displayed more than once and sometimes not whole
  • Incorrect modifier for dragging notes without components on Mac
Posted by Pavel Najvar 2013-11-29

Milestone CloverETL 3.5.0-M2

Second milestone release of CloverETL 3.5.0

New Features & Improvements:

  • Validator component
  • Encrypted parameters
  • JSONExtract component
  • Improved Server health monitoring
  • Improved Server usability

Fixes:

  • Fix-length file parsing fixed when metadata contains autofilling fields as last fields
  • JavaBeanReader performance improved by using "nodeName" direct access rather than XPath
  • SpreadsheetDataReader mapping dialog was not opening for corrupt style information in Excel files. (java.lang.IndexOutOfBoundsException fixed)
  • Fixed file operations paths: relative vs. absolute path for remote location – see attached issue for explanation
  • Function removeNonPrintable() no longer removes non-ASCII accented characters (e.g. chars ä, ö, etc.)
  • Fixed JSONReader: fields containing underscores were not populated in implicit mapping
  • HTML graph export – fixed performance and star ($out.0. = $in.0.) mapping redering
  • Removed limitation for "maximum cache size" in the lookup dialog
  • Fixed Server not being able to process a task (in a listener or scheduled) without "clustering" enabled... read more
Posted by Pavel Najvar 2013-10-31

New CloverETL 3.4.2

Bugfix release CloverETL 3.4.2

Improvements:

  • Fixes of graph to HTML export
  • EmailSender connection to SMTP server is initialized in specific phase
  • Upgrade JSch library to 0.1.50

Bugfixes:

  • Long profiler jobs on server fail due to unsuitable default JDBC pool settings
  • Problems with DBExecute component
  • Frozen edit component dialog
  • Stopping "view data" freezes Designer
  • PostgreSQLDataWriter gets stuck on windows machine
  • Trying to display component's tooltip fails with exception
  • Hadoop is missing in opensource engine build
  • Cannot delete generated html
  • SpreadsheetDataReader is not able to process xls file with Custom format cells
  • A graph with DBLookupTable used in a CTL code is not restartable in batch mode
  • Cannot swith to VTE after deleting invalid CTL code
Posted by Pavel Najvar 2013-09-19

Milestone CloverETL 3.5.0-M1

First milestone release of CloverETL 3.5.X:
- Import/export server configuration
- Cluster event listeners failover improvements
- Cluster status monitoring improvements
- MongoDB support
- Windows share support in readers, writers and file operations
- Improved metadata extraction for flat files TODO
- Improved timezones support
- New component Loop
- New component Sleep

Posted by Pavel Najvar 2013-08-13

New CloverETL 3.4.1

Bugfix release CloverETL 3.4.1

Improvements:

  • Export graph to HTML improvements
  • Implement two Time fields in Executions/Tasks history tabs as keyboard editable

Bugfixes:

  • Jobs on server can get stuck and cannot be killed via server gui
  • Data_Intersection component drops data silently
  • Graph description property is named "CloverETL Designer version"
  • NPE occurs when data flows from JSONReader to ExtHashJoin and both components are in the same phase
  • Checkconfig on DBOutputTable causes JNDI connection not released
  • SORT_WITHIN_GROUPS outputs more records than records on input
  • Lookup table can't read quoted data
  • XmlReader: when port is not selected NPE is thrown
  • A graph with DBLookupTable used in a CTL code is not restartable in batch mode
  • Cannot swith to VTE after deleting invalid CTL code
Posted by Pavel Najvar 2013-07-03

New CloverETL 3.4.0

Production release CloverETL 3.4.0 - contains all the features of its 2 milestones, and also:

  • Only one cluster node needs connection to server database - other nodes will connect through it
  • ClusterSimpleCopy component
  • Improved visualization of errors in graph log
  • HTTPConnector supports PUT and DELETE methods
Posted by Pavel Najvar 2013-05-15

Milestone CloverETL 3.4.0-M2

Second milestone release of CloverETL 3.4.X:

* Designer DB connections proxied through Server
* New Server web interface
* ExecuteMapReduce component
* Repartitioning & improved cluster tracking
* XSLT2.0 in XSLTransformer
* Weblogic 12c support on Server
* Additional usability improvements of Designer
* Profiler improved integration with Server

Posted by Pavel Najvar 2013-03-21

New CloverETL 3.3.1

Bugfix release:

* cpu load in graph tracking displayed as % (e.g. 25%)
- XMLExtract: autofilling does not work
- SystemExecute: Do not delete temp file if component fails
- Recapacity of this CloverBuffer is not allowed.
- CTLUtils class cannot be used in transformations
- Incorrect last modification date on SFTP
- Remove org.jetel.license dependency from org.jetel.component plugin

Posted by Pavel Najvar 2013-01-30

Milestone CloverETL 3.4.0-M1

First milestone release of CloverETL 3.4.X:
* precise cluster allocation
* dynamic load balancing in graphs and cluster
* big data support via Hadoop hdfs and Hive
* Data Profiler integration on the server
* usability improvements in Designer

Posted by Pavel Najvar 2012-12-19

Milestone CloverETL 3.3.0-M1

CloverETL is not being released in short milestones before the production release, making latest features readily available as soon as possible
* Introduction of the new Excel SpreadsheetDataReader/Writer components (commercial only)
* ComplexDataReader supports now regular expressions for data decisions (selector) commercial only
* MS Access/Generic ODBC improved
* Designer updates:
* Graph elements locking
* Metadata grouping in outline
* View data improvements

Posted by Pavel Najvar 2012-03-12