From: <zep...@us...> - 2007-06-27 15:35:36
|
Revision: 325 http://flatpack.svn.sourceforge.net/flatpack/?rev=325&view=rev Author: zepernick Date: 2007-06-27 08:35:28 -0700 (Wed, 27 Jun 2007) Log Message: ----------- Dropped the "PZ" from class names Changed the "PZ" to "FP" where class name may be common, such as, StringUtils. This will prevent the user from having to qualify the class name in certain situations Modified Paths: -------------- trunk/flatpack/src/main/java/net/sf/flatpack/DefaultDataSet.java trunk/flatpack/src/main/java/net/sf/flatpack/converter/ConvertBigDecimal.java trunk/flatpack/src/main/java/net/sf/flatpack/converter/ConvertDouble.java trunk/flatpack/src/main/java/net/sf/flatpack/converter/ConvertInteger.java trunk/flatpack/src/main/java/net/sf/flatpack/ordering/OrderBy.java trunk/flatpack/src/main/java/net/sf/flatpack/util/ExcelTransformer.java trunk/flatpack/src/main/java/net/sf/flatpack/util/FixedWidthParserUtils.java trunk/flatpack/src/main/java/net/sf/flatpack/util/ParserUtils.java trunk/flatpack/src/main/java/net/sf/flatpack/xml/XMLRecordElement.java Added Paths: ----------- trunk/flatpack/src/main/java/net/sf/flatpack/AbstractDelimiterParser.java trunk/flatpack/src/main/java/net/sf/flatpack/AbstractFixedLengthParser.java trunk/flatpack/src/main/java/net/sf/flatpack/AbstractParser.java trunk/flatpack/src/main/java/net/sf/flatpack/DBDelimiterParser.java trunk/flatpack/src/main/java/net/sf/flatpack/DBFixedLengthParser.java trunk/flatpack/src/main/java/net/sf/flatpack/DefaultParserFactory.java trunk/flatpack/src/main/java/net/sf/flatpack/DelimiterParser.java trunk/flatpack/src/main/java/net/sf/flatpack/FixedLengthParser.java trunk/flatpack/src/main/java/net/sf/flatpack/Parser.java trunk/flatpack/src/main/java/net/sf/flatpack/ParserFactory.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderDataSet.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderDelimParser.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderFixedParser.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderParseFactory.java trunk/flatpack/src/main/java/net/sf/flatpack/converter/Converter.java trunk/flatpack/src/main/java/net/sf/flatpack/converter/FPConvertException.java trunk/flatpack/src/main/java/net/sf/flatpack/util/FPConstants.java trunk/flatpack/src/main/java/net/sf/flatpack/util/FPStringUtils.java trunk/flatpack/src/main/java/net/sf/flatpack/xml/MapParser.java trunk/flatpack/src/main/java/net/sf/flatpack/xml/MetaData.java Removed Paths: ------------- trunk/flatpack/src/main/java/net/sf/flatpack/AbstractDelimiterPZParser.java trunk/flatpack/src/main/java/net/sf/flatpack/AbstractFixedLengthPZParser.java trunk/flatpack/src/main/java/net/sf/flatpack/AbstractPZParser.java trunk/flatpack/src/main/java/net/sf/flatpack/DBDelimiterPZParser.java trunk/flatpack/src/main/java/net/sf/flatpack/DBFixedLengthPZParser.java trunk/flatpack/src/main/java/net/sf/flatpack/DefaultPZParserFactory.java trunk/flatpack/src/main/java/net/sf/flatpack/DelimiterPZParser.java trunk/flatpack/src/main/java/net/sf/flatpack/FixedLengthPZParser.java trunk/flatpack/src/main/java/net/sf/flatpack/PZParser.java trunk/flatpack/src/main/java/net/sf/flatpack/PZParserFactory.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderDelimPZParser.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderFixedPZParser.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderPZDataSet.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderPZParseFactory.java trunk/flatpack/src/main/java/net/sf/flatpack/converter/PZConvertException.java trunk/flatpack/src/main/java/net/sf/flatpack/converter/PZConverter.java trunk/flatpack/src/main/java/net/sf/flatpack/util/PZConstants.java trunk/flatpack/src/main/java/net/sf/flatpack/util/PZStringUtils.java trunk/flatpack/src/main/java/net/sf/flatpack/xml/PZMapParser.java trunk/flatpack/src/main/java/net/sf/flatpack/xml/PZMetaData.java This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <be...@us...> - 2007-07-09 10:56:10
|
Revision: 330 http://flatpack.svn.sourceforge.net/flatpack/?rev=330&view=rev Author: benoitx Date: 2007-07-09 03:56:12 -0700 (Mon, 09 Jul 2007) Log Message: ----------- added toString methods. Modified Paths: -------------- trunk/flatpack/src/main/java/net/sf/flatpack/AbstractDelimiterParser.java trunk/flatpack/src/main/java/net/sf/flatpack/DefaultDataSet.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderDataSet.java This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <be...@us...> - 2007-07-09 10:56:34
|
Revision: 331 http://flatpack.svn.sourceforge.net/flatpack/?rev=331&view=rev Author: benoitx Date: 2007-07-09 03:56:35 -0700 (Mon, 09 Jul 2007) Log Message: ----------- added toString methods. Modified Paths: -------------- trunk/flatpack/src/main/java/net/sf/flatpack/structure/ColumnMetaData.java trunk/flatpack/src/main/java/net/sf/flatpack/xml/MetaData.java This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <zep...@us...> - 2007-07-30 18:09:18
|
Revision: 335 http://flatpack.svn.sourceforge.net/flatpack/?rev=335&view=rev Author: zepernick Date: 2007-07-30 11:09:18 -0700 (Mon, 30 Jul 2007) Log Message: ----------- added clearRows(), clearErrors(), and clearAll() to DataSet Modified Paths: -------------- trunk/flatpack/src/main/java/net/sf/flatpack/DataSet.java trunk/flatpack/src/main/java/net/sf/flatpack/DefaultDataSet.java This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <be...@us...> - 2007-07-31 22:21:10
|
Revision: 338 http://flatpack.svn.sourceforge.net/flatpack/?rev=338&view=rev Author: benoitx Date: 2007-07-31 15:21:10 -0700 (Tue, 31 Jul 2007) Log Message: ----------- Bit of tidy up. the doParse method ought to be protected as it is invalid to call directly (called through parse method). Modified Paths: -------------- trunk/flatpack/src/main/java/net/sf/flatpack/AbstractDelimiterParser.java trunk/flatpack/src/main/java/net/sf/flatpack/AbstractFixedLengthParser.java trunk/flatpack/src/main/java/net/sf/flatpack/AbstractParser.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderDelimParser.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderFixedParser.java Modified: trunk/flatpack/src/main/java/net/sf/flatpack/AbstractDelimiterParser.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/AbstractDelimiterParser.java 2007-07-30 20:22:12 UTC (rev 337) +++ trunk/flatpack/src/main/java/net/sf/flatpack/AbstractDelimiterParser.java 2007-07-31 22:21:10 UTC (rev 338) @@ -57,30 +57,6 @@ private final Logger logger = LoggerFactory.getLogger(AbstractDelimiterParser.class); - /*public AbstractDelimiterPZParser(final InputStream dataSourceStream, final String dataDefinition, final char delimiter, - final char qualifier, final boolean ignoreFirstRecord) { - super(dataSourceStream, dataDefinition); - this.delimiter = delimiter; - this.qualifier = qualifier; - this.ignoreFirstRecord = ignoreFirstRecord; - } - - public AbstractDelimiterPZParser(final File dataSource, final char delimiter, final char qualifier, - final boolean ignoreFirstRecord) { - super(dataSource); - this.delimiter = delimiter; - this.qualifier = qualifier; - this.ignoreFirstRecord = ignoreFirstRecord; - } - - public AbstractDelimiterPZParser(final InputStream dataSourceStream, final char delimiter, final char qualifier, - final boolean ignoreFirstRecord) { - super(dataSourceStream); - this.delimiter = delimiter; - this.qualifier = qualifier; - this.ignoreFirstRecord = ignoreFirstRecord; - }*/ - public AbstractDelimiterParser(final Reader dataSourceReader, final String dataDefinition, final char delimiter, final char qualifier, final boolean ignoreFirstRecord) { super(dataSourceReader, dataDefinition); @@ -96,23 +72,10 @@ this.ignoreFirstRecord = ignoreFirstRecord; } - public DataSet doParse() { + protected DataSet doParse() { try { lineCount = 0; return doDelimitedFile(getDataSourceReader(), shouldCreateMDFromFile()); - /* if (getDataSourceStream() != null) { - return doDelimitedFile(getDataSourceStream(), shouldCreateMDFromFile()); - } else { - InputStream stream = null; - try { - stream = ParserUtils.createInputStream(getDataSource()); - return doDelimitedFile(stream, shouldCreateMDFromFile()); - } finally { - if (stream != null) { - stream.close(); - } - } - }*/ } catch (final IOException e) { logger.error("error accessing/creating inputstream", e); } @@ -161,15 +124,11 @@ throw new NullPointerException("dataSource is null"); } BufferedReader br = null; - // final DefaultDataSet ds = new DefaultDataSet(getColumnMD(), this); final DefaultDataSet ds = new DefaultDataSet(getPzMetaData(), this); try { //gather the conversion properties ds.setPZConvertProps(ParserUtils.loadConvertProperties()); - // get the total column count - // columnCount = columnMD.size(); - br = new BufferedReader(dataSource); boolean processedFirst = false; @@ -182,9 +141,7 @@ continue; } else if (!processedFirst && createMDFromFile) { processedFirst = true; - // setColumnMD(ParserUtils.getColumnMDFromFile(line, delimiter, qualifier, this)); setPzMetaData(ParserUtils.getPZMetaDataFromFile(line, delimiter, qualifier, this)); - // ds.setColumnMD(getColumnMD()); ds.setMetaData(getPzMetaData()); continue; } @@ -192,8 +149,6 @@ // column values List columns = ParserUtils.splitLine(line, getDelimiter(), getQualifier(), FPConstants.SPLITLINE_SIZE_INIT); final String mdkey = ParserUtils.getCMDKeyForDelimitedFile(getPzMetaData(), columns); - // final String mdkey = ParserUtils.getCMDKeyForDelimitedFile(getColumnMD(), columns); - // final List cmds = ParserUtils.getColumnMetaData(mdkey, getColumnMD()); final List cmds = ParserUtils.getColumnMetaData(mdkey, getPzMetaData()); final int columnCount = cmds.size(); Modified: trunk/flatpack/src/main/java/net/sf/flatpack/AbstractFixedLengthParser.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/AbstractFixedLengthParser.java 2007-07-30 20:22:12 UTC (rev 337) +++ trunk/flatpack/src/main/java/net/sf/flatpack/AbstractFixedLengthParser.java 2007-07-31 22:21:10 UTC (rev 338) @@ -53,22 +53,6 @@ public abstract class AbstractFixedLengthParser extends AbstractParser { private final Logger logger = LoggerFactory.getLogger(AbstractFixedLengthParser.class); - /* protected AbstractFixedLengthPZParser(final File dataSource, final String dataDefinition) { - super(dataSource, dataDefinition); - } - - protected AbstractFixedLengthPZParser(final File dataSource) { - super(dataSource); - } - - protected AbstractFixedLengthPZParser(final InputStream dataSourceStream, final String dataDefinition) { - super(dataSourceStream, dataDefinition); - } - - protected AbstractFixedLengthPZParser(final InputStream dataSourceStream) { - super(dataSourceStream); - }*/ - protected AbstractFixedLengthParser(final Reader dataSourceReader, final String dataDefinition) { super(dataSourceReader, dataDefinition); } @@ -77,21 +61,8 @@ super(dataSourceReader); } - public DataSet doParse() { + protected DataSet doParse() { try { - /* if (getDataSourceStream() != null) { - return doFixedLengthFile(getDataSourceStream()); - } else { - InputStream stream; - stream = ParserUtils.createInputStream(getDataSource()); - try { - return doFixedLengthFile(stream); - } finally { - if (stream != null) { - stream.close(); - } - } - }*/ return doFixedLengthFile(getDataSourceReader()); } catch (final IOException e) { logger.error("error accessing/reading data", e); @@ -109,14 +80,12 @@ private DataSet doFixedLengthFile(final Reader dataSource) throws IOException { BufferedReader br = null; - // final DefaultDataSet ds = new DefaultDataSet(getColumnMD(), this); final DefaultDataSet ds = new DefaultDataSet(getPzMetaData(), this); try { //gather the conversion properties ds.setPZConvertProps(ParserUtils.loadConvertProperties()); - // final Map recordLengths = ParserUtils.calculateRecordLengths(getColumnMD()); final Map recordLengths = ParserUtils.calculateRecordLengths(getPzMetaData()); // Read in the flat file @@ -134,7 +103,6 @@ } final String mdkey = FixedWidthParserUtils.getCMDKey(getPzMetaData(), line); - // final String mdkey = FixedWidthParserUtils.getCMDKey(getColumnMD(), line); final int recordLength = ((Integer) recordLengths.get(mdkey)).intValue(); if (line.length() > recordLength) { @@ -168,18 +136,8 @@ final Row row = new Row(); row.setMdkey(mdkey.equals(FPConstants.DETAIL_ID) ? null : mdkey); // try - // final List cmds = ParserUtils.getColumnMetaData(mdkey, getColumnMD()); final List cmds = ParserUtils.getColumnMetaData(mdkey, getPzMetaData()); row.addColumn(FixedWidthParserUtils.splitFixedText(cmds, line)); - // to limit the memory use - // Build the columns for the row - // for (int i = 0; i < cmds.size(); i++) { - // final String tempValue = line.substring(recPosition - 1, - // recPosition - // + (((ColumnMetaData) cmds.get(i)).getColLength() - 1)); - // recPosition += ((ColumnMetaData) cmds.get(i)).getColLength(); - // row.addColumn(tempValue.trim()); - // } row.setRowNumber(lineCount); // add the row to the array ds.addRow(row); Modified: trunk/flatpack/src/main/java/net/sf/flatpack/AbstractParser.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/AbstractParser.java 2007-07-30 20:22:12 UTC (rev 337) +++ trunk/flatpack/src/main/java/net/sf/flatpack/AbstractParser.java 2007-07-31 22:21:10 UTC (rev 338) @@ -145,13 +145,6 @@ } } - // protected void addToColumnMD(final Object key, final Object value) { - // if (columnMD == null) { - // columnMD = new LinkedHashMap(); - // } - // columnMD.put(key, value); - // } - protected boolean isInitialised() { return initialised; } @@ -168,25 +161,6 @@ this.dataDefinition = dataDefinition; } - /* protected File getDataSource() { - return dataSource; - } - - protected void setDataSource(final File dataSource) { - this.dataSource = dataSource; - } - - protected InputStream getDataSourceStream() { - return dataSourceStream; - } - - protected void setDataSourceStream(final InputStream dataSourceStream) { - this.dataSourceStream = dataSourceStream; - }*/ - - // protected Map getColumnMD() { - // return columnMD; - // } /** * Adds a new error to this DataSet. These can be collected, and retreived * after processing @@ -252,5 +226,4 @@ public void setPzMetaData(final MetaData pzMap) { this.pzMetaData = pzMap; } - } Modified: trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderDelimParser.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderDelimParser.java 2007-07-30 20:22:12 UTC (rev 337) +++ trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderDelimParser.java 2007-07-31 22:21:10 UTC (rev 338) @@ -83,7 +83,7 @@ super(dataSourceStream, delimiter, qualifier, ignoreFirstRecord); } - public DataSet doParse() { + protected DataSet doParse() { // final DataSet ds = new BuffReaderPZDataSet(getColumnMD(), this); final DataSet ds = new BuffReaderDataSet(getPzMetaData(), this); try { Modified: trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderFixedParser.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderFixedParser.java 2007-07-30 20:22:12 UTC (rev 337) +++ trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderFixedParser.java 2007-07-31 22:21:10 UTC (rev 338) @@ -72,11 +72,9 @@ super(pzmapXML, dataSource); } - public DataSet doParse() { - // final DataSet ds = new BuffReaderPZDataSet(getColumnMD(), this); + protected DataSet doParse() { final DataSet ds = new BuffReaderDataSet(getPzMetaData(), this); lineCount = 0; - // recordLengths = ParserUtils.calculateRecordLengths(getColumnMD()); recordLengths = ParserUtils.calculateRecordLengths(getPzMetaData()); try { //gather the conversion properties @@ -110,7 +108,6 @@ continue; } - // final String mdkey = FixedWidthParserUtils.getCMDKey(getColumnMD(), line); final String mdkey = FixedWidthParserUtils.getCMDKey(getPzMetaData(), line); final int recordLength = ((Integer) recordLengths.get(mdkey)).intValue(); @@ -144,7 +141,6 @@ final Row row = new Row(); row.setMdkey(mdkey.equals(FPConstants.DETAIL_ID) ? null : mdkey); - // final List cmds = ParserUtils.getColumnMetaData(mdkey, getColumnMD()); final List cmds = ParserUtils.getColumnMetaData(mdkey, getPzMetaData()); row.addColumn(FixedWidthParserUtils.splitFixedText(cmds, line)); This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <be...@us...> - 2007-08-20 21:49:17
|
Revision: 342 http://flatpack.svn.sourceforge.net/flatpack/?rev=342&view=rev Author: benoitx Date: 2007-08-20 14:49:18 -0700 (Mon, 20 Aug 2007) Log Message: ----------- Fix a few PMD and Checkstyle warnings. Modified Paths: -------------- trunk/flatpack/src/main/java/net/sf/flatpack/DataSet.java trunk/flatpack/src/main/java/net/sf/flatpack/ParserFactory.java trunk/flatpack/src/main/java/net/sf/flatpack/util/FPStringUtils.java trunk/flatpack/src/main/java/net/sf/flatpack/xml/XMLRecordElement.java Modified: trunk/flatpack/src/main/java/net/sf/flatpack/DataSet.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/DataSet.java 2007-08-17 18:59:40 UTC (rev 341) +++ trunk/flatpack/src/main/java/net/sf/flatpack/DataSet.java 2007-08-20 21:49:18 UTC (rev 342) @@ -36,7 +36,6 @@ import java.text.SimpleDateFormat; import java.util.Date; import java.util.List; -import java.util.NoSuchElementException; import java.util.Properties; import net.sf.flatpack.ordering.OrderBy; Modified: trunk/flatpack/src/main/java/net/sf/flatpack/ParserFactory.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/ParserFactory.java 2007-08-17 18:59:40 UTC (rev 341) +++ trunk/flatpack/src/main/java/net/sf/flatpack/ParserFactory.java 2007-08-20 21:49:18 UTC (rev 342) @@ -244,7 +244,8 @@ * The File can be wrapped in a "new FileReader(File)" * @return PZParser */ - Parser newDelimitedParser(final File pzmapXML, final File dataSource, final char delimiter, final char qualifier, final boolean ignoreFirstRecord); + Parser newDelimitedParser(final File pzmapXML, final File dataSource, final char delimiter, final char qualifier, + final boolean ignoreFirstRecord); /** * Constructs a new DataSet using the PZMAP XML file layout method. This is Modified: trunk/flatpack/src/main/java/net/sf/flatpack/util/FPStringUtils.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/util/FPStringUtils.java 2007-08-17 18:59:40 UTC (rev 341) +++ trunk/flatpack/src/main/java/net/sf/flatpack/util/FPStringUtils.java 2007-08-20 21:49:18 UTC (rev 342) @@ -5,7 +5,11 @@ * * @author Jakarta Commons */ -public class FPStringUtils { +public final class FPStringUtils { + private FPStringUtils() { + + } + /** * <p>Checks if a String is whitespace, empty ("") or null.</p> * Modified: trunk/flatpack/src/main/java/net/sf/flatpack/xml/XMLRecordElement.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/xml/XMLRecordElement.java 2007-08-17 18:59:40 UTC (rev 341) +++ trunk/flatpack/src/main/java/net/sf/flatpack/xml/XMLRecordElement.java 2007-08-20 21:49:18 UTC (rev 342) @@ -113,13 +113,13 @@ } /** - * @param columns + * @param columnsToUse * The columns to set. * @param p * PZParser being used. Can be null. */ - public void setColumns(final List columns, final Parser p) { - this.columns = columns; + public void setColumns(final List columnsToUse, final Parser p) { + this.columns = columnsToUse; this.columnIndex = ParserUtils.buidColumnIndexMap(columns, p); } This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <be...@us...> - 2007-08-30 10:39:43
|
Revision: 343 http://flatpack.svn.sourceforge.net/flatpack/?rev=343&view=rev Author: benoitx Date: 2007-08-30 03:39:40 -0700 (Thu, 30 Aug 2007) Log Message: ----------- Tidy up (Checkstyle & co) Modified Paths: -------------- trunk/flatpack/src/main/java/net/sf/flatpack/DefaultDataSet.java trunk/flatpack/src/main/java/net/sf/flatpack/ParserFactory.java trunk/flatpack/src/main/java/net/sf/flatpack/util/FPStringUtils.java Modified: trunk/flatpack/src/main/java/net/sf/flatpack/DefaultDataSet.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/DefaultDataSet.java 2007-08-20 21:49:18 UTC (rev 342) +++ trunk/flatpack/src/main/java/net/sf/flatpack/DefaultDataSet.java 2007-08-30 10:39:40 UTC (rev 343) @@ -259,24 +259,20 @@ * @see net.sf.flatpack.IDataSet#getString(java.lang.String) */ public String getString(final String column) { - final String s = getStringValue(column); + String s = getStringValue(column); if (parser.isNullEmptyStrings() && FPStringUtils.isBlank(s)) { - return null; - } - - if (upperCase) { + s = null; + } else if (upperCase) { // convert data to uppercase before returning // return row.getValue(ParserUtils.findColumn(column, // cmds)).toUpperCase(Locale.getDefault()); - return s.toUpperCase(Locale.getDefault()); - } - - if (lowerCase) { + s = s.toUpperCase(Locale.getDefault()); + }else if (lowerCase) { // convert data to lowercase before returning // return row.getValue(ParserUtils.findColumn(column, // cmds)).toLowerCase(Locale.getDefault()); - return s.toLowerCase(Locale.getDefault()); + s = s.toLowerCase(Locale.getDefault()); } // return value as how it is in the file @@ -450,19 +446,18 @@ this.pointer = pointer; } - public void clearRows() { pointer = -1; //set the pointer back to -1 directly just in case this instance is a BuffReaderDataSet. rows.clear(); } - + public void clearAll() { clearRows(); clearErrors(); } - + public void clearErrors() { - errors.clear(); + errors.clear(); } public MetaData getMetaData() { @@ -482,21 +477,21 @@ buf.append("MetaData:").append(metaData).append(System.getProperty("line.separator")); return buf.toString(); } - + public boolean contains(String column) { if (pointer == -1) { throw new IndexOutOfBoundsException("dataset on invalid row. need to call next()"); - } - - final Iterator cmds = ParserUtils.getColumnMetaData(((Row) rows.get(pointer)).getMdkey(), metaData).iterator(); + } + + final Iterator cmds = ParserUtils.getColumnMetaData(((Row) rows.get(pointer)).getMdkey(), metaData).iterator(); while (cmds.hasNext()) { - final ColumnMetaData cmd = (ColumnMetaData)cmds.next(); + final ColumnMetaData cmd = (ColumnMetaData) cmds.next(); if (cmd.getColName().equalsIgnoreCase(column)) { return true; } } - + return false; - + } } Modified: trunk/flatpack/src/main/java/net/sf/flatpack/ParserFactory.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/ParserFactory.java 2007-08-20 21:49:18 UTC (rev 342) +++ trunk/flatpack/src/main/java/net/sf/flatpack/ParserFactory.java 2007-08-30 10:39:40 UTC (rev 343) @@ -218,8 +218,8 @@ * skips the first line that contains data in the file * @return PZParser */ - Parser newDelimitedParser(final Connection con, final Reader dataSource, final String dataDefinition, final char delimiter, final char qualifier, - final boolean ignoreFirstRecord); + Parser newDelimitedParser(final Connection con, final Reader dataSource, final String dataDefinition, final char delimiter, + final char qualifier, final boolean ignoreFirstRecord); /** * Constructs a new DataSet using the PZMAP XML file layout method. This is @@ -244,7 +244,7 @@ * The File can be wrapped in a "new FileReader(File)" * @return PZParser */ - Parser newDelimitedParser(final File pzmapXML, final File dataSource, final char delimiter, final char qualifier, + Parser newDelimitedParser(final File pzmapXML, final File dataSource, final char delimiter, final char qualifier, final boolean ignoreFirstRecord); /** Modified: trunk/flatpack/src/main/java/net/sf/flatpack/util/FPStringUtils.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/util/FPStringUtils.java 2007-08-20 21:49:18 UTC (rev 342) +++ trunk/flatpack/src/main/java/net/sf/flatpack/util/FPStringUtils.java 2007-08-30 10:39:40 UTC (rev 343) @@ -7,9 +7,8 @@ */ public final class FPStringUtils { private FPStringUtils() { - } - + /** * <p>Checks if a String is whitespace, empty ("") or null.</p> * This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <zep...@us...> - 2007-11-23 14:08:39
|
Revision: 380 http://flatpack.svn.sourceforge.net/flatpack/?rev=380&view=rev Author: zepernick Date: 2007-11-23 06:08:40 -0800 (Fri, 23 Nov 2007) Log Message: ----------- added an option to store the raw data in the DataError object when a parse error occurs Modified Paths: -------------- trunk/flatpack/src/main/java/net/sf/flatpack/AbstractDelimiterParser.java trunk/flatpack/src/main/java/net/sf/flatpack/AbstractFixedLengthParser.java trunk/flatpack/src/main/java/net/sf/flatpack/AbstractParser.java trunk/flatpack/src/main/java/net/sf/flatpack/DataError.java trunk/flatpack/src/main/java/net/sf/flatpack/Parser.java trunk/flatpack/src/main/java/net/sf/flatpack/Version.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderDelimParser.java trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderFixedParser.java Modified: trunk/flatpack/src/main/java/net/sf/flatpack/AbstractDelimiterParser.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/AbstractDelimiterParser.java 2007-11-17 16:15:41 UTC (rev 379) +++ trunk/flatpack/src/main/java/net/sf/flatpack/AbstractDelimiterParser.java 2007-11-23 14:08:40 UTC (rev 380) @@ -165,7 +165,8 @@ columns = columns.subList(0, columnCount); addError(ds, "TRUNCATED LINE TO CORRECT NUMBER OF COLUMNS", lineCount, 1); } else { - addError(ds, "TOO MANY COLUMNS WANTED: " + columnCount + " GOT: " + columns.size(), lineCount, 2); + addError(ds, "TOO MANY COLUMNS WANTED: " + columnCount + " GOT: " + columns.size(), lineCount, 2, + isStoreRawDataToDataError() ? line : null); continue; } } else if (columns.size() < columnCount) { @@ -179,7 +180,8 @@ addError(ds, "PADDED LINE TO CORRECT NUMBER OF COLUMNS", lineCount, 1); } else { - addError(ds, "TOO FEW COLUMNS WANTED: " + columnCount + " GOT: " + columns.size(), lineCount, 2); + addError(ds, "TOO FEW COLUMNS WANTED: " + columnCount + " GOT: " + columns.size(), lineCount, 2, + isStoreRawDataToDataError() ? line : null); continue; } } Modified: trunk/flatpack/src/main/java/net/sf/flatpack/AbstractFixedLengthParser.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/AbstractFixedLengthParser.java 2007-11-17 16:15:41 UTC (rev 379) +++ trunk/flatpack/src/main/java/net/sf/flatpack/AbstractFixedLengthParser.java 2007-11-23 14:08:40 UTC (rev 380) @@ -115,7 +115,8 @@ line = line.substring(0, recordLength); addError(ds, "TRUNCATED LINE TO CORRECT LENGTH", lineCount, 1); } else { - addError(ds, "LINE TOO LONG. LINE IS " + line.length() + " LONG. SHOULD BE " + recordLength, lineCount, 2); + addError(ds, "LINE TOO LONG. LINE IS " + line.length() + " LONG. SHOULD BE " + recordLength, lineCount, 2, + isStoreRawDataToDataError() ? line : null); continue; } } else if (line.length() < recordLength) { @@ -127,7 +128,8 @@ addError(ds, "PADDED LINE TO CORRECT RECORD LENGTH", lineCount, 1); } else { - addError(ds, "LINE TOO SHORT. LINE IS " + line.length() + " LONG. SHOULD BE " + recordLength, lineCount, 2); + addError(ds, "LINE TOO SHORT. LINE IS " + line.length() + " LONG. SHOULD BE " + recordLength, lineCount, 2, + isStoreRawDataToDataError() ? line : null); continue; } } Modified: trunk/flatpack/src/main/java/net/sf/flatpack/AbstractParser.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/AbstractParser.java 2007-11-17 16:15:41 UTC (rev 379) +++ trunk/flatpack/src/main/java/net/sf/flatpack/AbstractParser.java 2007-11-23 14:08:40 UTC (rev 380) @@ -71,6 +71,8 @@ private List readersToClose = null; private boolean flagEmptyRows; + + private boolean storeRawDataToDataError; protected AbstractParser(final Reader dataSourceReader) { this.dataSourceReader = dataSourceReader; @@ -178,23 +180,41 @@ } /** - * Adds a new error to this DataSet. These can be collected, and retreived + * Adds a new error to this DataSet. These can be collected, and retrieved * after processing * - * @param errorDesc - + * @param errorDesc * String description of error - * @param lineNo - - * int line number error occured on - * @param errorLevel - - * int errorLevel 1,2,3 1=warning 2=error 3= severe error + * @param lineNo + * line number error occurred on + * @param errorLevel + * errorLevel 1,2,3 1=warning 2=error 3= severe error */ protected void addError(final DefaultDataSet ds, final String errorDesc, final int lineNo, final int errorLevel) { + addError(ds, errorDesc, lineNo, errorLevel, null); + } + + /** + * Adds a new error to this DataSet. These can be collected, and retrieved + * after processing + * + * @param errorDesc + * String description of error + * @param lineNo + * line number error occurred on + * @param errorLevel + * errorLevel 1,2,3 1=warning 2=error 3= severe error' + * @param lineData + * Data of the line which failed the parse + */ + protected void addError(final DefaultDataSet ds, final String errorDesc, + final int lineNo, final int errorLevel, final String lineData) { if (errorLevel == 1 && isIgnoreParseWarnings()) { // user has selected to not log warnings in the parser return; } - final DataError de = new DataError(errorDesc, lineNo, errorLevel); + final DataError de = new DataError(errorDesc, lineNo, errorLevel, lineData); ds.addError(de); } @@ -259,4 +279,18 @@ public void setFlagEmptyRows(boolean flagEmptyRows) { this.flagEmptyRows = flagEmptyRows; } + + /** + * @return the storeRawDataToDataError + */ + public boolean isStoreRawDataToDataError() { + return storeRawDataToDataError; + } + + /** + * @param storeRawDataToDataError the storeRawDataToDataError to set + */ + public void setStoreRawDataToDataError(boolean storeRawDataToDataError) { + this.storeRawDataToDataError = storeRawDataToDataError; + } } Modified: trunk/flatpack/src/main/java/net/sf/flatpack/DataError.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/DataError.java 2007-11-17 16:15:41 UTC (rev 379) +++ trunk/flatpack/src/main/java/net/sf/flatpack/DataError.java 2007-11-23 14:08:40 UTC (rev 380) @@ -33,7 +33,7 @@ package net.sf.flatpack; /** - * This class holds errors that occured while parsing or processing a data file. + * This class holds errors that occurred while parsing or processing a data file. * * @author Paul Zepernick * @version 2.0 @@ -44,21 +44,39 @@ private final int lineNo; private final int errorLevel; + + private final String rawData; /** * * @param errorDesc - * Text description of the error that occured + * Text description of the error that occurred * @param lineNo - * Line number in the data file the error occured on + * Line number in the data file the error occurred on * @param errorLevel * Level of the error (1=warning, 2=moderate, 3=severe) */ public DataError(final String errorDesc, final int lineNo, final int errorLevel) { - this.errorDesc = errorDesc; - this.lineNo = lineNo; - this.errorLevel = errorLevel; + this(errorDesc, lineNo, errorLevel, null); } + + /** + * + * @param errorDesc + * Text description of the error that occurred + * @param lineNo + * Line number in the data file the error occurred on + * @param errorLevel + * Level of the error (1=warning, 2=moderate, 3=severe) + * @param rawData + * String of data which the parse failed on + */ + public DataError(final String errorDesc, final int lineNo, final int errorLevel, final String rawData) { + this.errorDesc = errorDesc; + this.lineNo = lineNo; + this.errorLevel = errorLevel; + this.rawData = rawData; + } /** * Returns the errorDesc. @@ -93,4 +111,14 @@ System.getProperty("line.separator")); return buf.toString(); } + + /** + * Option must be set on parser, otherwise this is + * null by default + * + * @return the rawData + */ + protected String getRawData() { + return rawData; + } } Modified: trunk/flatpack/src/main/java/net/sf/flatpack/Parser.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/Parser.java 2007-11-17 16:15:41 UTC (rev 379) +++ trunk/flatpack/src/main/java/net/sf/flatpack/Parser.java 2007-11-23 14:08:40 UTC (rev 380) @@ -151,4 +151,13 @@ * @param flagEmptyRows */ void setFlagEmptyRows(final boolean flagEmptyRows); + + /** + * when true, the parser will place the data of the line which failed the parse and + * place it into the DataError object. DataError.getRawData() can be called to retrieve + * the line. + * + * @param storeRawDataToDataError + */ + void setStoreRawDataToDataError(final boolean storeRawDataToDataError); } Modified: trunk/flatpack/src/main/java/net/sf/flatpack/Version.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/Version.java 2007-11-17 16:15:41 UTC (rev 379) +++ trunk/flatpack/src/main/java/net/sf/flatpack/Version.java 2007-11-23 14:08:40 UTC (rev 380) @@ -19,5 +19,5 @@ * Static class which stores the version of this FlatPack */ public class Version { - public static final String VERSION = "3.1.1"; + public static final String VERSION = "3.2.0-SNAPSHOT"; } Modified: trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderDelimParser.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderDelimParser.java 2007-11-17 16:15:41 UTC (rev 379) +++ trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderDelimParser.java 2007-11-23 14:08:40 UTC (rev 380) @@ -152,7 +152,8 @@ addError(ds, "TRUNCATED LINE TO CORRECT NUMBER OF COLUMNS", getLineCount(), 1); } else { //log the error - addError(ds, "TOO MANY COLUMNS WANTED: " + columnCount + " GOT: " + columns.size(), getLineCount(), 2); + addError(ds, "TOO MANY COLUMNS WANTED: " + columnCount + " GOT: " + columns.size(), getLineCount(), 2, + isStoreRawDataToDataError() ? line : null); continue; } } else if (columns.size() < columnCount) { @@ -166,7 +167,8 @@ addError(ds, "PADDED LINE TO CORRECT NUMBER OF COLUMNS", getLineCount(), 1); } else { - addError(ds, "TOO FEW COLUMNS WANTED: " + columnCount + " GOT: " + columns.size(), getLineCount(), 2); + addError(ds, "TOO FEW COLUMNS WANTED: " + columnCount + " GOT: " + columns.size(), getLineCount(), 2, + isStoreRawDataToDataError() ? line : null); continue; } } @@ -177,6 +179,11 @@ row.setCols(columns); row.setRowNumber(getLineCount()); + if (isFlagEmptyRows()) { + //user has elected to have the parser flag rows that are empty + row.setEmpty(ParserUtils.isListElementsEmpty(columns)); + } + return row; } } Modified: trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderFixedParser.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderFixedParser.java 2007-11-17 16:15:41 UTC (rev 379) +++ trunk/flatpack/src/main/java/net/sf/flatpack/brparse/BuffReaderFixedParser.java 2007-11-23 14:08:40 UTC (rev 380) @@ -116,12 +116,13 @@ // be included in the // dataset if (isIgnoreExtraColumns()) { - //user has choosen to ignore the fact that we have too many bytes in the fixed + //user has chosen to ignore the fact that we have too many bytes in the fixed //width file. Truncate the line to the correct length line = line.substring(0, recordLength); addError(ds, "TRUNCATED LINE TO CORRECT LENGTH", lineCount, 1); } else { - addError(ds, "LINE TOO LONG. LINE IS " + line.length() + " LONG. SHOULD BE " + recordLength, lineCount, 2); + addError(ds, "LINE TOO LONG. LINE IS " + line.length() + " LONG. SHOULD BE " + recordLength, lineCount, 2, + isStoreRawDataToDataError() ? line : null); continue; } } else if (line.length() < recordLength) { @@ -133,7 +134,8 @@ addError(ds, "PADDED LINE TO CORRECT RECORD LENGTH", lineCount, 1); } else { - addError(ds, "LINE TOO SHORT. LINE IS " + line.length() + " LONG. SHOULD BE " + recordLength, lineCount, 2); + addError(ds, "LINE TOO SHORT. LINE IS " + line.length() + " LONG. SHOULD BE " + recordLength, lineCount, 2, + isStoreRawDataToDataError() ? line : null); continue; } } @@ -145,6 +147,11 @@ row.addColumn(FixedWidthParserUtils.splitFixedText(cmds, line)); row.setRowNumber(lineCount); + + if (isFlagEmptyRows()) { + //user has elected to have the parser flag rows that are empty + row.setEmpty(ParserUtils.isListElementsEmpty(row.getCols())); + } return row; } This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <zep...@us...> - 2008-01-09 19:32:46
|
Revision: 387 http://flatpack.svn.sourceforge.net/flatpack/?rev=387&view=rev Author: zepernick Date: 2008-01-09 11:32:52 -0800 (Wed, 09 Jan 2008) Log Message: ----------- fixed a bug in the database fixed length parser Modified Paths: -------------- trunk/flatpack/src/main/java/net/sf/flatpack/DBFixedLengthParser.java trunk/flatpack/src/main/java/net/sf/flatpack/xml/MetaData.java Modified: trunk/flatpack/src/main/java/net/sf/flatpack/DBFixedLengthParser.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/DBFixedLengthParser.java 2007-11-25 22:55:14 UTC (rev 386) +++ trunk/flatpack/src/main/java/net/sf/flatpack/DBFixedLengthParser.java 2008-01-09 19:32:52 UTC (rev 387) @@ -110,7 +110,7 @@ } } - public DataSet doParse() { - return null; - } +// public DataSet doParse() { +// return null; +// } } Modified: trunk/flatpack/src/main/java/net/sf/flatpack/xml/MetaData.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/xml/MetaData.java 2007-11-25 22:55:14 UTC (rev 386) +++ trunk/flatpack/src/main/java/net/sf/flatpack/xml/MetaData.java 2008-01-09 19:32:52 UTC (rev 387) @@ -4,6 +4,7 @@ package net.sf.flatpack.xml; import java.util.Collections; +import java.util.HashMap; import java.util.Iterator; import java.util.List; import java.util.Map; @@ -28,6 +29,7 @@ public MetaData(final List columnNames, final Map columnIndexMap) { this.columnsNames = Collections.unmodifiableList(columnNames); this.columnIndexMap = Collections.unmodifiableMap(columnIndexMap); + this.xmlRecordElements = new HashMap(); } MetaData(final List columnNames, final Map columnIndexMap, final Map xmlRecordElements) { This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |
From: <zep...@us...> - 2008-03-18 12:31:12
|
Revision: 398 http://flatpack.svn.sourceforge.net/flatpack/?rev=398&view=rev Author: zepernick Date: 2008-03-18 05:31:17 -0700 (Tue, 18 Mar 2008) Log Message: ----------- added getLong() to DataSet Modified Paths: -------------- trunk/flatpack/src/main/java/net/sf/flatpack/DataSet.java trunk/flatpack/src/main/java/net/sf/flatpack/DefaultDataSet.java Modified: trunk/flatpack/src/main/java/net/sf/flatpack/DataSet.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/DataSet.java 2008-03-18 12:30:58 UTC (rev 397) +++ trunk/flatpack/src/main/java/net/sf/flatpack/DataSet.java 2008-03-18 12:31:17 UTC (rev 398) @@ -92,15 +92,26 @@ double getDouble(final String column); /** - * Returns the interger value of a specified column + * Returns the integer value of a specified column * * @param column - * Name of the column * @exception NoSuchElementException * @exception NumberFormatException - * @return double + * @return int */ int getInt(final String column); + + /** + * Returns the long value of a specified column + * + * @param column - + * Name of the column + * @exception NoSuchElementException + * @exception NumberFormatException + * @return long + */ + long getLong(final String column); /** * Returns the date value of a specified column. This assumes the date is in Modified: trunk/flatpack/src/main/java/net/sf/flatpack/DefaultDataSet.java =================================================================== --- trunk/flatpack/src/main/java/net/sf/flatpack/DefaultDataSet.java 2008-03-18 12:30:58 UTC (rev 397) +++ trunk/flatpack/src/main/java/net/sf/flatpack/DefaultDataSet.java 2008-03-18 12:31:17 UTC (rev 398) @@ -235,7 +235,17 @@ return Integer.parseInt(s); } + + public long getLong(final String column) { + final String s = getStringValue(column); + if (!strictNumericParse) { + return Long.parseLong(ParserUtils.stripNonLongChars(s)); + } + + return Long.parseLong(s); + } + /* * (non-Javadoc) * This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site. |