Columns modified by Chris DeGreef

Chris DeGreef — Fri, 13 Dec 2013 20:54:04 -0000

Columns are simply a way to define the layout of the input file(s). A column can then be reference in the --[OrderBy clause] and the --[Where clause]. Other than being reference in this way they do not directly have any affect on the sort process.

An interesting way to organize your sorting commands is to define the input (using --columns of course) and store just that in a command file (Trade.def). Then include that command file in subsequent sort commands.

The Trade.def file:

--columns (String --name BrokerBadgeId --offset 10 --length 5) (String --name State --offset 78 --length 2) (String --name TradeId --offset 0 --length 8)

Subsequent use in a sort command:

funnel trades.txt @Trade.def --where(State='IL') --orderBy(BrokerBadgeId)(TradeId desc)

Funnel allows for any number of columns to be defined. The order they are specified is not important.

Although --keys will always be supported the --orderBy is the author's preferred means of specifying the order of the file. --keys and --orderBy can not be used in the same sort.

The general format of a column specification is

 --columns (one column definition)(a second column)(a third column) and so on.

A column definition can take on one of two forms. "format" is optional. The "offset" is always zero-relative.

 (columnType, --name myKeyName --offset offset, --length length, --format format)

The second form is the CSV column. It must be used if the file is defined as a CSV file.

 (columnType, --name myKeyName --field fieldNo, --format format)

Column Type	Description
String	normal readable character data like "ABC"
Integer	a readable integer value like "1234"
Float	a readable floating point number like "456.78"
BInteger	an encoded integer value
BFloat	an encoded floating point number
Date	a readable Java-like date (SimpleDateFormat)

The encoded columns are the standard Java encoded numbers for Byte, Short, Integer, Long, Float, and Double. The length of the key determines which one it is.

Key Type	Data Type	Length
BInteger	Byte	1
BInteger	Short	2
BInteger	Integer	4
BInteger	Long	8
BFloat	Float	4
BFloat	Double	8

Recent changes to Columns

Columns modified by Chris DeGreef