fields.awk Files

fields.awk
Flat-file Database Script
Shell script + awk

This README file is designed to guide you through the basics.

The program is self-contained in a single script file "fields.awk", which must
first be made executable.
	$ chmod +x fields.awk

Store the script in a location folder of your choice, navigate there and
execute the following command, as root, to create a symbolic link to
/usr/local/bin or other suitable bin directory.

	# ln -sf `pwd`/fields.awk /usr/local/bin/fields

Now the script can be called from any directory, as regular user, by typing the
command "fields" [ENTER]

The first time fields.awk is launched, it will look for default folder
~/.fields and its sub-folders ~/.fields/db-archive and ~/.fields/SEARCHES
When it fails to find them it will immediately create them.
It is possible to re-define those defaults by editing the following lines:

FIELDS_FOLDER=~/.fields
DB_FOLDER=~/.fields/db-archive
SEARCH_DIR=~/.fields/SEARCHES

The initialization routine will then substitute your values upon first startup.


The script acts upon datafiles with a first line of the following structure:
@@@FIELD1::FIELD2::FIELD3::.....::FIELDn::
That is, line1 is a declaration of field names, separated by double-colon ("::"
delimiters.)

Fields.awk looks for its data files in $DB_FOLDER (~/.fields/db-archive) and
they can be selected from the main menu. However the datafile can also be
specified on the command line. (legacy behavior)
	
	$ fields [path-to-your-datafile]

The Main Selections screen presents the following choices:
 (R) Read from: 
 (W) Write to: 
 (E) Edit (Selected database): 
 (S) SELECT (an existing) database 
 (N) NEW database (create one) 
 (T) TALLY (total) a column in: 
 (F) Saved-FILES (in ~/.fields/SEARCHES)
 (X) Encrypt/Decrypt (files in ~/.fields/db-archive)
 (Q) Quit 

The script comes with a bold/color highlighted main selections display; however
if that is undesirable, or if the escape sequences for some reason create
problems with your version of bash shell or terminal used, the following basic
code can be used to replace the supplied code, for a bare-bones display:


echo  "======================================"
echo  " (R) Read from: " $DATAFILE
echo  " (W) Write to: " $DATAFILE
echo  " (E) Edit (Selected database): " $DATAFILE
echo  " (S) SELECT (an existing) database "
echo  " (N) NEW database (create one) "
echo  " (T) TALLY (total) a column in: " $DATAFILE
echo  " (F) Saved-FILES (in $SEARCH_DIR)"
echo  " (X) Encrypt/Decrypt (files in $DB_FOLDER)"
echo  " (Q) Quit "
echo  "**************************************"; echo ""


Assuming you have fields.awk launching correctly, and the default folders
created, you need to have at least one data file created and saved in
$DB_FOLDER. You can either use a text editor to create the one-line header for
your chosen first database, OR use the "N" ("NEW database") selection (just hit
the N or n keys). This will prompt you for the number of fields/record your
database will contain, and then prompt you for the names of those fields, from
first to last. It will then create the file in $DB_FOLDER and automatically 
select it.

If more than one database file exists in $DB_FOLDER, you can select between
them using the "S" selection.

Once a database is selected, you can use the "WRITE to" function to begin
populating it with records. WRITE prompts for either a NEW mode (create a new
record) or MODIFY mode (modify a field in an existing record). After creating a
few new records, try the MODIFY mode to alter one of the fields you have
created.

When querying an existing database consisting of multiple records, the "READ
from" function is selected. It will present three Y/N prompts:
	save to file?
	field-filter output?
	case-sensitive search?
Just hitting ENTER in any of those cases defaults to NO.
Then you are prompted for a search string, which will be used to narrow down
the records you wish to display. If you hit ENTER here, i.e. no search string,
then ALL records will be displayed.

Output displays are by default presented in the "less" pager. Consequently you
can scroll up/down with arrow keys or use other navigation methods inherent in
the "less" pager. Hitting "Q" exits back to the Main Selections.

There is a bold-highlighted output display option which you can select by
choosing between the two options of PRINT STATEMENT within the READ_from
module:

#PRINT STATEMENT (un-comment the desired selection)
#1 (default)
printf "`cat $OP`\n" | fold -s | less; clear
#2 (for bold-highlighting, depends on "most" pager)
#printf "`awk '{gsub(" = "," = \\033[1m"); gsub("^","\\033[0m"); print}' $OP`\n" | fold -s | most; clear

Note #1 is the default scenario, no highlighting, using "less" pager.
If you want to try option 2, then comment-out the default line, and un-comment
the second printf command.
Note: that printf command must be all one line! And it depends on having the
"most" pager utility installed. This is because the "less" pager does not
support the ANSI escape sequences used in the bold highlighting.


The EDIT selection will open your selected database in your default text
editor. This is vim by default but you can edit the line
	EDITOR=vim
to change that.
Using a text editor on the datafile directly allows you to make alterations to
a long comments field, etc., more easily than the WRITE->modify method permits.
However, bear in mind that ALL RECORDS MUST BE SINGLE LINES. That is,
fields.awk reads each line as a single record so any record lines that are
split into two with an errant EOL or CR character will not be read properly.
So, if a line wrap occurs during an edit session, you must pull the record back
into a single line before saving the edit.


If one of your fields in a database is devoted to numerical data (financial or
otherwise) you can tally the data from that field using the TALLY selection.
This will list the fields structure of the current database, and prompt you for
a number (1-n) representing the column/field to be tallied. The result will be
reported immediately and you can then optionally do a filtered tally on
selected records.


If, during the READ mode you have opted to save-to-file the outputs of any
searches, then they will end up in $SEARCH_DIR. The "F" (Saved-FILES) selection
is used to edit/print/delete those files. Of those three, the print option is
dependent on you having a command-line option for your printer setup. The
command in the script is suitable for a CUPS installation on any Linux system.
That is: "lp -d $PRINTER [filename]"
You must define $PRINTER in the variables section near the beginning of your
script:
$PRINTER=[your_printer_name]

If you have a different printer installation, whatever command sequence works
for you on the command line will also work from within the script; so you can
edit the line:
	[Pp]*) lp -d $PRINTER $SELECTION ;;
in the "saved_files()" module accordingly.


As mentioned above, the "READ from" function has a field-filter option during
any record search. If you have a large number of fields/record in a given
database, and want to display only selected fields, use this option to do so.
For example, if I have a database of vehicle maintenance with 7 fields/record,
and I only want to display fields# 1 and 5, I would enter those numbers in a
comma-separated list. (Note: that's a comma+space)

Field-filter output? 
y
DATE || WORK DESCRIPTION || PART(S) || PART NUMBER || TOTAL COST || COMMENTS ||
ODOMETER ||

Enter the field-numbers you want to display, separated by commas:
1, 5

and I might get an output like so:

*******************************************

==> Matching record 10 :
DATE = 05-22-07
TOTAL COST = 29
*******************************************

==> Matching record 11 :
DATE = 05-28-07
TOTAL COST = 12.18
*******************************************

==> Matching record 12 :
DATE = 05-30-07
TOTAL COST = 21.01
*******************************************

==> Matching record 13 :
DATE = 06-03-07
TOTAL COST = 30.94
*******************************************


Finally, the "X" selection refers to an encryption option. This is only
available if you have the "ccrypt" utility installed. This is readily available
for Linux, Mac, and FreeBSD systems, free and easy-to-install. It is supposed
to have a reasonably powerful algorithm according to the MAN file:

"ccrypt is a utility for encrypting and decrypting files and streams. It was 
designed to replace the standard unix crypt utility, which is notorious  for 
using a very weak encryption algorithm.  ccrypt is based on the Rijndael block 
cipher, a version of which was also chosen by the U.S. government as the
Advanced  Encryption Standard  (AES,  see http://www.nist.gov/aes). This cipher
is believed to provide very strong cryptographic security."

Ccrypt has two sub-commands: ccencrypt & ccdecrypt
These are simply called from within the "en_decrypt()" module of fields.awk, to act upon selected datafiles (encrypt at the end of a session; decrypt at the
beginning of a session) which you might deem so sensitive as to require this
kind of protection. But that said, beware the risk inherent in using this
option: Lose/forget that encryption key, and your data is lost anyway.


Those are the basics of Fields.awk.
Enjoy ☺ 
Ian R. Forsyth