Contents of shell archive myexport, submitted by Art S. Kagel:
Version dated: July 8, 2016
----------------------------------------------------------------------------
Utility: myexport/myimport
Synopsis:
Replacements for dbexport & dbimport which do not suffer from many of
the problems from which those utilities suffer. Most importantly
myexport/myimport does not lock the database. Note the Caviat section
on this subject. Myexport and myimport also extend dbexport/dbimport
with some new and useful features. There are options to allow myexport
to use the myonpload utility by S. R. Krishna for the actual data
unload instead of sqlunload. Future versions may allow myimport to
also use myonpload.
These are ksh scripts which use myschema (from my own utils2_ak
package version 2.238 or later), ksh, awk, and sqlunload or sqlreload
and sqlcmd (from Jonathan Leffler's sqlcmd package) or Ravi Krishna's
myonpload and the HP Loader to get the job done. The output from
myexport should be compatible with dbimport and likewise the output
from dbexport will load using myimport. You will need myschema v2.238
or later to take advantage of the -E/-e export/import using EXTERNAL
TABLES.
The only caviats to compatibility are the -m & -u options. If you
myexport with -m and use dbimport to reload the data you will have to
manually execute the index/constraint creation script which is named
<databasename>2.sql and is found in the .exp directory. Similarly the
data distributions script created with the myexport -u option, named
<databasename>3.sql, will not be executed by dbimport. Inversely if
you dbexport you cannot use the myimport -m option unless you use
myschema's schema split feature (with the -l dbimport compatility flag)
to create the two separate scripts or manually break out the indexes
and constraints using an editor and place and name the scripts
appropriately. And likewise you will have to use either the myexport
-u (or -U filename) option or dostats -f filename to create the data
distributions script to be able to use the myexport -u flag if you are
importing a dbexport unload with an older dbexport version (since IDS
10.00 dbexport includes UPDATE STATISTICS commands equivalent to
myexport with -u).
Author(s): Art S. Kagel
Revision: myimport: 3.14, myexport: 3.13
Features Version: 3.14
Version(s) supported: IDS (Modifications would be needed to support SE)
File(s):
README.1st -- This file
myexport -- Myexport script, perms should be 550 or better
myimport -- Myimport script, perms should be 550 or better
ak_launcher.c -- Program source for a utility to implement the -r,
reduced parallelization option.
simple.dbspace.file -- Simple dbspace map file for myimport -T
complex.dbspace.file -- More complex dbspace map file for myimport -T
to_external.awk -- Used by myexport to create the export using
external tables. Should be placed in the user's path with myexport.
from_external.awk -- Used by myimport to reload the export using
external tables. Should be placed in the user's path with myimport.
Comments:
Supports most of the commandline options of dbexport/dbimport with the
notable exception of writing and reading tapes.
Release Notes and Requirements:
Releases 3.12-3.14 of myimport fix typos and update the copyright
notice as do releases 3.12-3.13 of myexport.
Release 3.11 of myimport implements -L & -P for imports.
Release 3.11 of myexport implements -L & -P for external table exports.
Release 3.10 of myexport implements -Q for sqlcmd exports.
Release 3.10 of myimport adds new option -N <commit-cnt> to commit
every N rows when using the default loader.
Release 3.9 of myexport adds new options: -N use existing schema
file. -Q <escape-char> to specify a different escape character.
-Q requires the June 30 release of utils2_ak with myschema version
version 2.346 or later.
Release 3.8 Added new mapping support for indexes. This is ONLY active
if the export was made using myexport with the -m option to separate the
index DDL from the table DDL. If you append a third field containing
the letter 'I' to the end of a map file line it is treated as only
affecting mappings within the schema file containing index DDL and will
not affect and dbspace entries in the main schema file. This can be
used, for example, to create indexes in a different dbspace than tables
on the target database even if they were colocated in the source
database.
Release 3.7 Eliminated the requirement for sqlcmd when exporting using
external tables. Fixed an error message and several usage texts. Fixed
parallel import using external tables. Documented the -q (quiet) option
to myimport. Added support for PDQPRIORITY environment variable
defaulted to 100 for index builds and update statistics processing.
Release 3.6 Added new mapping support for blobspace dumb blobs. If you
append a third field containing the letter 'B' to the end of a map file
line it is treated as as a dumb blobspace mapping and only affects TEXT
and BYTE type column definitions. The complex.dbspace.file map sample
contains a sample line.
Release 3.4 & 3.5 Added new myexport option, -O, to order tables for
loading in order by dependency on foreign keys. Requires myschema v2.296
or later.
Release 3.3 Added support for mapping smart blobspaces. If you append
a third field containing the letter 'S' to the end of a map file line it
is treated as a smart blobspace mapping and only affects "PUT colname IN
( spacename )" clauses. The complex.dbspace.file map sample contains a
sample line.
Release 3.2 Added support for messages in syserrors added by datablades.
Lots of new stuff.
New option, -i <isolation level> specifies the isolation level to use
when exporting using external tables. Requires myschema rev. 2.301+
Version 2.7 & 2.8 implement exporting and importing using EXTERNAL
TABLES for servers that support them (11.50.xC4+). The -E option
for myexport generates scripts to export data using EXTERNAL TABLES and
also generates load scripts for myimport to use. The -e option to
myexport generates the load scripts to run in EXPRESS mode. Myimport
has the corresponding -E option to use the load scripts generated by
myimport. Also, if myimport detects that the export was performed by
dbexport, it will produce its own import scripts that use EXTERNAL
TABLES. NOTE: The scripts that myimport produces to load dbexport data
are NOT compatible with tables that contain hidden columns because they
create the EXTERNAL TABLES using SAMEAS which includes the hidden
columns in the definition of the EXTERNAL TABLE as normal columns.
Exports produced by myexport v2.8 avoid this problem by generating
complete column lists to define the EXTERNAL TABLES.
Version 2.5 & 2.6 add support in myimport for only loading a specified
number of tables concurrently when the -r (optionally with -R & -M) is
specified rather than -p. It is still recommended that myexport be
used with the -m option (and that -m be included to myimport) when one
is to perform even a limited parallel load using -r.
Version 2.4 add myimport support for loading with myonpload. Previous
releases only supported unloading using myonpload. Deluxe and Express
modes are supported. Also new option -F for myexport to enable
filtering in the schema file so that parallel exports will result in a
consistent database after dbimport/myimport loads the data. Use of -F
requires myschema Revision 2.165 (utils2_ak dated July 12, 2006) or
later.
Version 2.3 adds support for secure connections to the database using
myschema Revision 2.162 or later and sqlcmd (onpload does not support
secure connections). Thanks to Mark Montalvo for alerting me to the
need.
Version 2.2 fixes a problem passing relative paths to the -i option to
myimport. Added a '-l nolog' option to myimport.
Version 2.1 fixes some awk problems running on systems, notably HPUX
without gawk, with particularly pedantic versions of nawk. Substituted
sqlcmd for dbaccess in myimport to create the database and tables so
myimport can be run on a pure client machine.
Version 2.0 includes the new -m & -u options and the myexport script
also includes support for using the myonpload utility with the -D, -C,
or -S flag.
Version 1.4 improves myimport making it more forgiving of the location
of the dbspace map file which defaults to current directory or the
directory specified in the -i option. The -f option has been removed
as it is really an adjunct to the tape option which is not supported.
The SQL file is not restored to its unmodified condition when the
script completes. The -q option has been implemented in myimport.
Problems with parsing some combinations of options in myimport have
been.
Version 1.3 adds several extension features NOT found in
dbexport/dbimport. See the Extensions section for details.
Version 1.2 fixes some confusion if the path supplied to -o was not an
absolute path. Now works correctly with relative paths.
Version 1.1 adds the -p options to myexport which causes the unloads
to occur in parallel reducing data integrity difficulties from not
locking the database as dbexport does and improving total throughput.
No known problems. You will need myschema and sqlcmd V5.2 or later
and a compatible ksh version. Conversion to use bash or perl should
not be difficult, I just like ksh. Also the version of awk I use is
GNU Awk (GAWK) so you may need that or NAWK if you only have an older
awk, I really have not checked if I am using anything version specific
beyond the "-v variable=value" commandline option which most awk
versions now support.
Caviats:
Note, since dbexport and dbimport syntax permits it, myimport/myexport
will accept the database name either as the first argument or
following ALL flags and their optional arguments even though this
violates the UNIX command syntax rules. However, the database name
may not be intermixed with the optional flags and arguments.
Because myexport does not lock the database when exporting one should
be extremely careful about using it for more than emergency archiving
and/or checkpointing if referential integrity is to be maintained upon
reloading the data. It is ideal for data migration and syncing sister
servers especially if other mechanisms are used to maintain the
referential integrity of the database during the export such as
stopping or blocking update tasks. Unlike dbexport myexport can be
safely used while there are open FETCH cursors on the database and its
tables.
The parallel option to myimport is LIKELY to cause data dependency
problems in any database with relational integrity. In addition since
myschema is used by dbexport to generate the schema file and since that
utility does not yet analyse these dependencies it may be necessary to
reorder the tables in the schema or myexport and myimport with the -m
flag so that indexes and constraints are created after the data has
been loaded completely. If the myexport was run on an active server
you may want to edit the index/constraints script adding START
VIOLATIONS statements for tables that may experience consistency
problems during the reload and setting the mode of constraints to
FILTERING.
Using the -p option to myexport for unloading an active database can
reduce the possibility of experiencing integrity problems during
reloads.
Download Latest Version
tpc-c-ifmx.tar.gz (392.7 kB)
