[Bigdata-developers] Alternate install/deploy mechanism

[Brian M. <btm...@gm...>]

Just an FYI to those who might be interested.

Over the last few weeks I've been looking into a deployment
mechanism that might be used as an alternative to 'ant install'.
The investigation has currently taken the form of some code
that I've recently checked in to a personal branch (dev-btm).
If anyone is interested in taking a look at this work and
seeing how it might be used, one can follow the steps
outlined below.

BrianM

---------------------------------------------------------------------------
> cd <baseDir>

> svn checkout
  https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/dev-btm
    <baseDir>/bigdata/branches

> ant release-dist

> ls -al <baseDir>/bigdata/branches/dev-btm

  REL.bigdata-<version>-<date>.tgz (ex. REL.bigdata-0.82.0-210610.tgz)

Open 3 command windows, WinA, WinB, and WinC (use sudo or login as root)

---------------------------------------------------------------------------
-- WinA -- [install and deploy]

> su

# tar xzvof
  <baseDir>/bigdata/branches/dev-btm/REL.bigdata-<version>-<data>.tgz -C
/opt

# cp /opt/bigdata/var/config/deploy/example-deploy.properties
     /opt/bigdata/var/config/deploy/deploy.properties

# vi /opt/bigdata/var/config/deploy/deploy.properties

Un-comment the following items in deploy.properties:

#federation.name=com.bigdata.group.0

#node.type=standalone
#node.layout=1-of-1
#node.role=bigdata

Uncomment and set the node.serviceNetwork item to the name
of the node's network interface card (NIC); which can be
found by typing 'ifconfig' on linux or 'ipconfig /all' on
Windows.

#node.serviceNetwork=eth0

Next,

# cp /opt/bigdata/etc/bigdata.initd /etc/init.d/bigdata

# /etc/init.d/bigdata start

# /etc/init.d/bigdata status

---------------------------------------------------------------------------
-- WinB -- [for non-graphical command line discovery tool]

> su

# /opt/bigdata/bin/disco-tool -v -g com.bigdata.group.0

---------------------------------------------------------------------------
-- WinC -- [for testing restart capability]

> su

# ps -elf | grep java

Pick one of the pids from the output and kill that process. For example,
suppose the java process associated with the "shardlocator" process
is 24539 (that is, '# ps -elf | grep java | grep shardlocator' ==> 24539)

# kill -9 24539

Observe the removed-then-added events displayed by the discovery tool
in WinB.

# /etc/init.d/bigdata status

Observe that the output indicates that the shardlocator process is
in the RUNNING state.

# ps -elf | grep java | grep sharelocator

Note that the service with process tag "shardlocator" appears,
but its pid is no longer 24539; because the process was restarted
upon the death of process 24539.

# /etc/init.d/bigdata stop

# /etc/init.d/bigdata status

Observe that all processes are in the STOPPED state

---------------------------------------------------------------------------
-- START ON BOOT --

To achieve start/restart on boot/reboot, for X=0-5, one can
creaate the appropriate soft links from /etc/rcX.d/KXXbigdata
and /etc/rcX.d/SXXbigdata to /etc/init.d/bigdata

For example, on Ubuntu, one would do the following:

# update-rc.d bigdata defaults

[to remove the soft links type, '# update-rc.d -f bigdata remove']

---------------------------------------------------------------------------
-- NOTES & CAVEATS --

- The mechanism above is intended to support the installation
  and deployment of a system that may include other components
  as well as bigdata, or a system that includes only bigdata.
  Thus, although the files named default-deploy.properties and
  example-deploy.properties reference only a role value of
  "bigdata", other roles can be easily added.

- The file example-deploy.properties is intended to be a
  template for the deploy.properties file that is used to
  communicate the top-level configuration to the mechanism.

  A single deploy.properties file cannot be used since the
  contents of that file will generally be different on different
  nodes in the system; although the goal of the deploy.properties
  file is to minimize the number of items that do differ from
  node to node.

  The deploy.properties file can be created by copying
  example-deploy.properties and then modifying the resulting
  file for the desired configuration (as shown above). Or
  it can be auto-generated by some tool (ex. scripts, awk/sed,
  puppet, etc.)

- To avoid breaking existing code, the beginnings of smart
  proxy based counterparts to the bigdata services were
  created. Currently, those smart proxy based implementations
  include only the smart proxy pattern, the required public
  service interfaces, a common service attribute, and the
  necessary infrastructure for starting and stopping each
  service. None of these service implementations currently
  provide any bigdata-specific functionality.

  The smart proxy based service implementations are intended
  to share nothing but convenient helper utilities and the
  top-level deploy.properties configuration file (when different
  service implementations run on the same node). In addition
  to sharing a common Jini configuration file, the current,
  purely remote, service implementations share up to eight
  layers of common ancestry in the form of abstract and
  concrete super classes; which makes it unclear how much
  work it will take to either add the necessary functionality
  (and tests) to the smart proxy based implementations, or
  convert the current layered implementations to a smart
  proxy model. Thus, much more investigation and work needs
  to be done.