[Hive-users] Hive Devel Release 20000521 "Hindenburg"

[Tucker I S. <ts...@MI...>]

Hello Hive users,

We have been planning to get out a stable release of Hive for the last
several days, but numerous last minute bugs have delayed it.  Right
now, there is only one item left on our list, but unfortunately, both
Raffi and I need to work on a demo for DL this Wednesday, and won't be
able to get to this last bug until then.  Thus, we are releasing what
we have as a devel release, with a known bug being that there is a
thread locking problem on startup that can keep hive from booting.  We
will get this bug fixed soon after the demos end.  If it doesn't boot
at all on a certain computer, a workaround that *may* help (however
ugly it may be) is to set the debuglevel to notice, either on the
command line with the --debuglevel=notice or in the
HiveCellDescription file with the <DebugLevel>notice</DebugLevel> tag
(see below for more details).


- Hive Devel Team




	Release Notes for Hive-20000521: Hindenburg


Contents:

I.    Overview of current status
II.   New demos built with Hive
III.  Details of the major changes since the previous version
IV.   Incompatibilities with Release 19991210
V.    What's ahead for Hive

                I. Overview of current status

It's been nearly half a year since our last stable release, and big
changes have been taking place.  Now that it's finally stabilized,
Hive is much smaller, faster and more efficient than it ever has
been.  In fact, with code-god Raffi essentially completed with his
renovations of the internals, the core classes of Hive (Cell,
AgentMonitoringAgent, AgentMessagingShadow, etc.) are in a supremely
well tuned state, and will not be going through any more large changes
in the foreseeable future.  In addition, Hive has seen its user-base
grow, and is now the infrastructure of choice for many new demos in
both the Agents Group and the Wearables group here at the Media Lab.
Finally, all of Hive has been renamed from edu.mit.media.hive to
net.hivecell.hive.  Should be easier on the wrists.... In another
renaming spree, the net.hivecell.hive.xml package has been renamed to
net.hivecell.hive.description, and the classes that make up the
package have also been renamed:
        XMLDescription -> Description
        XMLDescribable -> Describable
        XMLDescSet     -> AgentSet
A new class, RemoteDescribable, has also been added to avoid needing
to test a local Describable object for RemoteExceptions when getting
and setting a description.  There is a perl script, PackageRenamer.pl,
that will do the name conversion for you.  It can be found on the
www.hivecell.net web-site in the download section.

On a personal note, Hive's leading developer, Raffi Krikorian, won the
Fano award for being the most outstanding UROP at MIT because of his
work on Hive.  He's been doing an unbelievable job, and it shows.
Hive has really come far since the last release because of the massive
amount of time and effort he's put into it.

Both of us (me being Tucker) will be away this summer, and any
releases (after the next one in a few weeks) will be small bugfixes,
not any attempts of answering research type questions.  Raffi will be
taking a leave of absence next term (he desperately needs a break from
school), but I will still be around, and I'm in the process of trying
to recruit more UROPs to keep Hive moving.  Maybe we'll still see
some check ins from Raffi when he's bored.

              II. New demos built with Hive

InShop 
(http://agents.www.media.mit.edu/groups/agents/projects/impulse/inshop/)
        Intelligent agents manage a customer's shopping list as they 
        wander around a grocery store, and suggest recipe options
        based on ingredients they have already bought, and lets them
        know what to buy to make certain recipes.  Hive allows several
        different types of agents to easily cooperate and share
        information, and also allow possible-but-not-yet-implemented
        features (such as a map of the store) to be easily added
        later.

WAPPointments
        This is a collaborative scheduling demo, which compares your
        appointment book with those of your friends to see if you will
        be in the same place at the same time, and suggests a meeting
        place if so desired.

streetWise
(http://agents.www.media.mit.edu/groups/agents/projects/impulse/streetwise/)
        This demo uses Hive as a framework for allowing agents with
        location awareness to communicate with other agents specified
        by the user.  These agents are in communication with one
        another, and let the user know when he is around places and
        things which are of interest to him.

        III. Details of the major changes since the previous version

1) Probably the largest change in the release is the complete re-write
   of the message handling system, also by Raffi.  Previously, each
   agent spawned a thread to handle messaging to other agents through
   RMI, which was a time consuming and inefficient process.  This also
   could eat up a large number of ports, depending on how many agents
   were messaging each other.  Now, a single shadow,
   AgentMessagingShadow, handles event communication between agents on
   different cells.  This means only one dedicated port is required to
   talk between any two given cells, and if a message is being sent to
   more than one agent, it is only transmitted once, and forwarded
   multiple times by the AgentMessagingShadow.  This allows Hive to
   send more than 500 events per second, up from about 50 or so
   previously.

2) The second largest change is the switch from RDF to XML as the
   internal mechanism for storing Agent's description and
   configuration information.  What these means to developers/users is
   that agent descriptions/config files are much easier to write, due
   to the simpler syntax of XML.  In addition, there are now way more
   functions in the description package for querying agents and sets
   of agents based on their descriptions.  Many of these new features
   are not being used anywhere (yet), so check the javadocs for
   net.hivecell.hive.description.* to see what the new package can do.
   Highlights include the ability to dynamically modify an agent's
   description, print out the description in XML, query a description
   based on XML attributes, and support for querying AgentSets
   (formerly XMLDescSets) that used to be available only to through
   the Cell's queryAgents method.

3) Debug improvements: Ability to set specific debug levels by
   class. This means you can block all messages less severe than, say,
   warnings for all classes except one, for which you can show all
   notices.  In addition, debug messages can be sent to an arbitrary
   stream, allowing for the easy creating of log files.  Also, a
   pre-processor has been written to insert file and line number
   everywhere it finds a debug statement.  This will allow us to get
   full Debugging information even in a JIT, as well as avoiding the
   overhead of throwing and catching exceptions to parse the
   information out.  However, no one currently on the Hive team knows
   how to integrate it into the Makefile, so any suggestions would be
   appreciated :) Also added the printStackTrace() method in case the
   debug output will show the entire stack trace instead of just file
   and line.

4) Cell.java code cleanup: Exceptions are being thrown in a more
   consistent manner.  Also, the multiple hive startup files are now
   stored in the config section of the cell's description,
   ~/.hive/HiveCellDescription.  See the file HiveCellDescription.sgml
   in docs for a complete listing the new startup format.  Shadow
   startup and management is handled similarly to the agents (see
   below), so there are now two new inner-classes that replace the
   functionality of ShadowDB.java and AgentGateKeeper.java.  Thread
   handling has been cleaned up a lot, so the intermittent lockups
   during boot should be a thing of the past.

5) There is also vastly improved infrastructure for handling shadows.
   There is no longer a shadowDB; agents and shadows are now managed
   the AgentTable and the ShadowTable in the Cell, and in addition,
   lazyShadowInstantiation has been removed.  Thanks to Raffi, there
   is now a complement to rmic for compiling stubs for shadows.  This
   means that Hive doesn't need to expose all the methods of a shadow
   to agents.  Functions that should only be called by the cell can be
   hidden, as the agent only sees a stub.

6) Agent startup is now more controlled, and Hive can make more
   promises on the state the agents are in during the stages of their
   creation.  Blank descriptions are created for agents by default
   when they are instantiated, so an agent's description is never
   null.  An AgentSetupException must be thrown if an agent realizes
   it can't be created in its doLocalSetup, instead of calling
   diePlease().  This will need to be changed in any existing agents.

7) Shiny New AgentMonitoringAgent: This fixes the cache coherency
   problems sometimes experienced in the previous versions of Hive,
   and provides better data synchronization.  In addition, this agent
   now has a much simpler remote interface.

8) ExternalInterfaceShadow: This shadow in the apps package is the
   first stage of the infrastructure that allows Hive to communicate
   with non-Hive devices by sending meesages over an XML stream.  This
   is used in the InShop demo for communicating with a PalmPilot,
   which doesn't have a JVM.  In the far distant future, all messaging
   will be done through XML streams in the AgentMessagingShadow.
   Then, non-hive devices would be able to be integrated with hive
   simply by talking the standard hive protocol.  Until then though,
   this shadow provides a means for letting non-Java devices to
   communicate with Hive.

9) The RMI registry is now exposed, so non-Hive RMI objects can use
   RMI to communicate with Hive.  This does for RMI what the
   ExternalInterfaceShadow does for messaging (see above).  Daniele de
   Francesco built an agent that also exposes itself through the
   registry, so that it can communicate with hive over RMI without
   using Hive.

                IV. Incompatibilities with Release 19991210

This is a (hopefully) complete list of the changes that will break
existing agents, either at compile time, or when they start up.  If
you find an incompatibility that isn't listed here, please send it to
hiv...@hi....

1) Agents using RDF need to be converted to use the XML description
   package.  The differences between the two packages are outlined in
   the docs, descriptions.sgml.

2) Different Exceptions are being thrown by Cell.java, so some
   try/catch blocks will need to be added to get old agents to
   compile.

3) diePlease() in doLocalSetup causes problems.  It needs to be
   changed to throw new a AgentSetupException

4) edu.mit.media.hive and references to the edu.mit.media.hive.xml
   package will need to be changed to the new net.hivecell.hive
   namespace, and references to the edu...hive.rdf or edu...hive.xml
   will also need to be changed to the description package.  A perl
   script, PackageRenamer.pl, that will do the name conversion for you
   can be found on the www.hivecell.net website in the downloads
   section.

5) Due to the new AgentMessagingShadow, new cells may have a problem
   sending events to old cells, and RMI between this version and old
   versions will not work at all.

                       V. What's ahead for Hive

Next Release:

1) Hive Classloader - By building a tree of classloaders, we hope to
   solve the version compatibility problem that currently prevents
   agents from different versions of Hive working together reliably.

2) More security - Mainly focused on protecting the Cell from
   malicious agents.  Protecting the agent from malicious Cells is a
   hard problem!

3) Better Control of debug output from the CellDescription - in the
   Cell's description, it will be possible to specify the level of
   debugging output based on a class by class basis, not just for the
   entire cell.

4) debugPreProcessor integrated into the Makefile - Any takers?

Beyond the next release:

1) No more ExternalInterfaceShadow, instead the AgentMessaging shadow
   will use XML instead of a serialized stream, and inter-cell event
   communication can happen thorough that. Thus, non-hive devices can
   talk to hive using the same standard protocol that hive uses to
   talk to hive.