Re: [Firebird-devel] Management and monitoring

[Dmitry Y. <di...@us...>]

"Jim Starkey" <ja...@ne...> wrote:
>
> I don't think that VIO is the appropriate place to implement virtual
> tables.

Agree.

> The existing RSB mechanism was designed around polymorphic
> record streams, each opened, fetched from, then closed.  It was
> implemented with omnibus functions each with a case because I didn't
> want to wait for C++ to be invented to ship the beta version of galaxy.

Well understood ;-)

> The best way to implement virtual tables is to encapsulation the RSE/RSB
> mechanism into a hierachy of  RSB classes.  Having done that, teaching
> the RSE compiler/optimizer to generate special RSB objects for your
> magic virtual tables would reduce the runtime pertubation to the new
> code and a single point if the optimizer.

Since I'm here to touch this part of the codebase, I see no good solution
but just do what you suggest. We may ignore code refactoring when bugs are
being fixed, but it's not the case when new features are being added.

> I would be so delighted to
> have somebody do the encapsulation that I'd be prepared to overlook by
> feeling that virtual tables are likely to introduce more problems than
> they solve, and a probably client library layered on the database info
> call is easier to implement, more general, more extensible, faster, and
> more efficient.

So far I tend to implement both interfaces, powered by a single
utility/class performing the actual retrieval of internal data (too much
overhead to be done just inside database_info, I suppose). Haven't decided
yet, whether both SQL and database_info interfaces will use this
utility/class, or the monitoring tables should be layered above
database_info call.

> Have you give due consideration to handling the metadata for virtual
> tables?  How does the metadata track the dozens of extensions you find
> yourself making as the feature is rolled out and other data is
> discovered to be worth tracking?

I'm afraid, currently, static definitions seem to be the only solution.
Everything else is just a hack against INI/MET subsystem. And since I expect
the engine to support SQL permissions for the monitoring relations, they're
required to be persistent in the schema.

Have you done anything in Vulcan to make dynamic metadata definitions
possible (or at least less painful)? Or perhaps you have other ideas?

> Does a new database have metadata
> populated with a dozen or so virtual tables?

Yes. And, to be honest, I don't see any problems in this fact.

> Has somebody given due consideration to namespace management for the
> plethera of new virtual tables?

Although I'd love to introduce real two- (or three-, if we're going to
follow the SQL standard) level namespaces in FB, this is the task for
another major version. So the only choice is to use some prefix to
distinguish between "real" system relations and "virtual" monitoring ones.
Borland has chosen TMP$ as a prefix, but it just brings confusion to users,
IMO. Suggestion anyone?

> I agree.  Putting monitoring information under transaction control seems
> like a lousy idea.  You understand, however, that you will need to start
> a transaction to execute virtual table SQL even if the result is not
> under transaction control.

Certainly.

> The database info mechanism, on the other
> hand, doesn't require a transaction, and consequently doesn't perturb
> the system being measured.

I hope to expose benefits of both solutions, even if one of them has known
limitations.

> A monitoring tool doing iteration fetches is a better solution than a
> user query.  If nobody wants to write a monitoring tool, then ISQL
> (mega-ugly) is a regretable fallback.
>
> The question is important.  If the primary client is a monitory program,
> then the API should use database_info for the reasons I mentioned.  If
> nobody want to write one, then maybe virtual tables is the way to go,
> though it does beg the questions as to whether the feature is worth
> implementing...

The primary client is a dozen of management tools like IBExpert or
IBWorkbench. But I'm pretty sure that PSQL is another very important client
of this feature. Given CURRENT_CONNECTION and CURRENT_TRANSACTION context
variables, you can query any necessary data from the monitoring tables in
your procedures/triggers, if required (and if you're given permissions to).

> >2) some RDMBS vendors (Oracle, InterBase) offer this kind of monitoring
via
> >virtual tables/views
> >
> I don't know who I'd least rather do my thinking for me, Oracle or
> Interbase.  If I had to, I think I'd take Martha Stewart ahead of either.
>
> Dmitry, think for yourself!

Everything mentioned above is _my_ thinking. So I try to ;-)

> >a) Create a virtual tables subsystem similar to the EXT one. Reserve a
flag
> >in RDB$RELATIONS to indicate a virtual relation. Inject appropriate calls
in
> >the looper (teach it to choose between VIO, EXT and VIRT calls - a
> >flag-based switch in FB or overloaded virtual calls in Vulcan, I think).
The
> >result is non-tranactional monitoring tables implemented by a separate
> >module.
> >
> I think you need to rethink this.  Flags are almost always a bad idea.
> Looper shouldn't have to know anything about tables of any sort.  Looper
> exists to bumble through statements, not worry about tables.

Agreed (see above regarding RSE encapsulation).

> Hint: think objects!

The flag was mentioned because of the current non-OO nature of RSB. If you
haven't changed it in Vulcan, there're no problems for me thinking object in
the FB2 tree to make it real.

> >b) Encapsulate everything related to the monitoring in a new BLR verb
(with
> >a few op-codes). Perhaps we could extend the existing blr_internal_info
for
> >that stuff. Teach this code to represent its data in a tabular format.
Allow
> >views to be based on this BLR verb, in addition to blr_relation/blr_rse
and
> >so on. The result is non-transactional monitoring views implemented by a
> >single EXE/EVL function and its RSE converter.
> >
> What!!!!!  Adding something to BLR when we're trying to get rid of it in
> favor of SQL?????

Cool down, this is not the design going to be chosen. But I always thought
that all possible solutions should be mentioned in discussion. Perhaps even
inadequate and bad ones. Just to allow people to see the benefits of
available alternatives.


Dmitry