Re: [Openinteract-help] sub-classing SPOPS

[Steve S. <sap...@gs...>]

Chris Winters wrote:

> Sorry for the delay in responding. This got put in my: "think about it 
> for a while" queue, which can take a little while to flush...

At least yours is measured in days.  Some of mine have been in the
queue for years now.  8-)

> In theory you can use SPOPS security without using anything else from 
> SPOPS. As long as you can identify your objects uniquely with a class 
> (or general object type) and a unique-to-that-class ID, you should be 
> able to get and set security levels. It's probably not the most 
> efficient means for doing this, but it would work.

This is possible, so that's one option.

I wrote:

>> Just using implicit sequence numbers seems to fit.  It's not clear to 
 >> me how far up the need for an> ID bubbles though ...  assigning my
>> own would not give higher level users any universally known key (like 
 >> a user name, etc.).  What sort of usage impact does that have within the
 >> entire OI framework?

Chris responded:

> I'm not exactly sure what you're getting at. OI doesn't really care what 
> form your ID takes, just as long as it's unique. From the OI/SPOPS view 
> there's nothing technically wrong with having an ID of 'email' for a 
> 'users' table. And I'm 98% certain the ID you specify in SPOPS doesn't 
> even have to be the field the database recognizes as the primary key -- 
> if in that same 'users' table you had a sequenced ID field, SPOPS 
> wouldn't really care as long as it was self-maintaining (with a 
> trigger/sequence/generator or IDENTITY/auto-increment field) and you 
> told SPOPS never to insert any data for it.

I guess the more pointed question is how much of the built-in OI
capabilities require an ID.  I see them used in the example configs.
The fruit example, for instance.  Are those IDs then required to get
the data out to the page, or is that strictly dependent on how you
define your TT templates (assuming TT since I'll be using that)?

> The 'fetch_iterator()' does this fairly well, I think. And you can 
> standardize on using it in TT (or other views) since even when you have 
> a list of objects you can use SPOPS::Iterator::WrapList to present a 
> common interface.
> 
> I feel like I'm missing the point of what you're really asking though. 
> Maybe you can present a small example?

Yes, I guess that would help ...  I'm going to abbreviate some of
the class names, etc. but you should get the idea.

How this all started was with a need to take data from various
sources (mostly databases) and easily output it in different
formats.  An example of a simple SQL translation to Excel would
look like this:

use DataTranslate;

my $map =
[      DBQuery => {SQL => 'SELECT * FROM FOO', DB_NAME => 'ourdb'}
     => Excel   => {FILE => '/tmp/foo.xls'}
];

my $x = DataTranslate->new();
$x->translate(MAP => $map);

So under all this are factories, etc. to make this all work.  Parsing
is rather simple since we're using Perl list/array constructs.  It
does get more complicated since nesting and multiple inputs/outputs
are allowed but that complexity is not needed to illustrate.  There
are 3 basic types of objects associated with each map: Input, Filter,
Output.  Each has a defined interface to "plug" into.  A simple example
of a filter added to the above would be:

use DataTranslate;

my $map =
[      DBQuery   => {SQL => 'SELECT * FROM FOO', DB_NAME => 'ourdb'}
     => StripHTML => {COLUMN => 'DESCRIPTION'}
     => Excel     => {FILE => '/tmp/foo.xls'}
];

my $x = DataTranslate->new();
$x->translate(MAP => $map);

So basically, each map is a bulk translation.  But later, we found the
need to use these in other ways so we added the ability to create a map
and fetch from it a row at a time.  To fetch, you either specify which
"node" to fetch from (nodes can have IDs); or by default it fetches from
the last.  You can also have Null inputs/outputs to create maps that are
balanced enough to parse unambiguously that you just want to fetch from --
without an input or output things become confusing, so Null is a no-op
placeholder.  To fetch from the above, I could do this:

use DataTranslate;
use Row;

my $map =
[      DBQuery   => {SQL => 'SELECT * FROM FOO', DB_NAME => 'ourdb'}
     => StripHTML => {COLUMN => 'DESCRIPTION'}
     => Null      => {}
];

my $x = DataTranslate->new();
my $row;
$x->open();
while (defined($row = $x->next()))
{
     # Do whatever with the row.
}
$x->close();

I could also fetch from the original map, in which case the Excel file
would be created as I got my rows via the next() call.

Rows are also objects.  For bulk translations, you never need to see
those since they're just passed along internally.  For fetching, that's
what you get back and you have to deal with it.

Just to look a little further in, we can also put data into tables:

use DataTranslate;

my $map =
[      CSV       => {FILE => '/tmp/bulkdata.csv'}
     => StripHTML => {COLUMN => 'DESCRIPTION'}
     => DBTable   => {SQL => 'INSERT INTO FOO ...'}
];

my $x = DataTranslate->new();
$x->translate(MAP => $map);

You can map columns to placeholder positions, etc.

The basic model is that each input coerces its data to a Row object.
Once you have those rows you can do whatever.  It's kind of neat since
we've plugged in IO from/to databases, directories, mailboxes, and
many different file formats.  XML, for example, is coerced via a
templating system that maps tagged data to row/column values.

In all cases above, we've canned up any row level identifiers as
part of the map.  All a client sees is operations on that map as
a whole.

For web presentation, I'd like to pull from these.  Since some data
sets can be big, the right thing to do seems to be to fetch each
row.  Looking at SPOPS, it seems like a fit:  I subclass it and
my fetch maps my Row object into the SPOPS view of the world.  The
only thing is, I have no IDs to fetch by -- I always do a "fetch
next."

Security is needed:  If I define a map that pulls sensitive
financial data, I may want only certain people to see that.

I can also see areas where I'd want to use OI out of the
box with the SPOPS DB objects/interfaces it provides.  So
ideally, I fit these maps into the framework as just another
thing OI can deal with.

A layer above all these maps is a class hierarchy built around
them.  And that's not all maps.  Each class there represents a
specific higher level data object we deal with.  e.g., if we
export a specific set of data to a specific client, we'd set
up a class for that client.  Each of these higher level classes
may have several actions associated with it; e.g., export data,
send resulting files, generate reports, etc.  Some of these
actions may invoke translations of canned maps, others may not.
It seems to me that I could map these higher level classes to
OI packages.  Those packages would be derived from SPOPS classes
built around my maps instead of from the ones OI provides for
databases.

-- 
Steve Sapovits    GSI Commerce, Inc.    http://www.gsicommerce.com
Email:   sap...@gs...