Re: [SQLObject] Re: problem: implementing BoolCol

[Ian B. <ia...@co...>]

On Mon, 2003-05-05 at 16:57, Nick wrote:
> On Mon, 2003-05-05 at 14:39, Ian Bicking wrote:
> > Validation may be important when fetching from the database, if
> > SQLObject doesn't have total control of the database, and the
> > constraints aren't as restrictive as the validation.  That might be an
> > obscure case, though.
> 
> Conversion is important, especially for DBM and SQLite, which only store
> strings, coming out of the database.  Hoever, *validating* data coming
> back out of the database can be devastating to an application if it
> can't control the data going into the database from another source. 
> You'd never be able to retrieve a row, even to fix the data, if you got
> an exception because the data already in there is bad.

DBMConnection actually is more shelve-like, pickling all its data, so
you get full types in and out.  But if you want to move to another
backend, validation to restrict that flexibility would serve you well,
so it's not beyond all of this.

At first I thought: yes, validating input from the backend isn't very
useful, since there's no good way to recover in the case of invalid
input.

But on second thought, one design decision that I feel is good in
Validator (and its predicessors) is that validation and conversion are a
single process, because they are whether you want them to be or not.

The easiest example would be for an integer (from a string).  The
converter would be int() itself (convenient :).  But then there's an
implicit validation too, because int("abc") isn't going to work.  You
will get an exception.

So you need a way to handle these exceptions, whether or not you want
to.  How you do that isn't clear to me... in some cases they may be an
example of corrupt data in the database (an all too common occurance). 
Throwing an exception is a pain, but really the only thing you can do at
that point is to provide an alternate mechanism so a person can fix
that.  Doing direct queries is one possibility right now.

In another instance, your validation is too tight.  It's a bug, and
should be fixed in your code.  You should always be able to do 
obj.x = obj.x, and I think it deserves an error if something gets in the
way of doing that (and if the validation is really odd or eclectic, you
should be overriding _get_x and _set_x so you can be as eclectic as you
want).  That error will show up eventually -- better sooner than later.

But, in both cases it's a real pain to deal with it automatically or
programmatically.  But it's not clear to me how it should be dealt with.

Using Validator you can actually be explicit about it, like:
  Validator.Int(ifInvalid=None)
So "x" becomes None when you validate it.  Or you could do:
  Validator.Any(Validator.Int(), Validator.Validator())
Which if an input can't be turned into an integer, it returns it as is
(Validator.Validator incidentally being the identity function of
validators). 

That said, I'm still open to more ideas, but handling exceptional
behavior here is a pain in the but, most definitely.  Maybe Validator
could distinguish between validating input and validating output (which
it doesn't currently -- it just distinguishes conversion).  Then you
could at least choose.

I think with some more long-winded programming, even with its features
Validator should be fairly efficient (i.e., multiple implementations of
attemptToPython and attemptFromPython, so you only pay for the features
you use).  Of course, I don't have any performance data at all, so maybe
that's premature optimization...

  Ian