pypgsql-users

[Pypgsql-users] c-jdbc

[Dick K. <D.J...@ch...>]

Hi list,

C-jdbc is a tool that makes access possible from Java to clustered
databases. This provides load balancing, hot failover and all kinds of
other goodies that are most welcome to sites that are working on
different locations.

Is there a possibility that you folks could provide something like a
connection to c-jdbc, in such a way that we could use its potential
also? I haven't got a clue how to go about in this, as I am a python
programmer and certainly not a C or Java programmer, but if I can help
in any way if someone is willing to take a look at this, I most
certainly will.

Kind regards,
Dick Kniep

Re: [Pypgsql-users] c-jdbc (or rather sqlrelay ?)

[Karsten H. <Kar...@gm...>]

Dick,

> Is there a possibility that you folks could provide something like a
> connection to c-jdbc, in such a way that we could use its potential
> also?
Maybe I am wrong here but as I understand it c-jdbc and
pyPgSQL are at the same level of abstraction (with c-jdbc
following the jdbc model of things and pyPgSQL following the
Python DB-API 2.0 model of things). It seems to be a weird mix
to try to connect the latter to the former (since they are
both designed to connect to the same thing - a SQL-driven
RDBMS).

Does "sqlrelay" address some of your needs, however ? It's a
DB-API Python module that provides a few goodies resembling
some of the things you mention about c-jdbc.

Karsten
-- 
GPG key ID E4071346 @ wwwkeys.pgp.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346

Re: [Pypgsql-users] c-jdbc (or rather sqlrelay ?)

[Dick K. <D.J...@ch...>]

On Sat, 2003-07-26 at 00:51, Karsten Hilbert wrote:
 
Hi  Karsten

I understand that I am comparing apples and oranges. And I do not expect
that anything is done. However, I like the discussion, and possibly,
unexpectedly, an answer comes from someone on the list.

Maybe I should have made it clear why I posed this strange question. We
have developed a product which uses Postgresql as a database, Python,
wxPython and Pypgsql. This works OK, no problems whatsoever.

However, some larger companies want to use the application as well. One
company has some 15 locations where people are working. Off course this
can be supported with high bandwidth online connections, but these
connections are expensive, and unreliable. Therefor I have been looking
for a way to get rid of the direct online connection. So, what I want is
a local connection to the database, with asynchronous updates to
databases that are in the same 'logical' cluster of databases. c-jdbc
looks like the proper candidate for this action, however no luck because
we developed using Python.....

Gains from this setup would be:
- if the connection goes down, work can continue
- all databases are mirrors of each other, so if a server has a hardware
problem, by rerouting the trafic, work can continue
- performance of the local setups would be significantly better
- Predictability of the responsetime would be better
- lower bandwidth requirements, as the updates are asynchronously done

On the downside:
- Extra hardware (diskspace and servers) is required
- complexity of the updates to all databases
- a short delay (minutes) between updates to one database and updates to
the others

I have looked at SQLRelay, and it does address some, but not all the
requirements, because there is no mechanism to direct update queries
transparently to other machines. Off course I want to hide the
complexity of updating multiple databases from my application. It looks
as though it can be used with pyPgsql. Is that the case?

Thanks for anyone responding

Dick Kniep

Re: [Pypgsql-users] c-jdbc (or rather sqlrelay ?)

[Karsten H. <Kar...@gm...>]

> However, some larger companies want to use the application as well. One
> company has some 15 locations where people are working. Off course this
> can be supported with high bandwidth online connections, but these
> connections are expensive, and unreliable. Therefor I have been looking
> for a way to get rid of the direct online connection. So, what I want is
> a local connection to the database, with asynchronous updates to
> databases that are in the same 'logical' cluster of databases. c-jdbc
> looks like the proper candidate for this action, however no luck because
> we developed using Python.....
Sounds like you are looking at a middleware-solution where
frontends talk to a locally running server (say, XML-RPC one)
which in turn talks to the DBs (or not if disconnected). You
might start working against a hot-spare DB on the local host
when the remote DB goes dead and start logging all SQL
statements during disconnected operation. Once reconnected you
can try replaying the DML log (which isn't guaranteed to
succeed due to other users' DMLs).

Karsten
-- 
GPG key ID E4071346 @ wwwkeys.pgp.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346

Re: [Pypgsql-users] c-jdbc

[<gh...@gh...>]

Dick Kniep wrote:
> Hi list,
> 
> C-jdbc is a tool that makes access possible from Java to clustered
> databases. [...]

Interesting as this library may be, I have no idea why you post this to 
the pyPgSQL-users list.

-- Gerhard

Re: [Pypgsql-users] c-jdbc

[Dick K. <D.J...@ch...>]

On Sat, 2003-07-26 at 01:29, Gerhard H=E4ring wrote:

> Interesting as this library may be, I have no idea why you post this to=20
> the pyPgSQL-users list.

Hi Gerhard,

Yep, maybe complete off-topic. However, wouldn't it be nice if we could
use our Python programs with Pypgsql using this library? I think I'm
daydreaming.....

Kind regards,
Dick

>=20
> -- Gerhard

Re: [Pypgsql-users] c-jdbc

[<gh...@gh...>]

Dick Kniep wrote:
> On Sat, 2003-07-26 at 01:29, Gerhard H=E4ring wrote:
>=20
>>Interesting as this library may be, I have no idea why you post this to=
=20
>>the pyPgSQL-users list.
>=20
> Hi Gerhard,
>=20
> Yep, maybe complete off-topic. However, wouldn't it be nice if we could
> use our Python programs with Pypgsql using this library? I think I'm
> daydreaming.....

Yes you are daydreaming, if not hallucinating :-P

As far as I see this, this is just a Java library (I haven't read much,=20
let alone downloaded and tried it). To use Java libraries, there's Jython=
.

To use Java stuff from CPython, there's JPE, but I don't know (and=20
doubt) wether it fully works. JPE certainly doesn't look to be well=20
supported from what I can judge from its CVS tree.

-- Gerhard

Re: [Pypgsql-users] c-jdbc (or rather sqlrelay ?)

[<gh...@gh...>]

Dick Kniep wrote:
> [...] So, what I want is
> a local connection to the database, with asynchronous updates to
> databases that are in the same 'logical' cluster of databases. c-jdbc
> looks like the proper candidate for this action, 

The description of c-jdbc looks neat, however I have my doubts wether it 
can work reliably because database replication is a very complex topic. 
I have no experience in it myself, but I briefly talked with a colleague 
who did create a whitepaper for adding database replication to an 
existing Oracle database. All I can say is that it gets complicated 
really quick.

> however no luck because
> we developed using Python.....

Well, there are multiple possibilities to use c-jdbc from Python 
nevertheless. I'd recommend to create a Java server proxy that uses JDBC 
and that you access from Python via CORBA, XML/RPC or SOAP.

Alternatively, c-jdbc being open-source one could look at the algorithms 
used and create a similar library for Python. I had something like this 
in mind recently :)

> [...] I have looked at SQLRelay, and it does address some, but not all the
> requirements, because there is no mechanism to direct update queries
> transparently to other machines. Off course I want to hide the
> complexity of updating multiple databases from my application. It looks
> as though it can be used with pyPgsql. Is that the case?

No, SQLRelay cannot be used with pyPgSQL. There's, however, a DB-API 
Python library to access a SQLRelay backend. It doesn't have pyPgSQL's 
special features like Unicode, PgResultSets etc. of course.

-- Gerhard

Re: [Pypgsql-users] c-jdbc (or rather sqlrelay ?)

[Dick K. <D.J...@ch...>]

Hi Gerhard,

The python server would be neat, if we could pull it off. 

When you get going, send me some notes, I would like to participate. 

In the mean time the users will have to use the system the old fashioned
way (with a single database and communication facilities).

Kind regards,
Dick