firebird-devel

[Firebird-devel] How do I do this under Windows?

[Chris K. <ch...@ai...>]

Howdy,

Sorry for the off-topic post but I'm wondering how to do the following under
Windows:

Under FreeBSD, I'm able to run several hundred instances of Firebird CS,
each listening on a separate IP (for those interested, this is done using
FreeBSD's jail facility). The reason for this is to have databases available
to different authentication domains (think accounting system, think multiple
companies).

I'd like to be able to do the same thing under Windows. Is it possible to
have multiple Firebird installs, each listening to their own IP and
authenticating against their own isc4.gdb?

If not, which bits of code do I need to modify to make it do so? ie I
really, really need to do this (think money :-)).

I'm not interested in replies containing the word VMware either.

I've tried to make it on-topic by talking about possible code changes :-)

Regards,
Chris Knight
Systems Administrator
AIMS Independent Computer Professionals
Tel: +61 3 6334 6664  Fax: +61 3 6331 7032  Mob: +61 419 528 795
Web: http://www.aims.com.au

Re: [Firebird-devel] How do I do this under Windows?

[Mike N. <ta...@al...>]

Chris Knight wrote:

> Sorry for the off-topic post

Not off-topic.

> but I'm wondering how to do the following under
> Windows:
>
> Under FreeBSD, I'm able to run several hundred instances of Firebird CS,
> each listening on a separate IP
[...]
> I'd like to be able to do the same thing under Windows.

I assume you are aware that the Microsoft licensing costs to be able to do
this, and the machine needed for several hundred of quite heavy processes,
would probably sum up to some pretty large number.

But, are you asking about running several hundred CS processes on Win32? SS
processes would probably be quite easy to fix, but CS is AFAIK not something
to use on Win32, especially since it's (again AFAIK) not supported and Win32
doesn't come with an inetd.

> Is it possible to
> have multiple Firebird installs, each listening to their own IP and
> authenticating against their own isc4.gdb?

Currently, and in order, no, no and yes.

> If not, which bits of code do I need to modify to make it do so?

It depends on how you want to be able to start the processes. If you want to
start them as services I think you are in for a bumpy ride. If you however
accept to start them as console processes it's doable.

A non-complete list would be something like:
The registry-handling (you can not use the ini-file-from-hell).  The easiest
way to do this is by starting the processes as console processes, setting
the environment variable(s) needed to point to different DB locations for
each process. Search for ISC_ENV inside jrd/gds.c(pp). Maybe this means you
really don't need to change that part of the code.

You need some kind of major-multi-IP-address-able-per-NIC TCP stack. I don't
know if the MS stack handles hundreds of virtual/additional IP's, and even
if it does I wouldn't have too high hopes about its performance. Anyway, you
would need to make the engine bind() to specified interfaces, changing
sin_addr.s_addr from INADDR_ANY to a specific IP to be gotten from ini-file,
command line, environment variable or some other means you can give
different processes different input. You therefore need to add parsing for
this and change the code inside (at least) inet.c(pp) to bind() to the right
IP. Another way of doing it could be to put another machine (with a _real_
OS :-) ) in front of the window box, and then forward its incoming
connections on different IPs to the same IP but different ports on the Win32
box, since the port Firebird listens to can be configured.

You would then probably need to remove the check for already started server
from remote/window.c(pp), and most definitely you'd need to create
differently named file-mapping objects (ipserver/ipserver.c(pp) - probably
just by appending the process ID to the name).

Then you probably need to rename the window (possibly even the window-class)
to different names for each instance. Note that if you require the Guardian
to start the processes, this should probably be a command-line parameter to
the server so that the guardian process(es) knows what server to check for.
Also note that this doubles the amount of processes that have to run on the
machine, and you would still need to start the different guardian processes
from consoles and make it set/forward environment variables and command-line
parameters to the server process.

You'd also manually have to do the "load balancing" of setting CPU_AFFINITY
for the different processes.

There are probably also a bunch of other places to change, but this should
give you a start.


> I'm not interested in replies containing the word VMware either.

How about Bochs? ;-)

/Mike

RE: [Firebird-devel] How do I do this under Windows?

[Chris K. <ch...@ai...>]

Howdy,

> -----Original Message-----
> From: fir...@li...
> [mailto:fir...@li...]On Behalf Of Mike
> Nordell
> Sent: Monday, 18 March 2002 20:57
> To: fir...@li...
> Subject: Re: [Firebird-devel] How do I do this under Windows?
> 
> [snip]
> 
> I assume you are aware that the Microsoft licensing costs to 
> be able to do this, and the machine needed for several hundred of
> quite heavy processes, would probably sum up to some pretty large
> number.
>
That's OK. I don't have to pay for the licenses or the hardware - I
just need to provide a reference platform for acceptance. The client
would prefer to run the system on Windows and not FreeBSD. I'd like
to try to use the existing infrastructure I've already developed under
FreeBSD if possible.
 
> But, are you asking about running several hundred CS processes on
> Win32? SS processes would probably be quite easy to fix, but CS is 
> AFAIK not something to use on Win32, especially since it's (again
> AFAIK) not supported and Win32 doesn't come with an inetd.
> 
SS is fine. I only built CS on FreeBSD due to threading issues a while
back and haven't managed to allocate the time to revisit this and
build a SS version for FreeBSD.

> > Is it possible to have multiple Firebird installs, each listening
> > to their own IP and authenticating against their own isc4.gdb?
> 
> Currently, and in order, no, no and yes.
> 
OK, how do I do the last bit? This is probably the key bit - I need
the ability for the server to have multiple isc4.gdbs, with each
isc4.gdb having one or more databases associated with it. But not
vice versa - a database can't be accessed by two different isc4.gdbs.

> > If not, which bits of code do I need to modify to make it do so?
> 
> It depends on how you want to be able to start the processes. 
> If you want to start them as services I think you are in for a bumpy
> ride. If you however accept to start them as console processes it's
> doable.
>
Sounds good so far.
 
> A non-complete list would be something like:
> The registry-handling (you can not use the ini-file-from-hell). The
> easiest way to do this is by starting the processes as console 
> processes, setting the environment variable(s) needed to point to
> different DB locations for each process. Search for ISC_ENV inside
> jrd/gds.c(pp). Maybe this means you really don't need to change that
> part of the code.
>
Still OK.
 
> You need some kind of major-multi-IP-address-able-per-NIC TCP 
> stack. I don't know if the MS stack handles hundreds of
> virtual/additional IP's, and even if it does I wouldn't have too
> high hopes about its performance. Anyway, you would need to make the

My understanding is that W2K handles multiple IPs per interface better
than NT4. There's a 15 IP address limit prior to SP2 due to AD problems
and a possible 51 IP address limit if the W2K box is also a DC.

> engine bind() to specified interfaces, changing sin_addr.s_addr from
> INADDR_ANY to a specific IP to be gotten from ini-file, command line,
> environment variable or some other means you can give different
> processes different input. You therefore need to add parsing for
> this and change the code inside (at least) inet.c(pp) to bind() to
> the right IP. Another way of doing it could be to put another machine 
> (with a _real_ OS :-) ) in front of the window box, and then forward
> its incoming connections on different IPs to the same IP but
> different ports on the Win32 box, since the port Firebird listens to
> can be configured.
> 
That would be OK. It's a Web app, so I could simply add a Port column
to the database used to keep track of who's database is where.

> [really useful stuff snipped]
> 
> > I'm not interested in replies containing the word VMware either.
> 
> How about Bochs? ;-)
> 
The problem with these is they're too full on. The jail approach under
FreeBSD is good as the isolation level is just enough for an effective
VM without taking all the associated performance hits of a full VM.

Thanks very much for the detailed reply. If a Firebird install can use
multiple isc4.gdb databases, then all the other suggestions should
result in a working system without too much change to my current
infrastructure and hopefully little or no changes to the Firebird code.

Regards,
Chris Knight
Systems Administrator
AIMS Independent Computer Professionals
Tel: +61 3 6334 6664  Fax: +61 3 6331 7032  Mob: +61 419 528 795
Web: http://www.aims.com.au

Re: [Firebird-devel] How do I do this under Windows?

[Mike N. <ta...@al...>]

Chris Knight wrote:
[...]
> > > Is it possible to have multiple Firebird installs, each listening
> > > to their own IP and authenticating against their own isc4.gdb?
> >
> > Currently, and in order, no, no and yes.
> >
> OK, how do I do the last bit?

That's what the environment variable I pointed to is used for. It points to
a directory containing e.g. the isc4.gdb.


If you do get it to work, I hope you post a little something about problems
and performance.


/Mike

RE: [Firebird-devel] How do I do this under Windows?

[Chris K. <ch...@ai...>]

Howdy,

> -----Original Message-----
> From: fir...@li...
> [mailto:fir...@li...]On Behalf Of Mike
> Nordell
> Sent: Sunday, 24 March 2002 5:56
> To: fir...@li...
> Subject: Re: [Firebird-devel] How do I do this under Windows?
> 
> 
> Chris Knight wrote:
> [...]
> > > > Is it possible to have multiple Firebird installs, each 
> > > > listening to their own IP and authenticating against their own
> > > > isc4.gdb?
> > >
> > > Currently, and in order, no, no and yes.
> > >
> > OK, how do I do the last bit?
> 
> That's what the environment variable I pointed to is used for. It
> points to a directory containing e.g. the isc4.gdb.
> 
Gotcha.

> If you do get it to work, I hope you post a little something 
> about problems and performance.
> 
Will do. Thanks again.

Regards,
Chris Knight
Systems Administrator
AIMS Independent Computer Professionals
Tel: +61 3 6334 6664  Fax: +61 3 6331 7032  Mob: +61 419 528 795
Web: http://www.aims.com.au

RE: [Firebird-devel] How do I do this under Windows?

[Chris K. <ch...@ai...>]

Howdy,

For those that don't know, I'm trying to get multiple instances of Firebird
running under Win32. The reason is to have each instance of Firebird use a
different isc4.gdb.
The problem I'm running into at the moment is that isc4.gdb is referenced in
the following files directly via USER_INFO_NAME:

jrd/pwd.c
jrd/pwd.h
remote/inet_server.c
utilities/ibmgr.h
utilities/security.e
utilities/srvrmgr.c

and the following files reference USER_INFO_NAME via PWD_get_user_dbpath:

jrd/pwd.c
jrd/svc.c
/utilities/gsec.c

I've got ibserver.exe starting with a -e parameter that gives me an engine
instance id. My plan is to use the instance ID to allow listening on
different ports (3049 + engine_id) and to access different isc4.gdbs
(%INTERBASE%\engine_id\isc4.gdb). The port listening works OK.
Now, I only need to conditionalise the compile for the ibserver.exe case, as
it's the only bit that knows about the engine_id. Which files MUST I modify?
I don't want to unnecessarily change files, and security.e has me unsettled.
Thanks in advance for any help.

Regards,
Chris Knight
Systems Administrator
AIMS Independent Computer Professionals
Tel: +61 3 6334 6664  Fax: +61 3 6331 7032  Mob: +61 419 528 795
Web: http://www.aims.com.au

RE: [Firebird-devel] How do I do this under Windows?

[Chris K. <ch...@ai...>]

Howdy,

Here's something I sent to Mike Nordell earlier this evening, as he's been
encouraging me in my stupidity :-)

=====
Quick question (hopefully a quick answer :-)):

Will I need differently named semaphores and events? The only reason I ask
is I found the following anomaly between the defines in ipc.h and the code
in ipclient.c. Source is from the build 796 tarball on SourceForge. I could
be missing something however.

ipserver/ipc.h
#define	IPI_CLIENT_SEM_NAME	"InterBaseIPIClientSem%d_%d"
#define	IPI_SERVER_SEM_NAME	"InterBaseIPIServerSem%d_%d"
#define	IPI_EVENT_NAME		"InterBaseIPIEvent%d"
#define	IPI_EVENT_CLASS		"InterBaseIPIEventClass"
#define	IPI_EVENT_THREAD	"InterBaseIPIEventThread%d_%d"

ipserver/ipclient.c:
- GDS_QUE_EVENTS:
sprintf( name_buffer, IPI_EVENT_THREAD, evsem);
                                        ^^^^^
                                        needs another arg

- event_packer:
sprintf( name_buffer, IPI_EVENT_NAME, icc->icc_file, icc->icc_slot);
                                      ^^^^^^^^^^^^^
                                      only this one used

Regards,
Chris Knight
Systems Administrator
AIMS Independent Computer Professionals
Tel: +61 3 6334 6664  Fax: +61 3 6331 7032  Mob: +61 419 528 795
Web: http://www.aims.com.au