sqlrelay-discussion Mailing List for SQL Relay (Page 9)
Brought to you by:
mused
You can subscribe to this list here.
2005 |
Jan
|
Feb
(20) |
Mar
(27) |
Apr
(17) |
May
(32) |
Jun
(45) |
Jul
(49) |
Aug
(68) |
Sep
(44) |
Oct
(29) |
Nov
(64) |
Dec
(25) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2006 |
Jan
(61) |
Feb
(22) |
Mar
(25) |
Apr
(31) |
May
(18) |
Jun
(28) |
Jul
(19) |
Aug
(16) |
Sep
(8) |
Oct
(17) |
Nov
(32) |
Dec
(4) |
2007 |
Jan
(20) |
Feb
(25) |
Mar
(5) |
Apr
(12) |
May
(11) |
Jun
(18) |
Jul
(16) |
Aug
(22) |
Sep
(37) |
Oct
(20) |
Nov
(11) |
Dec
(2) |
2008 |
Jan
(11) |
Feb
(33) |
Mar
(12) |
Apr
(18) |
May
(22) |
Jun
(31) |
Jul
(23) |
Aug
(6) |
Sep
|
Oct
(10) |
Nov
(22) |
Dec
|
2009 |
Jan
(12) |
Feb
(8) |
Mar
(11) |
Apr
(20) |
May
(18) |
Jun
(7) |
Jul
(27) |
Aug
(2) |
Sep
(10) |
Oct
(5) |
Nov
(2) |
Dec
(1) |
2010 |
Jan
(11) |
Feb
(18) |
Mar
(10) |
Apr
(28) |
May
(28) |
Jun
|
Jul
(27) |
Aug
(9) |
Sep
(21) |
Oct
(2) |
Nov
(2) |
Dec
(11) |
2011 |
Jan
|
Feb
(2) |
Mar
(4) |
Apr
(2) |
May
(2) |
Jun
(44) |
Jul
(9) |
Aug
(2) |
Sep
(12) |
Oct
(7) |
Nov
(11) |
Dec
(7) |
2012 |
Jan
(5) |
Feb
|
Mar
(9) |
Apr
(9) |
May
(12) |
Jun
|
Jul
(13) |
Aug
(3) |
Sep
(3) |
Oct
(1) |
Nov
(1) |
Dec
(10) |
2013 |
Jan
(21) |
Feb
(3) |
Mar
(4) |
Apr
|
May
(3) |
Jun
(2) |
Jul
(3) |
Aug
(3) |
Sep
(3) |
Oct
|
Nov
|
Dec
(4) |
2014 |
Jan
(7) |
Feb
|
Mar
(1) |
Apr
|
May
(2) |
Jun
|
Jul
(4) |
Aug
(2) |
Sep
|
Oct
(1) |
Nov
|
Dec
|
2016 |
Jan
|
Feb
|
Mar
(1) |
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2017 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2018 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2021 |
Jan
|
Feb
|
Mar
|
Apr
(3) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2022 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: David M. <dav...@fi...> - 2011-06-28 16:54:23
|
This patch looks good. I applied it. Also, just for good measure, I added code to handle the other semaphore failures in the same method in the same way. I don't think waiting on sem(2) is an issue though, if a connection is immediately available, then the listener will just hand off to it, otherwise, it will fork a child to wait on sem(2) while it loops back to handle more client connections. Dave On 06/28/2011 05:57 AM, Renat Sabitov wrote: > Hi David, > > I am doing some sort of stability testing. What exactly I am doing is > starting two dozens of sqlrelay clients constantly connecting, > executing and disconnecting. Than I run > killall -s 9 sqlr-connection-oracle8 several times to emulate abnormal > exiting of connection processes. > > What I found is that sometimes a forked listener process can't get a > connection and stopped waiting for sem(2) forever in getAConnection. > Ok, not a big deal, the situation is quite synthetic and could be > solved by starting new connection process. > > But then I tried to stop sqlrelay and found that some listener > processes start runnig an infinite loop in getAConnection having > wait() and signal() returning -1. Only the manual killing of this > listeners helps. > > I think it would be a good idea to check semaphore access in this loop > for errors. The patch is attached. > > What more, it might be meaningful to wait for some amount of time on > sem(2), not forever. The main listener than could take another client > and start more connections. > > > ------------------------------------------------------------------------------ > All of the data generated in your IT infrastructure is seriously valuable. > Why? It contains a definitive record of application performance, security > threats, fraudulent activity, and more. Splunk takes this data and makes > sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-d2d-c2 > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com > > > _______________________________________________ > Sqlrelay-discussion mailing list > Sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion > > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com _______________________________________________________ Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting http://www.doteasy.com |
From: David M. <dav...@fi...> - 2011-06-28 16:35:24
|
Thanks, I removed the files that were getting created from the .in files. Oops :) I'm not sure about the Makefile patch though, the init script looks in the file /etc/sqlrelay for a list of instances to start at boot, or at least that's what it's supposed to do. We could move that file to /etc/sqlrelay/sqlrelay but it would require that the init script be modified too. A long time ago, some guys got SQL Relay into the debian repository and submitted similar debian directories for rudiments and sqlrelay. I initially incorporated them into CVS, but later they asked me to pull them back out. Apparently the debian package archive maintainers prefer to add them as patches rather than have them in the software distro. I think it's so they can tweak on them without having to coordinate with anyone. Dave On 06/25/2011 10:00 PM, Renat Sabitov wrote: > On 25.06.2011 05:56, David Muse wrote: >> >> I'm ready to make the 0.42 release. I wouldn't mind waiting a day or >> two though, if you think you can get the Debian build working quickly. >> > Although I got debian build working, I am not sure that it is correct. > I used the old debian directory from 0.39 and modified it for the new > debian standards and the new release. It builds well on Debian testing > and stable with rudiments 0.33, but I am not a debian developer and > really do not know how to put the package in debian repository. I > tried some time ago to put 0.41 through mentors.debian.net, but my > efforts was not succeded. > > Further to patches and thoughs. It might be good to delete these files > since they are made from .in: > init/netbsd/rc.d/sqlrelay > init/freebsd/rc.d/sqlrelay > > debian.init.patch - there is a directory /etc/sqlrelay with file > /etc/sqlrelay/sqlrelay > > 2 debian directories for rudiments and sqlrelay to build debian packages. > > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense.. > http://p.sf.net/sfu/splunk-d2d-c1 > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com > > > _______________________________________________ > Sqlrelay-discussion mailing list > Sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion > > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com _______________________________________________________ Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting http://www.doteasy.com |
From: Renat S. <sr...@st...> - 2011-06-28 09:58:03
|
Hi David, I am doing some sort of stability testing. What exactly I am doing is starting two dozens of sqlrelay clients constantly connecting, executing and disconnecting. Than I run killall -s 9 sqlr-connection-oracle8 several times to emulate abnormal exiting of connection processes. What I found is that sometimes a forked listener process can't get a connection and stopped waiting for sem(2) forever in getAConnection. Ok, not a big deal, the situation is quite synthetic and could be solved by starting new connection process. But then I tried to stop sqlrelay and found that some listener processes start runnig an infinite loop in getAConnection having wait() and signal() returning -1. Only the manual killing of this listeners helps. I think it would be a good idea to check semaphore access in this loop for errors. The patch is attached. What more, it might be meaningful to wait for some amount of time on sem(2), not forever. The main listener than could take another client and start more connections. -- Renat Sabitov e-mail: sr...@st... Stack Soft jid: sr...@ja... |
From: Renat S. <sr...@st...> - 2011-06-27 13:48:00
|
Hi David, I am afraid there is an issue with counting currentconnections. From one perspective it is counted with countConnections() from shm. From another this variable is changed by incConnections() and decConnections(). I made the patch with fork() to have 100% correct information on how many connections are started by scaler, because connection processes not always incremented and decremented currentconnections value in shared memory. With my patch, scaler do not look on shm, but do counting by itself. There is a problem in my patch: scaler do not care about connections started manually or by the sqlr-start. With your adoption of the patch currentconnections in scaler is changed by two ways, and this is actually an issue. I think that scaler should take control under shm->totalconnections for dynamically spawned connections. I've created a patch against current CVS, please take a look and write your thoughts. I tested patch by killing connections with SIGKILL. Only the first connection (= not spawned) was unable to decrement counter, therefore status shows 2 connections in scaler's view with only 1 process started. Please notice buggy status of open connections/cursors, this is the result of my massacre. Open Server Connections: 13 Opened Server Connections: 57 Open Client Connections: 34203 Opened Client Connections: 34203 Open Server Cursors: 65 Opened Server Cursors: 294 Times New Cursor Used: 68578 Times Cursor Reused: 68592 Total Queries: 68587 Total Errors: 0 Forked Listeners: 0 Scaler's view: Connections: 2 Sessions: 2 Semaphores: +---------------------------------------------+ | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | +---+---+---+---+---+---+---+---+---+---+-----+ | 1 | 1 | 1 | 3 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | +---------------------------------------------+ -- Renat Sabitov e-mail: sr...@st... Stack Soft jid: sr...@ja... |
From: Renat S. <sr...@st...> - 2011-06-26 02:00:56
|
On 25.06.2011 05:56, David Muse wrote: > > I'm ready to make the 0.42 release. I wouldn't mind waiting a day or > two though, if you think you can get the Debian build working quickly. > Although I got debian build working, I am not sure that it is correct. I used the old debian directory from 0.39 and modified it for the new debian standards and the new release. It builds well on Debian testing and stable with rudiments 0.33, but I am not a debian developer and really do not know how to put the package in debian repository. I tried some time ago to put 0.41 through mentors.debian.net, but my efforts was not succeded. Further to patches and thoughs. It might be good to delete these files since they are made from .in: init/netbsd/rc.d/sqlrelay init/freebsd/rc.d/sqlrelay debian.init.patch - there is a directory /etc/sqlrelay with file /etc/sqlrelay/sqlrelay 2 debian directories for rudiments and sqlrelay to build debian packages. |
From: David M. <dav...@fi...> - 2011-06-25 01:56:51
|
Cool. I applied your patch. I've been fighting DESTDIR with ruby for years. The helper scripts in the src/api/ruby directory try to insert DESTDIR into their output, but they don't always succeed. It's really complicated why, but if a platform builds its packages by calling: export DESTDIR=somedir make install then everything works fine without your patch, but if a platform builds its packages by calling: DESTDIR=somedir make install then it fails to insert DESTDIR. Apparently the rpmbuild command uses the first method, but the command for building debian packages uses the second. It always seemed like I could get it to work for either platform, but never both. When I first applied your patch, it ended up prepending DESTDIR twice when building RPM's, but I realized I could modify the ruby helper scripts to strip $(DESTDIR) out of their output and just let the Makefile handle DESTDIR itself. Now, RPM's build fine. I committed the code to CVS, let me know if it works on Debian too. I'm ready to make the 0.42 release. I wouldn't mind waiting a day or two though, if you think you can get the Debian build working quickly. Dave On 06/24/2011 10:32 AM, Renat Sabitov wrote: > Hello, David. > > I am trying to debianize 0.42 and found that install goal for ruby api > do not uses $(DESTDIR). The patch is attached. > > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense.. > http://p.sf.net/sfu/splunk-d2d-c1 > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com > > > _______________________________________________ > Sqlrelay-discussion mailing list > Sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion > > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com _______________________________________________________ Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting http://www.doteasy.com |
From: Renat S. <sr...@st...> - 2011-06-24 14:32:48
|
Hello, David. I am trying to debianize 0.42 and found that install goal for ruby api do not uses $(DESTDIR). The patch is attached. -- Renat Sabitov e-mail: sr...@st... Stack Soft jid: sr...@ja... |
From: David M. <dav...@fi...> - 2011-06-23 18:46:30
|
Yeah, I hate semaphores. They're great in theory, not so great in practice. I think resetting sem(8) to 0 before starting the connection will solve the problem. I'm not sure if it's the best solution, but it should take care of it. We might be able to do without sem(8), that's a fairly invasive change though. Maybe in the next release. Dave On 06/23/2011 02:58 AM, Renat Sabitov wrote: > Yes it could :(. Semaphores are such a thing one should stay away if it > is possible. Could we just don't use sem(8)? It is used to signal the > scaler that connection is started and current number of connections is > incremented. Now with fork() scaler is aware of the connections spawned > and actually do not have to wait to get the number of connections. > > If not, it is possible to wait for a small amount of time for sem(8) > after connection process is killed to decrement sem(8). > > 23.06.2011 07:45, David Muse wrote: >> Now that I'm testing this patch, it appears to work, but I think it >> might introduce a race condition. >> >> If the new connection takes just slightly longer than 10 seconds to >> start up, the scaler's wait could fall through, then the connection >> could still signal on sem(8) before the scaler kills it, leaving sem(8) >> incremented, and nothing to wait on it. >> >> What do you think? > _______________________________________________________ Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting http://www.doteasy.com |
From: David M. <dav...@fi...> - 2011-06-23 16:08:22
|
I think it does that already and then sets a bunch of macros to map sqlite3 calls to older calls at the top of the file. It looks like I forgot to add a mapping for sqlite3_free. It should be a simple fix. Dave On 06/23/2011 05:58 AM, Renat Sabitov wrote: > David, > > I am trying to compile CVS version of sqlrelay. > > You have sqlite3_free() call in sqlitecursor::runQuery, which brokes the > compatibility with sqlite.h. It might worth to change autoconf to test > whether sqlite3 is installed or not. > _______________________________________________________ Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting http://www.doteasy.com |
From: David M. <dav...@fi...> - 2011-06-23 16:04:55
|
Yes, I've been working really hard for the past few days to get a release out. There's this issue, and a postgresql memory leak that need to be fixed, then it should be ready. Dave On 06/23/2011 03:21 AM, Renat Sabitov wrote: > 23.06.2011 07:32, David Muse wrote: >> Nevermind patching against current CVS, I got it figured out :) > Thanks a lot for that, because we actually do not use the current CVS > version. > > Do you plan to make a new release for sqlrelay? It is about 2 years from > the 0.41, may be it is the time for step up? :) > _______________________________________________________ Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting http://www.doteasy.com |
From: Renat S. <sr...@st...> - 2011-06-23 09:58:23
|
David, I am trying to compile CVS version of sqlrelay. You have sqlite3_free() call in sqlitecursor::runQuery, which brokes the compatibility with sqlite.h. It might worth to change autoconf to test whether sqlite3 is installed or not. -- Renat Sabitov e-mail: sr...@st... Stack Soft jid: sr...@ja... |
From: Renat S. <sr...@st...> - 2011-06-23 07:21:36
|
23.06.2011 07:32, David Muse wrote: > Nevermind patching against current CVS, I got it figured out :) Thanks a lot for that, because we actually do not use the current CVS version. Do you plan to make a new release for sqlrelay? It is about 2 years from the 0.41, may be it is the time for step up? :) -- Renat Sabitov e-mail: sr...@st... Stack Soft jid: sr...@ja... |
From: Renat S. <sr...@st...> - 2011-06-23 06:58:14
|
Yes it could :(. Semaphores are such a thing one should stay away if it is possible. Could we just don't use sem(8)? It is used to signal the scaler that connection is started and current number of connections is incremented. Now with fork() scaler is aware of the connections spawned and actually do not have to wait to get the number of connections. If not, it is possible to wait for a small amount of time for sem(8) after connection process is killed to decrement sem(8). 23.06.2011 07:45, David Muse wrote: > Now that I'm testing this patch, it appears to work, but I think it > might introduce a race condition. > > If the new connection takes just slightly longer than 10 seconds to > start up, the scaler's wait could fall through, then the connection > could still signal on sem(8) before the scaler kills it, leaving sem(8) > incremented, and nothing to wait on it. > > What do you think? -- Renat Sabitov e-mail: sr...@st... Stack Soft jid: sr...@ja... |
From: David M. <dav...@fi...> - 2011-06-23 03:45:48
|
Now that I'm testing this patch, it appears to work, but I think it might introduce a race condition. If the new connection takes just slightly longer than 10 seconds to start up, the scaler's wait could fall through, then the connection could still signal on sem(8) before the scaler kills it, leaving sem(8) incremented, and nothing to wait on it. What do you think? Dave On 06/22/2011 11:32 PM, David Muse wrote: > Nevermind patching against current CVS, I got it figured out :) > > On 06/20/2011 04:59 PM, David Muse wrote: >> Ahh, yes. Such an obvious solution :) >> >> Could you resend me your patch against the current CVS code? There >> are some differences vs. the code you patched against. I think >> mainly I made it so that it always uses fork(). >> >> Dave >> >> On 06/15/2011 12:55 AM, Renat Sabitov wrote: >>> 15.06.2011 00:16, David Muse wrote: >>>> The >>>> connections could only signal sem8 if they were started by the scaler, >>>> but what would be the right way for a connection to know it was >>>> started >>>> by the scaler? I'm open to suggestions here. >>> I suppose it could be a new option, say "-scaled". :) >>> >> > _______________________________________________________ Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting http://www.doteasy.com |
From: David M. <dav...@fi...> - 2011-06-23 03:33:22
|
Nevermind patching against current CVS, I got it figured out :) On 06/20/2011 04:59 PM, David Muse wrote: > Ahh, yes. Such an obvious solution :) > > Could you resend me your patch against the current CVS code? There > are some differences vs. the code you patched against. I think mainly > I made it so that it always uses fork(). > > Dave > > On 06/15/2011 12:55 AM, Renat Sabitov wrote: >> 15.06.2011 00:16, David Muse wrote: >>> The >>> connections could only signal sem8 if they were started by the scaler, >>> but what would be the right way for a connection to know it was started >>> by the scaler? I'm open to suggestions here. >> I suppose it could be a new option, say "-scaled". :) >> > _______________________________________________________ Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting http://www.doteasy.com |
From: David M. <dav...@fi...> - 2011-06-20 21:00:12
|
Ahh, yes. Such an obvious solution :) Could you resend me your patch against the current CVS code? There are some differences vs. the code you patched against. I think mainly I made it so that it always uses fork(). Dave On 06/15/2011 12:55 AM, Renat Sabitov wrote: > 15.06.2011 00:16, David Muse wrote: >> The >> connections could only signal sem8 if they were started by the scaler, >> but what would be the right way for a connection to know it was started >> by the scaler? I'm open to suggestions here. > I suppose it could be a new option, say "-scaled". :) > _______________________________________________________ Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting http://www.doteasy.com |
From: David M. <dav...@fi...> - 2011-06-15 15:48:03
|
Hi Keith, Yeah, I'm not sure there's a solution here for you with SQL Relay, at least not currently. Along with general windows compatibility, I've been working on an ODBC driver, but it's not working well enough for general release yet. Man, I wish there was an easy solution here. I don't know of any other products that would do what you're looking to do. I don't know if it's an option for you guys or not, but I do contract work related to SQL Relay all the time. It would take some work, but I might be able to get the ODBC driver working well enough for you to use. Right now the ODBC driver is about midway through my list of priorities, but if there was a contract involved, then it would bump up to the top. Just food for thought. Thanks for your interest, sorry there wasn't an immediate solution. Dave dav...@fi... On 06/14/2011 07:09 PM, Keith wrote: > Hi David, > Thanks for the detailed reply. We've been asked to punch > some holes in our firewalls to allow some database connections through. > We can do this but we would prefer to put some kind of SQL proxy in > between the servers and had hoped to use SQL Relay for this. There are > two database servers that we need to connect to, both are MS SQL Servers > (2000 or 2005) and the clients are both MS Servers. I don't think the DB > drivers could be changed but will ask. We could maybe get the clients to > connect using ODBC if SQL Relay would accept that ? > > If not do you know of any other product that might fit the bill ? I have > looked at GreenSQL but they want a monthly subscription that I don't > agree with then there's SQLfirewall that seems like it might have been > viable but has been abandoned by it's creators and their website is > broken so I am unable to buy it or to email them ! > > Anyway, thanks for info. I had been planning on spending most of > tomorrow trying to trouble shoot SQL Relay so you've saved there. > > Cheers > Keith > > On 14/06/2011 19:04, David Muse wrote: >> Keith, >> >> SQL Relay can connect to an MS SQL Server database, but cannot accept >> connections from MS SQL Server clients. An MS SQL Server client speaks >> to the database using the "Tabular Datastream Protocol" but SQL Relay >> does not know how to speak that protocol, instead it has it's own protocol. >> >> There are SQL Relay drivers for Perl DBI, Python DB, Ruby DBI, PHP Pear >> DBI and Zope so if you have an application that uses one of those >> frameworks, then you can usually just point it at SQL Relay, it will >> load the appropriate driver and "just work" but otherwise if you want >> your app to use SQL Relay, you'd have to rewrite it to use one of the >> SQL Relay API's. >> >> Alternatively, if your app uses MySQL or PostgreSQL, then there is the >> possibility of using one of the drop-in replacement libraries that maps >> Mysql/Postgresql API calls to SQL Relay API calls. >> >> All that said, none of this works on Windows yet. I've been working on >> it for a while, but it's not ready. >> >> If you tell me more about your application environment, I might be able >> to come up with some way of making it work, but from what I've gathered, >> it doesn't look like there's a way to do it directly. >> >> Dave >> dav...@fi... >> >> On 06/10/2011 06:21 AM, keith wrote: >>> Hi, We want to put a SQL Relay server between our Web servers and a >>> MSSQL 2005 server. We have a dedicated SQL Relay server and can start >>> SQLRelay and can see it binding to an interface on port 1433 and can >>> also see three connections going through to our MS SQL Server and it all >>> looks good. But doesn't seem to be working. >>> >>> From a windows machine if we create a UDL File and try to make a >>> connection directly to the SQL Server then we can. If we make a >>> connection to the SQL server via SQL Relay we can't connect. We are >>> trying to log in with user=user1 password=password1 although are not >>> sure about what the database name should be so have tried a few and each >>> one doesn't work, we have also tried getting the list of databases on >>> the server but that doesn't work either. >>> >>> >>> If we use the UDL tool and point it to a non existing server / ip we get >>> the following error. >>> ================================================================================= >>> Test connection failed because of an error in initalzing provider. >>> [DBNETLIB][ConnectionOpen(Connect()).]SQL Server does not exist or >>> access denied. >>> >>> No other pop box appears! >>> >>> Then if we point the UDL tool to the SQL Relay server we get the >>> following error. >>> ============================================================================== >>> [DBNETLIB][ConnectionRead(recv()).]General network error. >>> Check your network documentation. >>> >>> If we click ok then another popup box appears with the following. >>> >>> Login failed. Catalog information cannot be retrieved. >>> >>> >>> We know that SQL Relay is connection to the MSSQL Server as we can see >>> that there are three connections to the specified database from SQLReay >>> in the sql enterprise manager. >>> >>> >>> When we start sqlr with sqlr-start -id mssql there is a message >>> about the cache manager that we aren't sure about. >>> >>> Starting cache manager: >>> sqlr-cachemanager >>> Warning: using default id. >>> >>> I have tried killing the process and restarting it with >>> sqlr-cachemanager -id mssql and it starts but we are still unable to >>> connect. >>> >>> >>> This is my sqlrelay.conf >>> >>> >>> <?xml version="1.0"?> >>> <!DOCTYPE instances SYSTEM "sqlrelay.dtd"> >>> >>> <instances> >>> >>> <!-- Regular SQL Relay Instance --> >>> <instance id="mssql" addresses="x.x.x.x" port="1433" >>> socket="/var/run/keith.socket" dbase="freetds" connections="3" >>> maxconnections="15" maxqueuelength="5" growby="1" ttl="60" >>> maxsessioncount="10" endofsession="commit" sessiontimeout="600" >>> runasuser="root" runasgroup="wheel" cursors="5" authtier="listener" >>> handoff="pass" deniedips="" allowedips="" debug="none" >>> maxquerysize="65536" maxstringbindvaluelength="4000" >>> maxlobbindvaluelength="71680" idleclienttimeout="-1" maxlisteners="-1" >>> listenertimeout="0" reloginatstart="no" timequeriessec="-1" >>> timequeriesusec="-1"> >>> <users> >>> <user user="user1" password="password1"/> >>> <user user="user2" password="password2"/> >>> <user user="user3" password="password3"/> >>> </users> >>> <connections> >>> <connection connectionid="db1" >>> string="sybase=/etc/freetds.conf;user=thesqluser;password=thesqlpassword;server=sqlserver80;db=thesqldatabase;" >>> metric="1"/> >>> </connections> >>> </instance> >>> </instances> >>> >>> >>> # cat /etc/freetds.conf >>> # $Id: freetds.conf,v 1.12 2007/12/25 06:02:36 jklowden Exp $ >>> # >>> # This file is installed by FreeTDS if no file by the same >>> # name is found in the installation directory. >>> # >>> # For information about the layout of this file and its settings, >>> # see the freetds.conf manpage "man freetds.conf". >>> >>> # Global settings are overridden by those in a database >>> # server specific section >>> [global] >>> # TDS protocol version; the OpenBSD package defaults to 8.0 to >>> # avoid plaintext passwords on the wire. You may set another >>> default >>> # here to communicate with certain types of server, or override it >>> # on a per-server basis below. >>> ; tds version = 4.2 >>> >>> # Whether to write a TDSDUMP file for diagnostic purposes >>> # (setting this to /tmp is insecure on a multi-user system) >>> ; dump file = /tmp/freetds.log >>> ; debug flags = 0xffff >>> >>> # Command and connection timeouts >>> ; timeout = 10 >>> ; connect timeout = 10 >>> >>> # If you get out-of-memory errors, it may mean that your client >>> # is trying to allocate a huge buffer for a TEXT field. >>> # Try setting 'text size' to a more reasonable limit >>> text size = 64512 >>> >>> # A typical Microsoft server >>> [egServer70] >>> host = 192.168.1.2 >>> port = 1433 >>> tds version = 7.0 >>> >>> [sqlserver80] >>> host = x.y.z.a >>> port = 1433 >>> tds version = 8.0 >>> >>> >>> >>> # sqlr-start -id mssql >>> >>> Starting listener: >>> sqlr-listener -id mssql -config /usr/local/firstworks/etc/sqlrelay.conf >>> >>> Starting 3 connections to db1 : >>> sqlr-connection-freetds -id mssql -connectionid db1 -config >>> /usr/local/firstworks/etc/sqlrelay.conf >>> sqlr-connection-freetds -id mssql -connectionid db1 -config >>> /usr/local/firstworks/etc/sqlrelay.conf >>> sqlr-connection-freetds -id mssql -connectionid db1 -config >>> /usr/local/firstworks/etc/sqlrelay.conf >>> >>> Starting scaler: >>> sqlr-scaler -id mssql -config /usr/local/firstworks/etc/sqlrelay.conf >>> >>> Starting cache manager: >>> sqlr-cachemanager >>> Warning: using default id. >>> >>> >>> >>> Does anyone have any suggestions ? >>> >>> Thanks >>> Keith >>> >>> >>> ------------------------------------------------------------------------------ >>> EditLive Enterprise is the world's most technically advanced content >>> authoring tool. Experience the power of Track Changes, Inline Image >>> Editing and ensure content is compliant with Accessibility Checking. >>> http://p.sf.net/sfu/ephox-dev2dev >>> _______________________________________________ >>> Sqlrelay-discussion mailing list >>> Sql...@li... >>> https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion >>> >>> >>> _______________________________________________________ >>> Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting >>> http://www.doteasy.com >> >> _______________________________________________________ >> Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting >> http://www.doteasy.com >> >> ------------------------------------------------------------------------------ >> EditLive Enterprise is the world's most technically advanced content >> authoring tool. Experience the power of Track Changes, Inline Image >> Editing and ensure content is compliant with Accessibility Checking. >> http://p.sf.net/sfu/ephox-dev2dev >> _______________________________________________ >> Sqlrelay-discussion mailing list >> Sql...@li... >> https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion > ------------------------------------------------------------------------------ > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > Sqlrelay-discussion mailing list > Sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion > > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com _______________________________________________________ Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting http://www.doteasy.com |
From: Renat S. <sr...@st...> - 2011-06-15 04:55:31
|
15.06.2011 00:16, David Muse wrote: > The > connections could only signal sem8 if they were started by the scaler, > but what would be the right way for a connection to know it was started > by the scaler? I'm open to suggestions here. I suppose it could be a new option, say "-scaled". :) -- Renat Sabitov e-mail: sr...@st... Stack Soft jid: sr...@ja... |
From: Keith <ke...@sc...> - 2011-06-14 23:10:05
|
Hi David, Thanks for the detailed reply. We've been asked to punch some holes in our firewalls to allow some database connections through. We can do this but we would prefer to put some kind of SQL proxy in between the servers and had hoped to use SQL Relay for this. There are two database servers that we need to connect to, both are MS SQL Servers (2000 or 2005) and the clients are both MS Servers. I don't think the DB drivers could be changed but will ask. We could maybe get the clients to connect using ODBC if SQL Relay would accept that ? If not do you know of any other product that might fit the bill ? I have looked at GreenSQL but they want a monthly subscription that I don't agree with then there's SQLfirewall that seems like it might have been viable but has been abandoned by it's creators and their website is broken so I am unable to buy it or to email them ! Anyway, thanks for info. I had been planning on spending most of tomorrow trying to trouble shoot SQL Relay so you've saved there. Cheers Keith On 14/06/2011 19:04, David Muse wrote: > Keith, > > SQL Relay can connect to an MS SQL Server database, but cannot accept > connections from MS SQL Server clients. An MS SQL Server client speaks > to the database using the "Tabular Datastream Protocol" but SQL Relay > does not know how to speak that protocol, instead it has it's own protocol. > > There are SQL Relay drivers for Perl DBI, Python DB, Ruby DBI, PHP Pear > DBI and Zope so if you have an application that uses one of those > frameworks, then you can usually just point it at SQL Relay, it will > load the appropriate driver and "just work" but otherwise if you want > your app to use SQL Relay, you'd have to rewrite it to use one of the > SQL Relay API's. > > Alternatively, if your app uses MySQL or PostgreSQL, then there is the > possibility of using one of the drop-in replacement libraries that maps > Mysql/Postgresql API calls to SQL Relay API calls. > > All that said, none of this works on Windows yet. I've been working on > it for a while, but it's not ready. > > If you tell me more about your application environment, I might be able > to come up with some way of making it work, but from what I've gathered, > it doesn't look like there's a way to do it directly. > > Dave > dav...@fi... > > On 06/10/2011 06:21 AM, keith wrote: >> Hi, We want to put a SQL Relay server between our Web servers and a >> MSSQL 2005 server. We have a dedicated SQL Relay server and can start >> SQLRelay and can see it binding to an interface on port 1433 and can >> also see three connections going through to our MS SQL Server and it all >> looks good. But doesn't seem to be working. >> >> From a windows machine if we create a UDL File and try to make a >> connection directly to the SQL Server then we can. If we make a >> connection to the SQL server via SQL Relay we can't connect. We are >> trying to log in with user=user1 password=password1 although are not >> sure about what the database name should be so have tried a few and each >> one doesn't work, we have also tried getting the list of databases on >> the server but that doesn't work either. >> >> >> If we use the UDL tool and point it to a non existing server / ip we get >> the following error. >> ================================================================================= >> Test connection failed because of an error in initalzing provider. >> [DBNETLIB][ConnectionOpen(Connect()).]SQL Server does not exist or >> access denied. >> >> No other pop box appears! >> >> Then if we point the UDL tool to the SQL Relay server we get the >> following error. >> ============================================================================== >> [DBNETLIB][ConnectionRead(recv()).]General network error. >> Check your network documentation. >> >> If we click ok then another popup box appears with the following. >> >> Login failed. Catalog information cannot be retrieved. >> >> >> We know that SQL Relay is connection to the MSSQL Server as we can see >> that there are three connections to the specified database from SQLReay >> in the sql enterprise manager. >> >> >> When we start sqlr with sqlr-start -id mssql there is a message >> about the cache manager that we aren't sure about. >> >> Starting cache manager: >> sqlr-cachemanager >> Warning: using default id. >> >> I have tried killing the process and restarting it with >> sqlr-cachemanager -id mssql and it starts but we are still unable to >> connect. >> >> >> This is my sqlrelay.conf >> >> >> <?xml version="1.0"?> >> <!DOCTYPE instances SYSTEM "sqlrelay.dtd"> >> >> <instances> >> >> <!-- Regular SQL Relay Instance --> >> <instance id="mssql" addresses="x.x.x.x" port="1433" >> socket="/var/run/keith.socket" dbase="freetds" connections="3" >> maxconnections="15" maxqueuelength="5" growby="1" ttl="60" >> maxsessioncount="10" endofsession="commit" sessiontimeout="600" >> runasuser="root" runasgroup="wheel" cursors="5" authtier="listener" >> handoff="pass" deniedips="" allowedips="" debug="none" >> maxquerysize="65536" maxstringbindvaluelength="4000" >> maxlobbindvaluelength="71680" idleclienttimeout="-1" maxlisteners="-1" >> listenertimeout="0" reloginatstart="no" timequeriessec="-1" >> timequeriesusec="-1"> >> <users> >> <user user="user1" password="password1"/> >> <user user="user2" password="password2"/> >> <user user="user3" password="password3"/> >> </users> >> <connections> >> <connection connectionid="db1" >> string="sybase=/etc/freetds.conf;user=thesqluser;password=thesqlpassword;server=sqlserver80;db=thesqldatabase;" >> metric="1"/> >> </connections> >> </instance> >> </instances> >> >> >> # cat /etc/freetds.conf >> # $Id: freetds.conf,v 1.12 2007/12/25 06:02:36 jklowden Exp $ >> # >> # This file is installed by FreeTDS if no file by the same >> # name is found in the installation directory. >> # >> # For information about the layout of this file and its settings, >> # see the freetds.conf manpage "man freetds.conf". >> >> # Global settings are overridden by those in a database >> # server specific section >> [global] >> # TDS protocol version; the OpenBSD package defaults to 8.0 to >> # avoid plaintext passwords on the wire. You may set another >> default >> # here to communicate with certain types of server, or override it >> # on a per-server basis below. >> ; tds version = 4.2 >> >> # Whether to write a TDSDUMP file for diagnostic purposes >> # (setting this to /tmp is insecure on a multi-user system) >> ; dump file = /tmp/freetds.log >> ; debug flags = 0xffff >> >> # Command and connection timeouts >> ; timeout = 10 >> ; connect timeout = 10 >> >> # If you get out-of-memory errors, it may mean that your client >> # is trying to allocate a huge buffer for a TEXT field. >> # Try setting 'text size' to a more reasonable limit >> text size = 64512 >> >> # A typical Microsoft server >> [egServer70] >> host = 192.168.1.2 >> port = 1433 >> tds version = 7.0 >> >> [sqlserver80] >> host = x.y.z.a >> port = 1433 >> tds version = 8.0 >> >> >> >> # sqlr-start -id mssql >> >> Starting listener: >> sqlr-listener -id mssql -config /usr/local/firstworks/etc/sqlrelay.conf >> >> Starting 3 connections to db1 : >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlrelay.conf >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlrelay.conf >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlrelay.conf >> >> Starting scaler: >> sqlr-scaler -id mssql -config /usr/local/firstworks/etc/sqlrelay.conf >> >> Starting cache manager: >> sqlr-cachemanager >> Warning: using default id. >> >> >> >> Does anyone have any suggestions ? >> >> Thanks >> Keith >> >> >> ------------------------------------------------------------------------------ >> EditLive Enterprise is the world's most technically advanced content >> authoring tool. Experience the power of Track Changes, Inline Image >> Editing and ensure content is compliant with Accessibility Checking. >> http://p.sf.net/sfu/ephox-dev2dev >> _______________________________________________ >> Sqlrelay-discussion mailing list >> Sql...@li... >> https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion >> >> >> _______________________________________________________ >> Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting >> http://www.doteasy.com > > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com > > ------------------------------------------------------------------------------ > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > Sqlrelay-discussion mailing list > Sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion |
From: David M. <dav...@fi...> - 2011-06-14 20:17:12
|
Hi Renat, Man, this is complicated :) I need to look at the patch carefully, but at first glance, it appears to solve the problem where the sqlr-connection starts up and either gets hung (maybe trying to log into the db) or dies before signalling on sem8. You're also right about it being an error that sem8 is greater than 0 when the scaler starts - incremented by the connections that were not started by the scaler. This is actually tricky to solve. sqlr-start probably shouldn't wait on them, because people sometimes use their own scripts to start the listener, connections and scaler. The scaler could wait on them when it starts up, but ideally you should be able to manually start up sqlr-connections whenever you want. It shouldn't be a rule that you have to start them before starting the scaler. The connections could only signal sem8 if they were started by the scaler, but what would be the right way for a connection to know it was started by the scaler? I'm open to suggestions here. Dave On 06/10/2011 09:57 AM, Renat Sabitov wrote: > Hi! > > Sqlrelay got stuck recently on one of our production servers. Symptoms > quite strange: there were 1 scaler, 1 connection and 17 listeners. I > checked the status and got these results: > > Open Server Connections: 1 > Opened Server Connections: 4016 > > Open Client Connections: 0 > Opened Client Connections: 510926 > > Open Server Cursors: 3 > Opened Server Cursors: 522974 > > Times New Cursor Used: 0 > Times Cursor Reused: 11604823 > > Total Queries: 6313112 > Total Errors: 4718 > > Forked Listeners: 16 > > Scaler's view: > Connections: 1 > Sessions: 18 > > Semaphores: > +---------------------------------------------+ > | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | > +---+---+---+---+---+---+---+---+---+---+-----+ > | 0 | 1 | 1 | 0 | 1 | 0 | 1 | 0 | 0 | 1 | 16 | > +---------------------------------------------+ > > Pay attention to semaphores. The normal state of them is > > +---------------------------------------------+ > | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | > +---+---+---+---+---+---+---+---+---+---+-----+ > | 0 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | > +---------------------------------------------+ > > There differences in semaphores 5,6,8 > > 5==0 and 6==1 means that listener got client request and signalled to > scaler to increase the number of connections if it is possible > (sqlrlistener::incrementSessionCount) > > Continious state of 6==1 signals that scaler stuck somewhere in the > cycle inside scaler::openMoreConnections(). > > 8==0 gives a clue that scaler is waiting for connection to fire up, > i.e. connects to DB and increment connection counter. > > By the way, it might be an error that the normal state of sem 8 is 1. > It is getting this state after the first non-scaled connection started > by sqlr-start and actually this semaphore changes state between 1 and > 2 instead of 0 and 1 while scaler is firing up new connections. It > might be a good idea to wait for sem8 in sqlr-start, isn't it? > > So why it is finally got state 0? I suppose it was the result of the > startup process of connection. Some connection processes started, but > got some errors and didn't signal sem8 and then exited. As a result > scaler stucked in an infinite wait on this semaphore. > > After all I came up with a solution to this issue. If scaler is > working in the "fork" mode, it is possible to wait for the connection > to fire up for a while using the timed semset->wait() and then just > terminate the buggy connection process. Please take a look on the diff > in attachment. I would be very grateful for any comments on it. > > > ------------------------------------------------------------------------------ > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com > > > _______________________________________________ > Sqlrelay-discussion mailing list > Sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion > > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com _______________________________________________________ Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting http://www.doteasy.com |
From: David M. <dav...@fi...> - 2011-06-14 18:21:46
|
Keith, SQL Relay can connect to an MS SQL Server database, but cannot accept connections from MS SQL Server clients. An MS SQL Server client speaks to the database using the "Tabular Datastream Protocol" but SQL Relay does not know how to speak that protocol, instead it has it's own protocol. There are SQL Relay drivers for Perl DBI, Python DB, Ruby DBI, PHP Pear DBI and Zope so if you have an application that uses one of those frameworks, then you can usually just point it at SQL Relay, it will load the appropriate driver and "just work" but otherwise if you want your app to use SQL Relay, you'd have to rewrite it to use one of the SQL Relay API's. Alternatively, if your app uses MySQL or PostgreSQL, then there is the possibility of using one of the drop-in replacement libraries that maps Mysql/Postgresql API calls to SQL Relay API calls. All that said, none of this works on Windows yet. I've been working on it for a while, but it's not ready. If you tell me more about your application environment, I might be able to come up with some way of making it work, but from what I've gathered, it doesn't look like there's a way to do it directly. Dave dav...@fi... On 06/10/2011 06:21 AM, keith wrote: > Hi, We want to put a SQL Relay server between our Web servers and a > MSSQL 2005 server. We have a dedicated SQL Relay server and can start > SQLRelay and can see it binding to an interface on port 1433 and can > also see three connections going through to our MS SQL Server and it all > looks good. But doesn't seem to be working. > > From a windows machine if we create a UDL File and try to make a > connection directly to the SQL Server then we can. If we make a > connection to the SQL server via SQL Relay we can't connect. We are > trying to log in with user=user1 password=password1 although are not > sure about what the database name should be so have tried a few and each > one doesn't work, we have also tried getting the list of databases on > the server but that doesn't work either. > > > If we use the UDL tool and point it to a non existing server / ip we get > the following error. > ================================================================================= > Test connection failed because of an error in initalzing provider. > [DBNETLIB][ConnectionOpen(Connect()).]SQL Server does not exist or > access denied. > > No other pop box appears! > > Then if we point the UDL tool to the SQL Relay server we get the > following error. > ============================================================================== > [DBNETLIB][ConnectionRead(recv()).]General network error. > Check your network documentation. > > If we click ok then another popup box appears with the following. > > Login failed. Catalog information cannot be retrieved. > > > We know that SQL Relay is connection to the MSSQL Server as we can see > that there are three connections to the specified database from SQLReay > in the sql enterprise manager. > > > When we start sqlr with sqlr-start -id mssql there is a message > about the cache manager that we aren't sure about. > > Starting cache manager: > sqlr-cachemanager > Warning: using default id. > > I have tried killing the process and restarting it with > sqlr-cachemanager -id mssql and it starts but we are still unable to > connect. > > > This is my sqlrelay.conf > > > <?xml version="1.0"?> > <!DOCTYPE instances SYSTEM "sqlrelay.dtd"> > > <instances> > > <!-- Regular SQL Relay Instance --> > <instance id="mssql" addresses="x.x.x.x" port="1433" > socket="/var/run/keith.socket" dbase="freetds" connections="3" > maxconnections="15" maxqueuelength="5" growby="1" ttl="60" > maxsessioncount="10" endofsession="commit" sessiontimeout="600" > runasuser="root" runasgroup="wheel" cursors="5" authtier="listener" > handoff="pass" deniedips="" allowedips="" debug="none" > maxquerysize="65536" maxstringbindvaluelength="4000" > maxlobbindvaluelength="71680" idleclienttimeout="-1" maxlisteners="-1" > listenertimeout="0" reloginatstart="no" timequeriessec="-1" > timequeriesusec="-1"> > <users> > <user user="user1" password="password1"/> > <user user="user2" password="password2"/> > <user user="user3" password="password3"/> > </users> > <connections> > <connection connectionid="db1" > string="sybase=/etc/freetds.conf;user=thesqluser;password=thesqlpassword;server=sqlserver80;db=thesqldatabase;" > metric="1"/> > </connections> > </instance> > </instances> > > > # cat /etc/freetds.conf > # $Id: freetds.conf,v 1.12 2007/12/25 06:02:36 jklowden Exp $ > # > # This file is installed by FreeTDS if no file by the same > # name is found in the installation directory. > # > # For information about the layout of this file and its settings, > # see the freetds.conf manpage "man freetds.conf". > > # Global settings are overridden by those in a database > # server specific section > [global] > # TDS protocol version; the OpenBSD package defaults to 8.0 to > # avoid plaintext passwords on the wire. You may set another > default > # here to communicate with certain types of server, or override it > # on a per-server basis below. > ; tds version = 4.2 > > # Whether to write a TDSDUMP file for diagnostic purposes > # (setting this to /tmp is insecure on a multi-user system) > ; dump file = /tmp/freetds.log > ; debug flags = 0xffff > > # Command and connection timeouts > ; timeout = 10 > ; connect timeout = 10 > > # If you get out-of-memory errors, it may mean that your client > # is trying to allocate a huge buffer for a TEXT field. > # Try setting 'text size' to a more reasonable limit > text size = 64512 > > # A typical Microsoft server > [egServer70] > host = 192.168.1.2 > port = 1433 > tds version = 7.0 > > [sqlserver80] > host = x.y.z.a > port = 1433 > tds version = 8.0 > > > > # sqlr-start -id mssql > > Starting listener: > sqlr-listener -id mssql -config /usr/local/firstworks/etc/sqlrelay.conf > > Starting 3 connections to db1 : > sqlr-connection-freetds -id mssql -connectionid db1 -config > /usr/local/firstworks/etc/sqlrelay.conf > sqlr-connection-freetds -id mssql -connectionid db1 -config > /usr/local/firstworks/etc/sqlrelay.conf > sqlr-connection-freetds -id mssql -connectionid db1 -config > /usr/local/firstworks/etc/sqlrelay.conf > > Starting scaler: > sqlr-scaler -id mssql -config /usr/local/firstworks/etc/sqlrelay.conf > > Starting cache manager: > sqlr-cachemanager > Warning: using default id. > > > > Does anyone have any suggestions ? > > Thanks > Keith > > > ------------------------------------------------------------------------------ > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > Sqlrelay-discussion mailing list > Sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion > > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com _______________________________________________________ Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting http://www.doteasy.com |
From: Renat S. <sr...@st...> - 2011-06-10 14:21:12
|
Hi! Sqlrelay got stuck recently on one of our production servers. Symptoms quite strange: there were 1 scaler, 1 connection and 17 listeners. I checked the status and got these results: Open Server Connections: 1 Opened Server Connections: 4016 Open Client Connections: 0 Opened Client Connections: 510926 Open Server Cursors: 3 Opened Server Cursors: 522974 Times New Cursor Used: 0 Times Cursor Reused: 11604823 Total Queries: 6313112 Total Errors: 4718 Forked Listeners: 16 Scaler's view: Connections: 1 Sessions: 18 Semaphores: +---------------------------------------------+ | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | +---+---+---+---+---+---+---+---+---+---+-----+ | 0 | 1 | 1 | 0 | 1 | 0 | 1 | 0 | 0 | 1 | 16 | +---------------------------------------------+ Pay attention to semaphores. The normal state of them is +---------------------------------------------+ | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | +---+---+---+---+---+---+---+---+---+---+-----+ | 0 | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | +---------------------------------------------+ There differences in semaphores 5,6,8 5==0 and 6==1 means that listener got client request and signalled to scaler to increase the number of connections if it is possible (sqlrlistener::incrementSessionCount) Continious state of 6==1 signals that scaler stuck somewhere in the cycle inside scaler::openMoreConnections(). 8==0 gives a clue that scaler is waiting for connection to fire up, i.e. connects to DB and increment connection counter. By the way, it might be an error that the normal state of sem 8 is 1. It is getting this state after the first non-scaled connection started by sqlr-start and actually this semaphore changes state between 1 and 2 instead of 0 and 1 while scaler is firing up new connections. It might be a good idea to wait for sem8 in sqlr-start, isn't it? So why it is finally got state 0? I suppose it was the result of the startup process of connection. Some connection processes started, but got some errors and didn't signal sem8 and then exited. As a result scaler stucked in an infinite wait on this semaphore. After all I came up with a solution to this issue. If scaler is working in the "fork" mode, it is possible to wait for the connection to fire up for a while using the timed semset->wait() and then just terminate the buggy connection process. Please take a look on the diff in attachment. I would be very grateful for any comments on it. -- Renat Sabitov e-mail: sr...@st... Stack Soft jid: sr...@ja... |
From: keith <ke...@sc...> - 2011-06-10 10:22:01
|
Hi, We want to put a SQL Relay server between our Web servers and a MSSQL 2005 server. We have a dedicated SQL Relay server and can start SQLRelay and can see it binding to an interface on port 1433 and can also see three connections going through to our MS SQL Server and it all looks good. But doesn't seem to be working. From a windows machine if we create a UDL File and try to make a connection directly to the SQL Server then we can. If we make a connection to the SQL server via SQL Relay we can't connect. We are trying to log in with user=user1 password=password1 although are not sure about what the database name should be so have tried a few and each one doesn't work, we have also tried getting the list of databases on the server but that doesn't work either. If we use the UDL tool and point it to a non existing server / ip we get the following error. ================================================================================= Test connection failed because of an error in initalzing provider. [DBNETLIB][ConnectionOpen(Connect()).]SQL Server does not exist or access denied. No other pop box appears! Then if we point the UDL tool to the SQL Relay server we get the following error. ============================================================================== [DBNETLIB][ConnectionRead(recv()).]General network error. Check your network documentation. If we click ok then another popup box appears with the following. Login failed. Catalog information cannot be retrieved. We know that SQL Relay is connection to the MSSQL Server as we can see that there are three connections to the specified database from SQLReay in the sql enterprise manager. When we start sqlr with sqlr-start -id mssql there is a message about the cache manager that we aren't sure about. Starting cache manager: sqlr-cachemanager Warning: using default id. I have tried killing the process and restarting it with sqlr-cachemanager -id mssql and it starts but we are still unable to connect. This is my sqlrelay.conf <?xml version="1.0"?> <!DOCTYPE instances SYSTEM "sqlrelay.dtd"> <instances> <!-- Regular SQL Relay Instance --> <instance id="mssql" addresses="x.x.x.x" port="1433" socket="/var/run/keith.socket" dbase="freetds" connections="3" maxconnections="15" maxqueuelength="5" growby="1" ttl="60" maxsessioncount="10" endofsession="commit" sessiontimeout="600" runasuser="root" runasgroup="wheel" cursors="5" authtier="listener" handoff="pass" deniedips="" allowedips="" debug="none" maxquerysize="65536" maxstringbindvaluelength="4000" maxlobbindvaluelength="71680" idleclienttimeout="-1" maxlisteners="-1" listenertimeout="0" reloginatstart="no" timequeriessec="-1" timequeriesusec="-1"> <users> <user user="user1" password="password1"/> <user user="user2" password="password2"/> <user user="user3" password="password3"/> </users> <connections> <connection connectionid="db1" string="sybase=/etc/freetds.conf;user=thesqluser;password=thesqlpassword;server=sqlserver80;db=thesqldatabase;" metric="1"/> </connections> </instance> </instances> # cat /etc/freetds.conf # $Id: freetds.conf,v 1.12 2007/12/25 06:02:36 jklowden Exp $ # # This file is installed by FreeTDS if no file by the same # name is found in the installation directory. # # For information about the layout of this file and its settings, # see the freetds.conf manpage "man freetds.conf". # Global settings are overridden by those in a database # server specific section [global] # TDS protocol version; the OpenBSD package defaults to 8.0 to # avoid plaintext passwords on the wire. You may set another default # here to communicate with certain types of server, or override it # on a per-server basis below. ; tds version = 4.2 # Whether to write a TDSDUMP file for diagnostic purposes # (setting this to /tmp is insecure on a multi-user system) ; dump file = /tmp/freetds.log ; debug flags = 0xffff # Command and connection timeouts ; timeout = 10 ; connect timeout = 10 # If you get out-of-memory errors, it may mean that your client # is trying to allocate a huge buffer for a TEXT field. # Try setting 'text size' to a more reasonable limit text size = 64512 # A typical Microsoft server [egServer70] host = 192.168.1.2 port = 1433 tds version = 7.0 [sqlserver80] host = x.y.z.a port = 1433 tds version = 8.0 # sqlr-start -id mssql Starting listener: sqlr-listener -id mssql -config /usr/local/firstworks/etc/sqlrelay.conf Starting 3 connections to db1 : sqlr-connection-freetds -id mssql -connectionid db1 -config /usr/local/firstworks/etc/sqlrelay.conf sqlr-connection-freetds -id mssql -connectionid db1 -config /usr/local/firstworks/etc/sqlrelay.conf sqlr-connection-freetds -id mssql -connectionid db1 -config /usr/local/firstworks/etc/sqlrelay.conf Starting scaler: sqlr-scaler -id mssql -config /usr/local/firstworks/etc/sqlrelay.conf Starting cache manager: sqlr-cachemanager Warning: using default id. Does anyone have any suggestions ? Thanks Keith |
From: keith <ke...@sc...> - 2011-06-10 08:46:47
|
Had a wee play with the address setting and it seems that the listener only binds if and IP address of an interface is given. address="" doesn't work address="0.0.0.0" doesn't work address="10.0.9.23" does work ! Getting closer to getting SQLRealy working now. Thanks keith On 09/06/2011 16:31, David Muse wrote: > Try adding an > > addresses="" > > field to the instance tag in your configuration. There's currently a > bug where if the tag is missing altogether, then it thinks you meant not > to listen on an inet port at all, even though the port is specified. > It's fixed in CVS, but a new version with the fix hasn't been released yet. > > David Muse > dav...@fi... > > On 06/08/2011 08:05 AM, Keith wrote: >> Hi, I am trying to get sqlrelay to connect to a mssql server. I have had >> 3 process / connections going to the MSSQL server but don't see anything >> listening on port 9000 that I can connect to. Here's my configuration, >> If someone could have a look at I would be really grateful. >> >> OS is Openbsd 4.9 x64 >> sqlrelay-0.41 >> rudiments-0.32.tar >> freetds-0.82.1pre20100728p3 >> >> freetds.conf >> [global] >> ; tds version = 4.2 >> ; dump file = /tmp/freetds.log >> ; debug flags = 0xffff >> ; timeout = 10 >> ; connect timeout = 10 >> text size = 64512 >> [egServer70] >> host = 192.168.1.2 >> port = 1433 >> tds version = 7.0 >> [sqlserver80] >> host = 192.168.1.2 >> port = 1433 >> tds version = 8.0 >> >> /usr/local/firstworks/etc/sqlrelay.conf >> >> >> <?xml version="1.0"?> >> <!DOCTYPE instances SYSTEM "sqlrelay.dtd"> >> <instances> >> <instance id="mssql" port="9000" socket="/tmp/example.socket" >> debug="listener" dbase="freetds" connections="3" maxconnections="15" >> maxqueuelength="5" \ >> growby="1" ttl="60" maxsessioncount="10" endofsession="commit" >> sessiontimeout="600" runasuser="nobody" runasgroup="nobody" \ >> cursors="5" authtier="listener" handoff="pass" deniedips="" >> allowedips="" maxquerysize="65536" maxstringbindvaluelength="4000" \ >> maxlobbindvaluelength="71680" idleclienttimeout="-1" >> maxlisteners="-1" listenertimeout="0" reloginatstart="no" >> timequeriessec="-1" timequeriesusec="-1"> >> <users> >> <user user="mssql1" password="mssql1"/> >> </users> >> <connections> >> <connection connectionid="db1" >> string="sybase=/etc/freetds.conf;user=AAA; >> password=BBB;server=sqlserver80;db=CCC;" metric="1"/> >> </connections> >> </instance> >> </instances> >> >> >> If I try to start sqlrelay this is what happens................. >> >> # sqlr-start -id mssql >> >> Starting listener: >> sqlr-listener -id mssql -config >> /usr/local/firstworks/etc/sqlrelay.conf -debug >> Debugging to: /usr/local/firstworks/var/sqlrelay/debug/sqlr-listener.11801 >> >> Starting 3 connections to db1 : >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlrelay.conf >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlrelay.conf >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlrelay.conf >> >> Starting scaler: >> sqlr-scaler -id mssql -config /usr/local/firstworks/etc/sqlrelay.conf >> >> sqlr-scaler error: >> The file >> /usr/local/firstworks/var/sqlrelay/tmp/pids/sqlr-listener-mssql was not >> found. >> This usually means that the sqlr-listener is not running. >> The sqlr-listener must be running for the sqlr-scaler to start. >> >> sqlr-scaler failed to start. >> >> >> # ps -aux | grep sql >> nobody 5280 0.0 0.4 3092 3108 ?? Ss 12:44PM 0:00.71 >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlre >> nobody 20552 0.0 0.4 3036 3108 ?? Ss 12:44PM 0:00.05 >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlre >> nobody 13263 0.0 0.4 2976 3064 ?? Ss 12:44PM 0:00.06 >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlre >> root 17795 0.0 0.0 152 232 p0 R+/2 12:47PM 0:00.00 grep sql >> >> >> # cat /usr/local/firstworks/var/sqlrelay/debug/sqlr-listener.11801 >> 06/08/2011 12:44:49 BST listener [11801] : creating shared memory and >> semaphores >> >> 06/08/2011 12:44:49 BST listener [11801] : id filename: >> >> 06/08/2011 12:44:49 BST listener [11801] : >> /usr/local/firstworks/var/sqlrelay/tmp/ipc/mssql >> >> 06/08/2011 12:44:49 BST listener [11801] : creating shared memory... >> >> 06/08/2011 12:44:49 BST listener [11801] : creating semaphores... >> >> I can see three connection to the MSSQL Server. but the sqlr-listener >> service is running. >> Strangly if I run the same command as sqlr-start did to start the >> listener but debug file is basicaly the same as it was but with a new IP >> >> >> ============== >> # >> # sqlr-listener -id mssql -config >> /usr/local/firstworks/etc/sqlrelay.conf -debug >> Debugging to: /usr/local/firstworks/var/sqlrelay/debug/sqlr-listener.15787 >> >> And the process starts ! >> nobody 32728 0.0 0.2 1224 1856 ?? Ss 12:58PM 0:00.02 >> sqlr-listener -id mssql -config /usr/local/firstworks/etc/sqlrelay.conf >> -debug >> >> Here is the icps output. >> >> # ipcs >> Message Queues: >> T ID KEY MODE OWNER GROUP >> >> Shared Memory: >> T ID KEY MODE OWNER GROUP >> m 458752 0 --rw-r----- nobody wheel >> m 65537 17248914 --rw-r----- nobody wheel >> >> Semaphores: >> T ID KEY MODE OWNER GROUP >> s 524288 17248914 --rw------- nobody wheel >> >> I don't know what to try next so any feedback will really be appreciated. >> >> Thanks >> Keith >> >> >> >> ------------------------------------------------------------------------------ >> EditLive Enterprise is the world's most technically advanced content >> authoring tool. Experience the power of Track Changes, Inline Image >> Editing and ensure content is compliant with Accessibility Checking. >> http://p.sf.net/sfu/ephox-dev2dev >> _______________________________________________ >> Sqlrelay-discussion mailing list >> Sql...@li... >> https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion >> >> >> _______________________________________________________ >> Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting >> http://www.doteasy.com > > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com > > ------------------------------------------------------------------------------ > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > Sqlrelay-discussion mailing list > Sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion |
From: Keith <ke...@sc...> - 2011-06-09 22:21:14
|
Yeah that's fixed it. I tried just addresses="" but that didn't work so put the servers IP in and I can see the service listening on the correct port so we have progress. Thank you Keith On 09/06/2011 16:31, David Muse wrote: > Try adding an > > addresses="" > > field to the instance tag in your configuration. There's currently a > bug where if the tag is missing altogether, then it thinks you meant not > to listen on an inet port at all, even though the port is specified. > It's fixed in CVS, but a new version with the fix hasn't been released yet. > > David Muse > dav...@fi... > > On 06/08/2011 08:05 AM, Keith wrote: >> Hi, I am trying to get sqlrelay to connect to a mssql server. I have had >> 3 process / connections going to the MSSQL server but don't see anything >> listening on port 9000 that I can connect to. Here's my configuration, >> If someone could have a look at I would be really grateful. >> >> OS is Openbsd 4.9 x64 >> sqlrelay-0.41 >> rudiments-0.32.tar >> freetds-0.82.1pre20100728p3 >> >> freetds.conf >> [global] >> ; tds version = 4.2 >> ; dump file = /tmp/freetds.log >> ; debug flags = 0xffff >> ; timeout = 10 >> ; connect timeout = 10 >> text size = 64512 >> [egServer70] >> host = 192.168.1.2 >> port = 1433 >> tds version = 7.0 >> [sqlserver80] >> host = 192.168.1.2 >> port = 1433 >> tds version = 8.0 >> >> /usr/local/firstworks/etc/sqlrelay.conf >> >> >> <?xml version="1.0"?> >> <!DOCTYPE instances SYSTEM "sqlrelay.dtd"> >> <instances> >> <instance id="mssql" port="9000" socket="/tmp/example.socket" >> debug="listener" dbase="freetds" connections="3" maxconnections="15" >> maxqueuelength="5" \ >> growby="1" ttl="60" maxsessioncount="10" endofsession="commit" >> sessiontimeout="600" runasuser="nobody" runasgroup="nobody" \ >> cursors="5" authtier="listener" handoff="pass" deniedips="" >> allowedips="" maxquerysize="65536" maxstringbindvaluelength="4000" \ >> maxlobbindvaluelength="71680" idleclienttimeout="-1" >> maxlisteners="-1" listenertimeout="0" reloginatstart="no" >> timequeriessec="-1" timequeriesusec="-1"> >> <users> >> <user user="mssql1" password="mssql1"/> >> </users> >> <connections> >> <connection connectionid="db1" >> string="sybase=/etc/freetds.conf;user=AAA; >> password=BBB;server=sqlserver80;db=CCC;" metric="1"/> >> </connections> >> </instance> >> </instances> >> >> >> If I try to start sqlrelay this is what happens................. >> >> # sqlr-start -id mssql >> >> Starting listener: >> sqlr-listener -id mssql -config >> /usr/local/firstworks/etc/sqlrelay.conf -debug >> Debugging to: /usr/local/firstworks/var/sqlrelay/debug/sqlr-listener.11801 >> >> Starting 3 connections to db1 : >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlrelay.conf >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlrelay.conf >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlrelay.conf >> >> Starting scaler: >> sqlr-scaler -id mssql -config /usr/local/firstworks/etc/sqlrelay.conf >> >> sqlr-scaler error: >> The file >> /usr/local/firstworks/var/sqlrelay/tmp/pids/sqlr-listener-mssql was not >> found. >> This usually means that the sqlr-listener is not running. >> The sqlr-listener must be running for the sqlr-scaler to start. >> >> sqlr-scaler failed to start. >> >> >> # ps -aux | grep sql >> nobody 5280 0.0 0.4 3092 3108 ?? Ss 12:44PM 0:00.71 >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlre >> nobody 20552 0.0 0.4 3036 3108 ?? Ss 12:44PM 0:00.05 >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlre >> nobody 13263 0.0 0.4 2976 3064 ?? Ss 12:44PM 0:00.06 >> sqlr-connection-freetds -id mssql -connectionid db1 -config >> /usr/local/firstworks/etc/sqlre >> root 17795 0.0 0.0 152 232 p0 R+/2 12:47PM 0:00.00 grep sql >> >> >> # cat /usr/local/firstworks/var/sqlrelay/debug/sqlr-listener.11801 >> 06/08/2011 12:44:49 BST listener [11801] : creating shared memory and >> semaphores >> >> 06/08/2011 12:44:49 BST listener [11801] : id filename: >> >> 06/08/2011 12:44:49 BST listener [11801] : >> /usr/local/firstworks/var/sqlrelay/tmp/ipc/mssql >> >> 06/08/2011 12:44:49 BST listener [11801] : creating shared memory... >> >> 06/08/2011 12:44:49 BST listener [11801] : creating semaphores... >> >> I can see three connection to the MSSQL Server. but the sqlr-listener >> service is running. >> Strangly if I run the same command as sqlr-start did to start the >> listener but debug file is basicaly the same as it was but with a new IP >> >> >> ============== >> # >> # sqlr-listener -id mssql -config >> /usr/local/firstworks/etc/sqlrelay.conf -debug >> Debugging to: /usr/local/firstworks/var/sqlrelay/debug/sqlr-listener.15787 >> >> And the process starts ! >> nobody 32728 0.0 0.2 1224 1856 ?? Ss 12:58PM 0:00.02 >> sqlr-listener -id mssql -config /usr/local/firstworks/etc/sqlrelay.conf >> -debug >> >> Here is the icps output. >> >> # ipcs >> Message Queues: >> T ID KEY MODE OWNER GROUP >> >> Shared Memory: >> T ID KEY MODE OWNER GROUP >> m 458752 0 --rw-r----- nobody wheel >> m 65537 17248914 --rw-r----- nobody wheel >> >> Semaphores: >> T ID KEY MODE OWNER GROUP >> s 524288 17248914 --rw------- nobody wheel >> >> I don't know what to try next so any feedback will really be appreciated. >> >> Thanks >> Keith >> >> >> >> ------------------------------------------------------------------------------ >> EditLive Enterprise is the world's most technically advanced content >> authoring tool. Experience the power of Track Changes, Inline Image >> Editing and ensure content is compliant with Accessibility Checking. >> http://p.sf.net/sfu/ephox-dev2dev >> _______________________________________________ >> Sqlrelay-discussion mailing list >> Sql...@li... >> https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion >> >> >> _______________________________________________________ >> Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting >> http://www.doteasy.com > > > _______________________________________________________ > Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting > http://www.doteasy.com > > ------------------------------------------------------------------------------ > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > Sqlrelay-discussion mailing list > Sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion |