Thread: [Quickfix-developers] file descriptor/memory leak in ThreadedSocketInitiator
Brought to you by:
orenmnero
Menu
▾
▴
quickfix-developers
|
From: Alex S. <al...@ya...> - 2007-01-08 14:49:18
|
Hi,
We left our application running over the holidays
continuously trying to reconnect to a server and
failing. When we came back, the whole server seemed to
be broken. All applications were failing because they
could not allocate any file descriptors from the OS
(Windows Server 2003)
Windows has a limit on the number of open file
descriptors including open sockets.
It turned out that quickfix fails to clean up sockets
when connections fail. With 7 connections failing
every 30 seconds, the limit was reached over a few
days.
The open sockets also use up memory.
The problem is in here:
bool ThreadedSocketInitiator::doConnect( const
SessionID& s, const Dictionary& d )
{ QF_STACK_PUSH(ThreadedSocketInitiator::doConnect)
try
{
Session* session = Session::lookupSession( s );
if( !session->isSessionTime() ) return false;
std::string address;
short port = 0;
getHost( s, d, address, port );
Log* log = session->getLog();
log->onEvent( "Connecting to " + address + " on
port " + IntConvertor::convert((unsigned short)port)
);
int socket = socket_createConnector();
if( socket_connect(socket, address.c_str(), port)
< 0 )
{
log->onEvent( "Connection failed" );
//!!!! Socket never closed because it is closed in
ThreadedSocketConnection::disconnect
return false;
}
log->onEvent( "Connection succeeded" );
ThreadedSocketConnection* pConnection =
new ThreadedSocketConnection( s, socket,
getApplication(), *this );
ThreadPair* pair = new ThreadPair( this,
pConnection );
{
Locker l( m_mutex );
unsigned thread;
if ( !thread_spawn( &socketThread, pair, thread
) )
delete pair;
addThread( socket, thread );
}
return true;
}
catch ( std::exception& ) { return false; }
QF_STACK_POP
}
Alex
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
|
|
From: Alex S. <al...@ya...> - 2007-01-05 18:41:39
|
Hi,
We left our application running over the holidays
continuously trying to reconnect to a server and
failing. When we came back, the whole server seemed to
be broken. All applications were failing because they
could not allocate any file descriptors from the OS
(Windows Server 2003)
Windows has a limit on the number of open file
descriptors including open sockets.
It turned out that quickfix fails to clean up sockets
when connections fail. With 7 connections failing
every 30 seconds, the limit was reached over a few
days.
The open sockets also use up memory.
The problem is in here:
bool ThreadedSocketInitiator::doConnect( const
SessionID& s, const Dictionary& d )
{ QF_STACK_PUSH(ThreadedSocketInitiator::doConnect)
try
{
Session* session = Session::lookupSession( s );
if( !session->isSessionTime() ) return false;
std::string address;
short port = 0;
getHost( s, d, address, port );
Log* log = session->getLog();
log->onEvent( "Connecting to " + address + " on
port " + IntConvertor::convert((unsigned short)port)
);
int socket = socket_createConnector();
if( socket_connect(socket, address.c_str(), port)
< 0 )
{
log->onEvent( "Connection failed" );
//!!!! Socket never closed because it is closed in
ThreadedSocketConnection::disconnect
return false;
}
log->onEvent( "Connection succeeded" );
ThreadedSocketConnection* pConnection =
new ThreadedSocketConnection( s, socket,
getApplication(), *this );
ThreadPair* pair = new ThreadPair( this,
pConnection );
{
Locker l( m_mutex );
unsigned thread;
if ( !thread_spawn( &socketThread, pair, thread
) )
delete pair;
addThread( socket, thread );
}
return true;
}
catch ( std::exception& ) { return false; }
QF_STACK_POP
}
Alex
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
|
|
From: Oren M. <or...@qu...> - 2007-01-08 15:00:51
|
Alex, what version of QuickFIX are you using? > Hi, > > We left our application running over the holidays > continuously trying to reconnect to a server and > failing. When we came back, the whole server seemed to > be broken. All applications were failing because they > could not allocate any file descriptors from the OS > (Windows Server 2003) |
|
From: Alex S. <al...@ya...> - 2007-01-08 15:03:35
|
Oren, I am using version 1.12.4 Alex --- Oren Miller <or...@qu...> wrote: > Alex, what version of QuickFIX are you using? > > > Hi, > > > > We left our application running over the holidays > > continuously trying to reconnect to a server and > > failing. When we came back, the whole server > seemed to > > be broken. All applications were failing because > they > > could not allocate any file descriptors from the > OS > > (Windows Server 2003) > > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com |
|
From: Alex S. <al...@ya...> - 2007-01-08 16:43:00
|
Oren,
I looked at the latest code in the repository and
revision 1725 and found that the problem still exists.
The fix you are referring to tries (I think) to
address socket cleanup inside of
ThreadedSocketConnection.
The problem occurs prior to the creation of the
ThreadedSocketConnection instance, when doConnect
returns without cleaning up the socket. I fixed it by
inserting "socket_close( socket );":
int socket = socket_createConnector();
if( socket_connect(socket, address.c_str(), port)
< 0 )
{
log->onEvent( "Connection failed" );
socket_close( socket );
return false;
}
log->onEvent( "Connection succeeded" );
ThreadedSocketConnection* pConnection =
new ThreadedSocketConnection( s, socket,
getApplication(), *this );
Alex
--- Oren Miller <or...@qu...> wrote:
> Alex, what version of QuickFIX are you using?
>
> > Hi,
> >
> > We left our application running over the holidays
> > continuously trying to reconnect to a server and
> > failing. When we came back, the whole server
> seemed to
> > be broken. All applications were failing because
> they
> > could not allocate any file descriptors from the
> OS
> > (Windows Server 2003)
>
>
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
|
|
From: Rich H. <rh...@st...> - 2007-01-08 15:09:32
|
We recently did an upgrade in our development tree and are seeing the problem with 1.20.4. I don't believe it was a problem in previous versions as we leave our production servers running all the time. Cheers, Rich Oren Miller wrote: > QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html > > Alex, what version of QuickFIX are you using? > > >> Hi, >> >> We left our application running over the holidays >> continuously trying to reconnect to a server and >> failing. When we came back, the whole server seemed to >> be broken. All applications were failing because they >> could not allocate any file descriptors from the OS >> (Windows Server 2003) >> > > > ------------------------------------------------------------------------- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Quickfix-developers mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-developers > This message is intended only for the personal and confidential use of the recipients named above. If the reader of this email is not the intended recipient, you have received this email in error and any review, dissemination, distribution or copying is strictly prohibited. If you have received this email in error, please notify the sender immediately by return email and permanently delete the copy you received. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. Wolverine is not responsible for any recommendation, solicitation, offer or agreement or any information about any transaction, customer account or account activity that may be attached to or contained in this communication. Wolverine accepts no liability for any content contained in the email, or any errors or omissions arising as a result of email transmission. Any opinions contained in this email constitute the sender's best judgment at this time and are subject to change without notice. |
|
From: Oren M. <or...@qu...> - 2007-01-08 16:09:56
|
It looks like this was probably fixed in the repository. I'm referring to this revision: r1725 | orenmnero | 2006-08-21 18:48:22 -0500 (Mon, 21 Aug 2006) | 1 line Initialize m_disconnect in constructors, fixes bug where file descriptors do not get released (Bill Robert) Can one of you verify this is the same problem and that the fix resolves it? --oren On Jan 8, 2007, at 9:08 AM, Rich Holm wrote: > QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/ > html/index.html > QuickFIX Support: http://www.quickfixengine.org/services.html > > > We recently did an upgrade in our development tree and are seeing the > problem with 1.20.4. I don't believe it was a problem in previous > versions > as we leave our production servers running all the time. > > Cheers, > Rich > > > > Oren Miller wrote: >> QuickFIX Documentation: http://www.quickfixengine.org/quickfix/doc/ >> html/index.html >> QuickFIX Support: http://www.quickfixengine.org/services.html >> >> Alex, what version of QuickFIX are you using? >> >> >>> Hi, >>> >>> We left our application running over the holidays >>> continuously trying to reconnect to a server and >>> failing. When we came back, the whole server seemed to >>> be broken. All applications were failing because they >>> could not allocate any file descriptors from the OS >>> (Windows Server 2003) >>> >> >> >> --------------------------------------------------------------------- >> ---- >> Take Surveys. Earn Cash. Influence the Future of IT >> Join SourceForge.net's Techsay panel and you'll get the chance to >> share your >> opinions on IT & business topics through brief surveys - and earn >> cash >> http://www.techsay.com/default.php? >> page=join.php&p=sourceforge&CID=DEVDEV >> _______________________________________________ >> Quickfix-developers mailing list >> Qui...@li... >> https://lists.sourceforge.net/lists/listinfo/quickfix-developers >> > > > > This message is intended only for the personal and confidential use > of the recipients named above. > If the reader of this email is not the intended recipient, you have > received this email in error and any review, > dissemination, distribution or copying is strictly prohibited. If > you have received this email in error, > please notify the sender immediately by return email and > permanently delete the copy you received. > > This message is provided for informational purposes and should not > be construed as a solicitation or offer > to buy or sell any securities or related financial instruments. > Wolverine is not responsible for any > recommendation, solicitation, offer or agreement or any information > about any transaction, customer account > or account activity that may be attached to or contained in this > communication. Wolverine accepts no > liability for any content contained in the email, or any errors or > omissions arising as a result of > email transmission. Any opinions contained in this email constitute > the sender's best judgment at this > time and are subject to change without notice. > > ---------------------------------------------------------------------- > --- > Take Surveys. Earn Cash. Influence the Future of IT > Join SourceForge.net's Techsay panel and you'll get the chance to > share your > opinions on IT & business topics through brief surveys - and earn cash > http://www.techsay.com/default.php? > page=join.php&p=sourceforge&CID=DEVDEV > _______________________________________________ > Quickfix-developers mailing list > Qui...@li... > https://lists.sourceforge.net/lists/listinfo/quickfix-developers > |
