Re: [Quickfix-developers] Fixed spurious System.NullReferenceException in .NET Application.toApp()

[Oren M. <or...@qu...>]

Fantastic!

We have a bug report open on this already and have had people report  
non-repeatable exceptions like this that happen very infrequently.   
When Reuters was doing their testing they reported similar problems  
under heavy load (which is why they ended up choosing to use the C++  
API).  This was our biggest open issue.  Thanks!  We'll add this to CVS  
immediately.

--oren

On Oct 19, 2004, at 12:03 PM, Vladimir Arnost wrote:

> QuickFIX Documentation:  
> http://www.quickfixengine.org/quickfix/doc/html/index.html
> QuickFIX FAQ:  
> http://www.quickfixengine.org/wikifix/index.php?QuickFixFAQ
> QuickFIX Support: http://www.quickfixengine.org/services.html
>
> Oren,
>
> I have fixed a nasty bug in .NET Application class in QuickFIX 1.9.2.
> From time to time, especially under high system load, sending a FIX
> message throws an exception like this one:
>
> System.NullReferenceException: Object reference not set to an instance
> of an object.
>    at FIX.message_order.=(message_order* , message_order* )
>    at FIX.message_order.__ctor(message_order* , message_order* copy)
>    at
> std._Tree<std::_Tmap_traits<int,FIX::FieldBase,FIX::message_order,...>
>> .key_comp(_Tree<...>* , message_order* )
>    at FIX.FieldMap.=(FieldMap* , FieldMap* )
>    at FIX.Message.=(Message* , Message* )
>    at Application.toApp(Application* , Message* message, SessionID*
> sessionID)
>    at FIX.Session.sendToTarget(Message* , SessionID* )
>    at QuickFix.Session.sendToTarget(Message message, SessionID
> sessionID)
>
> (The exception was thrown at random code locations, but always "below"
> FIX.FieldMap.operator=)
>
> As I have found, the problem lies in mixing unmanaged code and
> garbage-collected data structures. Method toApp() creates managed
> wrapper class QuickFix::Message for unmanaged class FIX::Message, calls
> managed method toApp() and copies the (modified?) message back to
> FIX::Message. The problem is in the last operation, which will fail if
> the .NET garbage collector decides to move QuickFix::Message in memory
> while the unmanaged assignment operator is obliviously copying
> toMessage->unmanaged() back to message. This happens quite rarely (once
> in 10000 messages or so), but the message is lost and never sent to the
> client (sequence gap does not occur), which makes the problem very
> severe.
>
>   void toApp( FIX::Message& message, const FIX::SessionID& sessionID )
> [simplified]
>   {
>     QuickFix::Message * toMessage = create( message );
>     m_application->toApp( toMessage, new QuickFix::SessionID( sessionID
> ) );
>     message = toMessage->unmanaged();
>     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   }
>
> The solution is quite easy: (well, once you identify the problem... ;)
> Simply 'tag' all pointers to GC'ed classes with "__pin" keyword:
>
>     QuickFix::Message __pin * toMessage = create( message );
>
> The toMessage object stays locked in memory in the method body even if
> GC is activated.
> I have seen the problem only in toApp(), but I believe that toAdmin()  
> is
> affected in the same way. Since 'pinning' a pointer for a short time
> does not hurt performance, I decided to 'pin' all QuickFix::Message
> pointers in the Application class.
>
> It is quite possible that other C#-to-C++ wrappers suffer from the same
> problem, but I have not analyzed them.
>
> Important note: I was unable to reproduce the problem while executing a
> QuickFIX application in Visual Studio.NET's debugger. The application
> must be executed stand-alone from the command line, because VS.NET
> changes the way the garbage collector works. Both Debug and Release
> builds produce the same erroneous behaviour.
>
>
> Here is the diff:
>
> --- Application.h	2004-05-31 15:27:42.000000000 +0200
> +++ Application.h.new	2004-10-19 17:22:15.011923000 +0200
> @@ -71,7 +71,7 @@
>
>    void toAdmin( FIX::Message& message, const FIX::SessionID& sessionID
> )
>    {
> -    QuickFix::Message * toMessage = create( message );
> +    QuickFix::Message __pin * toMessage = create( message );
>      m_application->toAdmin( toMessage, new QuickFix::SessionID(
> sessionID ) );
>      message = toMessage->unmanaged();
>    }
> @@ -79,7 +79,7 @@
>    void toApp( FIX::Message& message, const FIX::SessionID& sessionID )
>    throw( FIX::DoNotSend& )
>    {
> -    QuickFix::Message * toMessage = create( message );
> +    QuickFix::Message __pin * toMessage = create( message );
>      try
>      {
>        m_application->toApp( toMessage, new QuickFix::SessionID(
> sessionID ) );
> @@ -95,7 +95,7 @@
>  	 FIX::IncorrectTagValue&,
>  	 FIX::RejectLogon& )
>    {
> -    QuickFix::Message * toMessage = create( message );
> +    QuickFix::Message __pin * toMessage = create( message );
>      try
>      {
>        m_application->fromAdmin
> @@ -122,7 +122,7 @@
>  	 FIX::IncorrectTagValue&,
>  	 FIX::UnsupportedMessageType& )
>    {
> -    QuickFix::Message * toMessage = create( message );
> +    QuickFix::Message __pin * toMessage = create( message );
>      try
>      {
>        m_application->fromApp
> @@ -153,7 +153,7 @@
>      FIX::MsgType msgType;
>      unmanaged.getHeader().getField( beginString );
>      unmanaged.getHeader().getField( msgType );
> -    QuickFix::Message* message
> +    QuickFix::Message __pin * message
>        = m_factory->create
>        ( beginString.getValue().c_str(), msgType.getValue().c_str() );
>      message->setUnmanaged( unmanaged );
>
>
> Please put this patch to the official source tree ASAP.
>
> Cheers,
>   Vlad
>
> -------------------------------------------------------------
> Ing. Vladimir Arnost, Developer, FFastFill Europe Ltd.
> Vaclavske namesti 55, Prague, Czech Republic
>
> _______________________________________________________________________ 
> _
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> sec...@ff...
>
> This email has been scanned for all viruses by the FFastFill Email
> Security System.
> _______________________________________________________________________ 
> _
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: IT Product Guide on  
> ITManagersJournal
> Use IT products in your business? Tell us what you think of them. Give  
> us
> Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out  
> more
> http://productguide.itmanagersjournal.com/guidepromo.tmpl
> _______________________________________________
> Quickfix-developers mailing list
> Qui...@li...
> https://lists.sourceforge.net/lists/listinfo/quickfix-developers
>