Re: [opendds-main] Possible memory leak in v 2.3-RT14855-1

[Leeland A. <la...@re...>]

Hi,



I have been reading and I am confused according to the OpenDDS-2.3
Developers Guide (pg 41):



The HISTORY policy determines how samples are held in the data writer and

data reader for a particular instance. For data writers these values are
held

until the publisher retrieves them and successfully sends them to all
connected

subscribers. For data readers these values are held until “taken” by the

application.

...

The “keep last” value (KEEP_LAST_HISTORY_QOS) specifies that only the

last depth values should be kept. When a data writer contains depth samples

of a given instance, a write of new samples for that instance are queued for

delivery and the oldest unsent samples are discarded. When a data reader

contains depth samples of a given instance, any incoming samples for that

instance are kept and the oldest samples are discarded.



But Adam’s message implied that by using KEEP_ALL_HISTORY_QOS we are
keeping all messages forever.



So is the “history” a history of samples that have been published and
picked up by at least one subscriber or does it control the list of samples
to be held UNTIL picked up by a subscriber?



Would you agree that to achieve a configuration that holds all samples
published until picked up by a subscriber the QOS settings should be:



  char* topic_name = "top";

  DCS_Type topic_type = DCS_Svc_Stack::DCS_TAG_SAMPLE;

  int round_robin = 0;

  bool use_fault_tolerance = false;



  // Initialize the transports (uses SimpleTcp transport)

  OpenDDS::DCPS::TransportImpl_rch pub_tcp_impl =

    TheTransportFactory->create_transport_impl(TCP_IMPL_PUB_ID,

                                               ACE_TEXT("SimpleTcp"),

                                               OpenDDS::DCPS::AUTO_CONFIG);

  // Create publisher

  pub_ = dp_->create_publisher(PUBLISHER_QOS_DEFAULT,

                               DDS::PublisherListener::_nil(),

                               OpenDDS::DCPS::DEFAULT_STATUS_MASK);

  // Create topic for datawriter

  DDS::TopicQos durable_qos;

  dp_->get_default_topic_qos(durable_qos);

  //Changing durability.kind from DDS::VOLATILE_DURABILITY_QOS to
DDS::RELIABLE_RELIABILITY_QOS

  durable_qos.reliability.kind = DDS::RELIABLE_RELIABILITY_QOS;

  durable_qos.reliability.max_blocking_time.sec = 10;

  durable_qos.reliability.max_blocking_time.nanosec = 0;

  DDS::Topic_var topic = dp_->create_topic(topic_name,

                                           type_id(topic_type).c_str(),

                                           durable_qos,

                                           DDS::TopicListener::_nil(),


OpenDDS::DCPS::DEFAULT_STATUS_MASK);

  DDS::DataWriterQos dw_qos;

  pub_->get_default_datawriter_qos(dw_qos);

  pub_->copy_from_topic_qos(dw_qos, durable_qos);

  dw_qos.history.kind = DDS::KEEP_LAST_HISTORY_QOS; //This is the default,
do not need any sample replay

  dw_qos.resource_limits.max_samples = DDS::LENGTH_UNLIMITED; //This is the
default, just calling it out

  dw_qos.resource_limits.max_instances = DDS::LENGTH_UNLIMITED; //This is
the default, just calling it out

  dw_qos.resource_limits.max_samples_per_instance = DDS::LENGTH_UNLIMITED;
//This is the default, just calling it out

  // Create the datawriter

  DDS::DataWriter_var dw =

    pub_->create_datawriter(topic,

                            //DATAWRITER_QOS_USE_TOPIC_QOS,

                            dw_qos,

                            use_fault_tolerance ? ft_manager_ : 0,

                            OpenDDS::DCPS::DEFAULT_STATUS_MASK);





Thanks in advance!

+ Leeland



*From:* Leeland Artra [mailto:la...@re...]
*Sent:* Tuesday, November 08, 2011 9:14 AM
*To:* Adam Mitz
*Cc:* ope...@li...
*Subject:* RE: [opendds-main] Possible memory leak in v 2.3-RT14855-1



Hello Adam,



LOL I didn’t notice your signature line. :^) I’d have to get some
authorization to spend cash.



Your comment on the history keeping might indeed be the issue. I found
another section of code that had this additional comment:



  //We don't want to lose any messages so setting the history to keep all
and set limits to unlimited

  dr_qos.history.kind = DDS::KEEP_ALL_HISTORY_QOS;

  dr_qos.resource_limits.max_samples = DDS::LENGTH_UNLIMITED; //This is the
default, just calling it out

  dr_qos.resource_limits.max_instances = DDS::LENGTH_UNLIMITED; //This is
the default, just calling it out

  dr_qos.resource_limits.max_samples_per_instance = DDS::LENGTH_UNLIMITED;
//This is the default, just calling it out



Which seems wrong to me. What we really want is for all messages to be
assured of being passed on to the subscribers. What would you recommend I
set these parameters to?



As far as the id_ I am still trying to figure out where that is set, how it
is tracked, etc.



+ Leeland





*From:* Adam Mitz [mailto:mi...@oc...]
*Sent:* Tuesday, November 08, 2011 9:03 AM
*To:* Leeland Artra
*Cc:* ope...@li...
*Subject:* Re: [opendds-main] Possible memory leak in v 2.3-RT14855-1



Hi Leeland,

On 11/8/2011 10:55 AM, Leeland Artra wrote:

Hello Adam,



I inherited this issue and am a total newbie for OpenDDS. I am reading up
on ACE and OpenDDS. So I might be wrong on my answers.



> Let's hear a bit more about how you're using it. In particular, are you
using any non-default QoS policies?

For QoS policies there is a setting for:



  DDS::TopicQos durable_qos;

  dp_->get_default_topic_qos(durable_qos);

  durable_qos.reliability.kind = DDS::RELIABLE_RELIABILITY_QOS;

  durable_qos.reliability.max_blocking_time.sec = 10;

  durable_qos.reliability.max_blocking_time.nanosec = 0;

// [... some error code ...]

  DDS::DataWriterQos dw_qos;

  pub_->get_default_datawriter_qos(dw_qos);

  pub_->copy_from_topic_qos(dw_qos, durable_qos);

  dw_qos.history.kind = DDS::KEEP_ALL_HISTORY_QOS;




OK, so OpenDDS is configured to "keep all" DataWriter history.  This is
probably accounting for the memory use.

> The comment in the IDL snippet indicates that "id_" is the key field, is
that right?



Not 100% sure but I think so too.




See the corresponding #pragma in the IDL.

> Are you registering and/or disposing any instances?  How many instances
are alive?

> (My guess is that OpenDDS is intentionally keeping that memory to manage
the various instances).

I am unsure how to figure this out. I think there is one instance created
per connection and I managed to reproduce this memory leak like behavior in
a lab environment I created where this is one publisher and one subscriber.




If there is only one instance, and id_ is the key, that would mean that
every sample sent has the same id_.  (Or instances are explicitly
disposed/unregistered.)

Maybe we can IM chat on this if you are willing.




We can do that as part of our commercial support services.
http://www.opendds.org/support.html

Thanks,
Adam Mitz
Senior Software Engineer
Object Computing, Inc.