I'm currently developing an ISR that collects a number of measurement samples during multiple subsequent activations. The current design caches all the measurements locally, then when ready it creates an event, copies all the samples into the event, and publishes it.
However, the copying process can take about ~400µs of processor time. My question is, is there a safe way to check to see if a preallocated event has successfully been published, in order to avoid "orphaning" the event in the pool and thereby clogging it.
One thought is to place a "set flag" after the "Publish" call. That way if the flag is set, then I know that the event was successfully published. However, if the flag is not set, then I cannot be certain that "Publish" was called.
Is there an official/safe way to check and see if the event has been published or not? If it hasn't been published then I can reuse the event in another batch of sampling.
Cheers,
Timothy
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Once an event has been published, it is up to the QP framework to recycle it. Also note that the life cycle of an event can vary quite a lot - it can be re-posted or deferred. Therefore it is not desirable for the publisher to try to determine when an event has been been recycled.
Cheers,
Gawie
Last edit: Gawie de Vos 2015-05-19
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Good post, Gawie. I was also about to suggest to collect the data directly to an event object, possibly over many invocations of the ISR, instead of caching the data into a buffer and then copying it all into an event. This would "spread the load" more evenly and avoid the "big hiccup" of building the whole event in one swoop. As an added bonus, this would save the RAM, because the intermediate buffer would be unnecessary.
I am not quite sure that I understand the request for checking whether the publishing was successful. I can tell you right away: it was. Otherwise QF would assert.
Now, whether or not the event was actually posted to active objects, is another question, because it depends on who was subscribing to the event at this time. For example, if nobody happened to subscribe to this event, the framework will recycle the event right away.
But the good thing is that the framework takes care of the event from the point of publication and you don't need to worry about recycling the event. The framework is smart enough to "know" if the event is in use and it will recycle it as soon as possible, but not sooner. In that respect, the framework is better than anything you can invent.
--MMS
Last edit: Quantum Leaps 2015-05-19
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I had been thinking along similar lines, but wasn't sure if it would be a safe implementation or not.
As for "... is there a safe way to check to see if a preallocated event has successfully been published" - I wasn't questioning whether calling QF_PUBLISH would work correctly. I was questioning whether the ISR had successfully reached the calling/execution of QF_PUBLISH().
For example, my event is quite large relatively speaking, and so I only have a single "slot" allocated in this large sized event pool. If I were to preallocate an event, then fail to reach the state where the event is published, then if I were to try and preallocate another event the pool would be overdrawn and the code would fail.
I had considered setting the flag after the QF_PUBLISH(), so that if the flag was set then I would know for certain that the QF_PUBLISH code had been reached. However if the flag had not been set, then QF_PUBLISH could have been called, but the ISR could have been interrupted by a "higher" priority ISR. However after I thought about it, even if a higher priority ISR preempted the lower ISR the scenario where the processor does not return to the lower ISR to set the flag is virtually non-existent. And so I concluded that simply having a flag after the QF_PUBLISH would be safe and acceptable.
Thank you for the quick and helpful responses!
Cheers,
Timothy
Last edit: Timothy 2015-05-19
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The newer versions of QP/C and QP/C++ have now the non-asserting Q_NEW_X() macro, which returns the status of the allocation. Please see the documentation here:
The margin_ argument of Q_NEW_X() won't work, because this is the corner-case when the internal implementation asserts. But, I would recommend that you have a pool of 2 such big events. Then you can use a margin of 1, which will work. (You should be able to afford 2 such events, because you are getting rid of a private buffer, remember? The second event in the pool is taking over the role of your private buffer. But the gain now is that you avoid copying the contents.)
--MMS
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
As Miro noted, you're actually using the same amount of memory. Before you had 1 slot for a large event and (presumably) a similar sized intermediate buffer. Now you'll just have 2 slots for the large event and no intermediate buffer AND you get to skip the copying from the intermediate buffer to the event buffer.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Agreed. I realize there is a net change of zero between this and my original locally-cached method. However at the time of my reading the latest suggestion about Q_NEW_X(), I had already implemented the pre-allocated event method, and decreased the memory footprint. So by boosting the pool size back up, I thereby lost the memory savings that I had. But the savings are worth the safety, so it all worked out.
Cheers,
Timothy
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It would be preferrable to have a way to ask for an event with a margin of 0 and not have it assert. I too have some large events and having an extra event just sitting around not being used is not ideal.
I will likely just come up with another solution, possibly one that doesn't use QF memory pools, which I would rather not do.
So I would request that in a future version of the framework, we have a way to request a new object which will allow us to use all the objects in the pool without asserting.
Thanks,
Craig
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hey there,
I'm currently developing an ISR that collects a number of measurement samples during multiple subsequent activations. The current design caches all the measurements locally, then when ready it creates an event, copies all the samples into the event, and publishes it.
However, the copying process can take about ~400µs of processor time. My question is, is there a safe way to check to see if a preallocated event has successfully been published, in order to avoid "orphaning" the event in the pool and thereby clogging it.
One thought is to place a "set flag" after the "Publish" call. That way if the flag is set, then I know that the event was successfully published. However, if the flag is not set, then I cannot be certain that "Publish" was called.
Is there an official/safe way to check and see if the event has been published or not? If it hasn't been published then I can reuse the event in another batch of sampling.
Cheers,
Timothy
Hi Timothy
I would like suggest the following pseudo code that worked for me in the past.
void MY_ISR(void)
{
QK_ISR_ENTRY();
static MyEvt *pMyEvt = NULL; if(pMyEvt == NULL) { pMyEvt = Q_NEW(MyEvt, MY_SAMPLE_SIG); } if(pMyEvt != NULL) { /* Add sample to event data. */ if(time_to_publish_event) { QF_publish(pMyEvt); pMyEvt = NULL; } } QK_ISR_EXIT();}
Once an event has been published, it is up to the QP framework to recycle it. Also note that the life cycle of an event can vary quite a lot - it can be re-posted or deferred. Therefore it is not desirable for the publisher to try to determine when an event has been been recycled.
Cheers,
Gawie
Last edit: Gawie de Vos 2015-05-19
Good post, Gawie. I was also about to suggest to collect the data directly to an event object, possibly over many invocations of the ISR, instead of caching the data into a buffer and then copying it all into an event. This would "spread the load" more evenly and avoid the "big hiccup" of building the whole event in one swoop. As an added bonus, this would save the RAM, because the intermediate buffer would be unnecessary.
I am not quite sure that I understand the request for checking whether the publishing was successful. I can tell you right away: it was. Otherwise QF would assert.
Now, whether or not the event was actually posted to active objects, is another question, because it depends on who was subscribing to the event at this time. For example, if nobody happened to subscribe to this event, the framework will recycle the event right away.
But the good thing is that the framework takes care of the event from the point of publication and you don't need to worry about recycling the event. The framework is smart enough to "know" if the event is in use and it will recycle it as soon as possible, but not sooner. In that respect, the framework is better than anything you can invent.
--MMS
Last edit: Quantum Leaps 2015-05-19
Gawie & Miro,
I had been thinking along similar lines, but wasn't sure if it would be a safe implementation or not.
As for "... is there a safe way to check to see if a preallocated event has successfully been published" - I wasn't questioning whether calling QF_PUBLISH would work correctly. I was questioning whether the ISR had successfully reached the calling/execution of QF_PUBLISH().
For example, my event is quite large relatively speaking, and so I only have a single "slot" allocated in this large sized event pool. If I were to preallocate an event, then fail to reach the state where the event is published, then if I were to try and preallocate another event the pool would be overdrawn and the code would fail.
I had considered setting the flag after the QF_PUBLISH(), so that if the flag was set then I would know for certain that the QF_PUBLISH code had been reached. However if the flag had not been set, then QF_PUBLISH could have been called, but the ISR could have been interrupted by a "higher" priority ISR. However after I thought about it, even if a higher priority ISR preempted the lower ISR the scenario where the processor does not return to the lower ISR to set the flag is virtually non-existent. And so I concluded that simply having a flag after the QF_PUBLISH would be safe and acceptable.
Thank you for the quick and helpful responses!
Cheers,
Timothy
Last edit: Timothy 2015-05-19
The newer versions of QP/C and QP/C++ have now the non-asserting Q_NEW_X() macro, which returns the status of the allocation. Please see the documentation here:
http://www.state-machine.com/qpc/qf_8h.html#aa52b4d3c43a262022392c6e11be06436
--MMS
If I have a pool with a single slot, can I set the "margin_" to 0?
Cheers,
Timothy
The
margin_argument ofQ_NEW_X()won't work, because this is the corner-case when the internal implementation asserts. But, I would recommend that you have a pool of 2 such big events. Then you can use a margin of 1, which will work. (You should be able to afford 2 such events, because you are getting rid of a private buffer, remember? The second event in the pool is taking over the role of your private buffer. But the gain now is that you avoid copying the contents.)--MMS
I was trying to minimize memory usage, however the benefit of utilizing this method, I think, outweighs the memory savings. Thanks!
Cheers,
Timothy
As Miro noted, you're actually using the same amount of memory. Before you had 1 slot for a large event and (presumably) a similar sized intermediate buffer. Now you'll just have 2 slots for the large event and no intermediate buffer AND you get to skip the copying from the intermediate buffer to the event buffer.
Agreed. I realize there is a net change of zero between this and my original locally-cached method. However at the time of my reading the latest suggestion about Q_NEW_X(), I had already implemented the pre-allocated event method, and decreased the memory footprint. So by boosting the pool size back up, I thereby lost the memory savings that I had. But the savings are worth the safety, so it all worked out.
Cheers,
Timothy
It would be preferrable to have a way to ask for an event with a margin of 0 and not have it assert. I too have some large events and having an extra event just sitting around not being used is not ideal.
I will likely just come up with another solution, possibly one that doesn't use QF memory pools, which I would rather not do.
So I would request that in a future version of the framework, we have a way to request a new object which will allow us to use all the objects in the pool without asserting.
Thanks,
Craig
This sounds like a reasonable request. We'll look into this for the future QP/C/C++ releases. For now I just captured this as a feature request #126 Allow non-asserting event allocation for zero-margin allocations.
--MMS
The feature request #126 has been implemented in QP/C/C++ 5.9.3.
--MMS
Thanks!