My question is:
I try to manage Uart communication between stm32g4 on Nucleoboard and PC. I use dynamic events to prevent unnecessary coping of the message - bytes are directly stored into the event and if message is finished, event is posted. Hence memory pool is a shared resource and I need to have some security that I always have available memory for incoming message, I pre-created several events. As soon as is some event used and posted I want to create new event to always have my secure amount of memory ready to use for uart.
Now the case: Memory pool is full and I run out of all events. So, if is some memory freed I would like to let UART know that there is a free space so it can create event and continue with uart communication.
Right now I have a plan to use timer for it. So as soon as I post the event I start software timer witch is trying to create dynamic event every i.e. 1 ms and hopefully as soon as is some memory free, uart access it as a first active object.
Do you know about a better/more elegant solution how to let active object know that memory has been freed in the pool so it can create an event if it needs to?
Thank you in forward for any suggestions.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Polling an event pool at 1ms interval seems like overkill for what you're trying to achieve...
I would step back and perhaps rethink the design. The main question is why you are running out of events (?) The reasons might be:
you simply leak events somehow (please check!)
you produce too much data that cannot be sent over given the available bandwidth. If this is the case, no pool will be able to fix it, because it's unsustainable.
you produce data in bursts, but on average you produce less than the available bandwidth. In this case you should size your pool so that it can handle the worst-case burst.
All this assumes that you cannot tolerate lossy communication.
If you can tolerate dropped UART communication, you could simply use the non-asserting "extended" Q_NEW_X() macro, and send data when events are available and lose data when you cannot allocate new events (Q_NEW_X() returns NULL). This would be a workaround for case #2.
I'm not sure if my reply makes sense in your case. It would be good to know which of the cases 1..3 correspond to your situation (?)
--MMS
Last edit: Quantum Leaps 2021-12-03
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I would say the case 3 is my problem. The thing is I would like to do some "postprocessing" on the incoming messages as check if was transmitter correctly, evaluate a content of the message and so on... so I do not know how much time these operations take. So I could (most probably just theoretically) run out of memory. But I agree the system should be designet in such way it is not going to happen. But if it does, is there some feature QP acknowledge active object there was some event recycled so memory is available again?
I would like to avoid a dropp of communication.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
is there some feature QP acknowledge active object there was some event recycled so memory is available again?
No, such feature does not exist, because of rather fundamental reasons.
To quickly understand why, suppose that the QP framework would notify the application that some events are available in a pool. How long do you think such information would be valid? The problem is that it can change asynchronously (due to interrupt activity, for example), so by the time the application has a chance to react, the pool might NOT have enough events. I hope you see the problem...
I would like to avoid a drop of communication.
You are not alone here. I guess most developers would like that. But there is a price to pay: in the adequate sizing of your event pools and event queues and proper real-time analysis of your system.
so I do not know how much time these operations take. ..
That's exactly what I mean by "proper real-time analysis of your system". You need to know how much time things take. It's called "real-time programming" for a reason. Here I would like to recommend my video about RMA/RMS (Rate-Monotonic Analysis/Scheduling).
--MMS
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I assume that your messages follow a predifined protocol which have some kind of checksum to validate incoming messages. You also might have some message ids that you want ignore. I assume that these are the kind of post processing that you want to do.
I suggest that you do the post processing in your ISR while packing data in a dynamic created event. You should only handle one message at a time and not trying to allocate many events up front. Your messgaes might/should have a length paramater as part of the protocol which you can use the determine the size of the dynamic event to create. While collecting all the message data for the one message that you busy receiving, you can do you processing on the fly. If you do not want that message, you simply recycle the event right there (garbage collect) and repeat the process for the next message or you can reuse the already allocated event for the next incoming message.
This way you can minimise the number of events that you produce for the rest of your system to handle. Only create events for the messages that you really want. If you still have too messages during a burst, you will have thow away messages while your event pool is empty (Q_NEW_X()returns NULL) or you will have to increase your event pool size.
Hi Miro,
Thank you for explanation. I see the reason now.
Hello Gawie,
Yes, your assumption is correct. I agree that some operations could be done in ISR right away but my goal is to create some strict border in modules between those HW related and those witch knows the protocol + I minimise IRS time in this way. But otherwise is the implementation in such way you actually suggest, that I process the message while I am receiving another one.
My concern is more about solving the edge case "what if?"
So as much as I understand you and Miro the solution is to design the system to avoid this scenario
at all and if not, make the pool big enough/accept loose of some messages.
Thank you both for suggestions guys!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
My question is:
I try to manage Uart communication between stm32g4 on Nucleoboard and PC. I use dynamic events to prevent unnecessary coping of the message - bytes are directly stored into the event and if message is finished, event is posted. Hence memory pool is a shared resource and I need to have some security that I always have available memory for incoming message, I pre-created several events. As soon as is some event used and posted I want to create new event to always have my secure amount of memory ready to use for uart.
Now the case: Memory pool is full and I run out of all events. So, if is some memory freed I would like to let UART know that there is a free space so it can create event and continue with uart communication.
Right now I have a plan to use timer for it. So as soon as I post the event I start software timer witch is trying to create dynamic event every i.e. 1 ms and hopefully as soon as is some memory free, uart access it as a first active object.
Do you know about a better/more elegant solution how to let active object know that memory has been freed in the pool so it can create an event if it needs to?
Thank you in forward for any suggestions.
Polling an event pool at 1ms interval seems like overkill for what you're trying to achieve...
I would step back and perhaps rethink the design. The main question is why you are running out of events (?) The reasons might be:
All this assumes that you cannot tolerate lossy communication.
If you can tolerate dropped UART communication, you could simply use the non-asserting "extended"
Q_NEW_X()macro, and send data when events are available and lose data when you cannot allocate new events (Q_NEW_X()returns NULL). This would be a workaround for case #2.I'm not sure if my reply makes sense in your case. It would be good to know which of the cases 1..3 correspond to your situation (?)
--MMS
Last edit: Quantum Leaps 2021-12-03
Thank you very much for the fast response.
I would say the case 3 is my problem. The thing is I would like to do some "postprocessing" on the incoming messages as check if was transmitter correctly, evaluate a content of the message and so on... so I do not know how much time these operations take. So I could (most probably just theoretically) run out of memory. But I agree the system should be designet in such way it is not going to happen. But if it does, is there some feature QP acknowledge active object there was some event recycled so memory is available again?
I would like to avoid a dropp of communication.
Hi Jakub,
No, such feature does not exist, because of rather fundamental reasons.
To quickly understand why, suppose that the QP framework would notify the application that some events are available in a pool. How long do you think such information would be valid? The problem is that it can change asynchronously (due to interrupt activity, for example), so by the time the application has a chance to react, the pool might NOT have enough events. I hope you see the problem...
You are not alone here. I guess most developers would like that. But there is a price to pay: in the adequate sizing of your event pools and event queues and proper real-time analysis of your system.
That's exactly what I mean by "proper real-time analysis of your system". You need to know how much time things take. It's called "real-time programming" for a reason. Here I would like to recommend my video about RMA/RMS (Rate-Monotonic Analysis/Scheduling).
--MMS
Hi Jakub
I assume that your messages follow a predifined protocol which have some kind of checksum to validate incoming messages. You also might have some message ids that you want ignore. I assume that these are the kind of post processing that you want to do.
I suggest that you do the post processing in your ISR while packing data in a dynamic created event. You should only handle one message at a time and not trying to allocate many events up front. Your messgaes might/should have a length paramater as part of the protocol which you can use the determine the size of the dynamic event to create. While collecting all the message data for the one message that you busy receiving, you can do you processing on the fly. If you do not want that message, you simply recycle the event right there (garbage collect) and repeat the process for the next message or you can reuse the already allocated event for the next incoming message.
This way you can minimise the number of events that you produce for the rest of your system to handle. Only create events for the messages that you really want. If you still have too messages during a burst, you will have thow away messages while your event pool is empty (
Q_NEW_X()returns NULL) or you will have to increase your event pool size.Have a look at Safely Preallocating Events for an ISR for something similar.
Hope this helps.
Gawie
Hi Miro,
Thank you for explanation. I see the reason now.
Hello Gawie,
Yes, your assumption is correct. I agree that some operations could be done in ISR right away but my goal is to create some strict border in modules between those HW related and those witch knows the protocol + I minimise IRS time in this way. But otherwise is the implementation in such way you actually suggest, that I process the message while I am receiving another one.
My concern is more about solving the edge case "what if?"
So as much as I understand you and Miro the solution is to design the system to avoid this scenario
at all and if not, make the pool big enough/accept loose of some messages.
Thank you both for suggestions guys!