Hi Nicola,
I need to better understand the desired use case.
task B starts, task A is launched, but it must wait for task B completion: supported.
task B starts, task B completes, task A is launched and starts immediately: supported.
task A is launched but it can not start because task B was not executed: it is a non supported use case, but I could implement a "don't create resource" flag as a new command line and config option. In this case, task A wouldn't able to create the locking resource and should wait until task B created, locked and unlocked the resource.
The tricky part is supporting the second and the third scenario: FLOM is not intended as a batch scheduler and the resources are created and destroyed quite fast. This is the race condition applying an hypotetical "don't create resource": task B completes and task A is not launched immediately, task A will not be able to create the resource and will wait forever. I don't see a silver bullet workaround to this issue: FLOM is currently not designed to manage long term persistence. A quick and dirty patch could be the usage of a file produced by B and tested by A before choosing the "don't create resource" option, but it needs extra scripting and FLOM should avoid this burden.
Thank you for your feedback, I'm waiting your point of view about the above topics.
Regards
Ch.F.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you for your reply. Yes, the point 3 is exactly my "different" use case.
I know I can support this situation adding a control on the existence of a temporary files but I don't like to much this approach (but this is the approach I am currently using).
What do you think about create two different flags? A "don't create resource" and a "don't destroy resource" flag? I didn't look in the implementation details of your code, so I have no idea on the complexity of this addition (in particular I don't know what a resource is in practice). With this two flags I think flom is able to cover all possible use cases. Unfortunatly this add complexity to ensure that resources must be released at certain point.
In my system I have to manage an intricated set of startup scripts. At this moment I am using inotify and directory creation events to synchronize between those scripts. I think flom is a much more elegant solution.
Thanks, best regards
N.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi Nicola,
a "don't destroy resource" behavior is quite risky because it opens a set of issues related to the question "who is in charge to clean-up those resources"? In my honest opinion a more practical implementation might be a "resource clean-up delay" that will be 0 milliseconds by default and a different value on user demand.
Implementing the two new options "don't create resource" and "resource clean-up delay" is surely a non trivial task because FLOM daemon business logic must be expanded with an incubator container where clients waiting for a not existent resource must wait and an inactive resource depot container where used but not in use resources stays until a new client ask for them or clean-up delay expires.
If you did think this stuff could help you, you kindly should create a ticket for this improvement (put a reference to this discussion in the ticket, I'm not asking you to repeat this discussion...).
I can't promise you a release date, but I could start working on it after the next release I'm currently forging.
Thank you for your precious suggestions.
Regards
Ch.F.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Release 0.7.0, just released, implements the use case you suggested some time ago.
The implemented new options are:
-e, --resource-create Specify if the command can create the resource to lock (accepted values 'yes', 'no')
-i, --resource-idle-lifespan Specify how long (milliseconds) a resource will be kept after usage termination
The corresponding configuration keys inside config files are:
[Resource]Create=IdleLifespan=
In the hope it could help you.
Regards
Ch.F.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I have a question about a different use case.
Can a task A waits the completion of another task B, which it is might be started or not?
Is flom able to cover this use case?
Thanks a lot for this very useful tools.
Hi Nicola,
I need to better understand the desired use case.
The tricky part is supporting the second and the third scenario: FLOM is not intended as a batch scheduler and the resources are created and destroyed quite fast. This is the race condition applying an hypotetical "don't create resource": task B completes and task A is not launched immediately, task A will not be able to create the resource and will wait forever. I don't see a silver bullet workaround to this issue: FLOM is currently not designed to manage long term persistence. A quick and dirty patch could be the usage of a file produced by B and tested by A before choosing the "don't create resource" option, but it needs extra scripting and FLOM should avoid this burden.
Thank you for your feedback, I'm waiting your point of view about the above topics.
Regards
Ch.F.
Thank you for your reply. Yes, the point 3 is exactly my "different" use case.
I know I can support this situation adding a control on the existence of a temporary files but I don't like to much this approach (but this is the approach I am currently using).
What do you think about create two different flags? A "don't create resource" and a "don't destroy resource" flag? I didn't look in the implementation details of your code, so I have no idea on the complexity of this addition (in particular I don't know what a resource is in practice). With this two flags I think flom is able to cover all possible use cases. Unfortunatly this add complexity to ensure that resources must be released at certain point.
In my system I have to manage an intricated set of startup scripts. At this moment I am using inotify and directory creation events to synchronize between those scripts. I think flom is a much more elegant solution.
Thanks, best regards
N.
Hi Nicola,
a "don't destroy resource" behavior is quite risky because it opens a set of issues related to the question "who is in charge to clean-up those resources"? In my honest opinion a more practical implementation might be a "resource clean-up delay" that will be 0 milliseconds by default and a different value on user demand.
Implementing the two new options "don't create resource" and "resource clean-up delay" is surely a non trivial task because FLOM daemon business logic must be expanded with an incubator container where clients waiting for a not existent resource must wait and an inactive resource depot container where used but not in use resources stays until a new client ask for them or clean-up delay expires.
If you did think this stuff could help you, you kindly should create a ticket for this improvement (put a reference to this discussion in the ticket, I'm not asking you to repeat this discussion...).
I can't promise you a release date, but I could start working on it after the next release I'm currently forging.
Thank you for your precious suggestions.
Regards
Ch.F.
Release 0.7.0, just released, implements the use case you suggested some time ago.
The implemented new options are:
The corresponding configuration keys inside config files are:
In the hope it could help you.
Regards
Ch.F.