From: Sander A. <sa....@fz...> - 2021-08-19 06:32:22
Attachments:
smime.p7s
|
Good morning Krzysztof, all, we encountered a problem with configuration loss after restarts. We are using the configuration files everywhere where it is possible because we are using puppet as configuration management service. The configuration loss we encountered is e.g. - attached attribute classes - attribute statements If there is a large timeframe between changes and restart, they are kept. So it is difficult to reproduce this problem. Best regards, Sander -- Federated Systems and Data Juelich Supercomputing Centre phone: +49 2461 61 8847 fax: +49 2461 61 6656 email: sa....@fz... ----------------------------------------------------------------------- ----------------------------------------------------------------------- Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Volker Rieke Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht, Prof. Dr. Frauke Melchior ----------------------------------------------------------------------- ----------------------------------------------------------------------- |
From: Krzysztof B. <kb...@un...> - 2021-08-19 10:28:20
|
Hi Sander, W dniu 19.08.2021 o 08:32, Sander Apweiler pisze: > Good morning Krzysztof, all, > > we encountered a problem with configuration loss after restarts. We are > using the configuration files everywhere where it is possible because > we are using puppet as configuration management service. > > The configuration loss we encountered is e.g. > - attached attribute classes > - attribute statements > > If there is a large timeframe between changes and restart, they are > kept. So it is difficult to reproduce this problem. That sounds as a very serious problem, however doesn't ring any bell. Attribute classes and statements as attached to groups can be only stored to and loaded from DB. So I don't think that configuration files matter here. There is also no write-cache that could trigger such situation. I would be less confident in case of objects that are stored in DB but can be also reloaded from config files, but that's not the case here. I'd investigate whether perhaps you have some DB migration policy which looses some data written recently? Or maybe some of the data (e.g. groups) are re-initialized on each restart with either groovy script or via REST? In such case bugs in such automation may overwrite what is in DB. HTHm Krzysztof |
From: Sander A. <sa....@fz...> - 2021-08-19 10:41:09
Attachments:
smime.p7s
|
Hi Krzysztof, On Thu, 2021-08-19 at 12:28 +0200, Krzysztof Benedyczak wrote: > Hi Sander, > > W dniu 19.08.2021 o 08:32, Sander Apweiler pisze: > > Good morning Krzysztof, all, > > > > we encountered a problem with configuration loss after restarts. We > > are > > using the configuration files everywhere where it is possible > > because > > we are using puppet as configuration management service. > > > > The configuration loss we encountered is e.g. > > - attached attribute classes > > - attribute statements > > > > If there is a large timeframe between changes and restart, they are > > kept. So it is difficult to reproduce this problem. > > That sounds as a very serious problem, however doesn't ring any bell. > Attribute classes and statements as attached to groups can be only > stored to and loaded from DB. So I don't think that configuration > files > matter here. There is also no write-cache that could trigger such > situation. I would be less confident in case of objects that are > stored > in DB but can be also reloaded from config files, but that's not the > case here. > > I'd investigate whether perhaps you have some DB migration policy > which > looses some data written recently? I don't think so. We use a local mariadb instance only for unity with nightly db dumps as backup. > Or maybe some of the data (e.g. > groups) are re-initialized on each restart with either groovy script > or > via REST? In such case bugs in such automation may overwrite what is > in DB. We don't run any groovy scripts or API calls here. I will write down when I make online changes and check after reboots if they are still in place. When I can limit the time frame where it happens, I let you know. But at least the loss of attribute classes information happened more than once. Cheers, Sander > > HTHm > Krzysztof > > -- Federated Systems and Data Juelich Supercomputing Centre phone: +49 2461 61 8847 fax: +49 2461 61 6656 email: sa....@fz... ----------------------------------------------------------------------- ----------------------------------------------------------------------- Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Volker Rieke Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht, Prof. Dr. Frauke Melchior ----------------------------------------------------------------------- ----------------------------------------------------------------------- |
From: Krzysztof B. <kb...@un...> - 2021-08-20 12:49:27
|
W dniu 19.08.2021 o 12:40, Sander Apweiler pisze: > We don't run any groovy scripts or API calls here. > > I will write down when I make online changes and check after reboots if > they are still in place. When I can limit the time frame where it > happens, I let you know. > > But at least the loss of attribute classes information happened more > than once. Sounds reasonable - we need at least some rough hint on when this could be triggered. Cheers, Krzysztof |
From: Sander A. <sa....@fz...> - 2021-09-10 09:52:13
Attachments:
smime.p7s
|
Hi Krzysztof, I inspected one thing and can reproduce it. I thought it was caused by the restart but at least one information isn't. When I have added attribute classes to a subgroup and configure or even change the membership delegation thereafter, the attribute classes are dropped. Cheers, Sander On Fri, 2021-08-20 at 14:49 +0200, Krzysztof Benedyczak wrote: > W dniu 19.08.2021 o 12:40, Sander Apweiler pisze: > > We don't run any groovy scripts or API calls here. > > > > I will write down when I make online changes and check after > > reboots if > > they are still in place. When I can limit the time frame where it > > happens, I let you know. > > > > But at least the loss of attribute classes information happened > > more > > than once. > > Sounds reasonable - we need at least some rough hint on when this > could > be triggered. > > Cheers, > Krzysztof > -- Federated Systems and Data Juelich Supercomputing Centre phone: +49 2461 61 8847 fax: +49 2461 61 6656 email: sa....@fz... ----------------------------------------------------------------------- ----------------------------------------------------------------------- Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Volker Rieke Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht, Prof. Dr. Frauke Melchior ----------------------------------------------------------------------- ----------------------------------------------------------------------- |
From: Krzysztof B. <kb...@un...> - 2021-09-13 13:20:43
|
Hi Sander, W dniu 10.09.2021 o 11:52, Sander Apweiler pisze: > Hi Krzysztof, > I inspected one thing and can reproduce it. I thought it was caused by > the restart but at least one information isn't. When I have added > attribute classes to a subgroup and configure or even change the > membership delegation thereafter, the attribute classes are dropped. Got it. Actually this happens only if you do the above without any reset/reload of the groups tree in between the changes. Will be addressed in 3.6.0 which we are finalizing slowly. Thank you for precise bug report! Krzysztof > Cheers, > Sander > > On Fri, 2021-08-20 at 14:49 +0200, Krzysztof Benedyczak wrote: >> W dniu 19.08.2021 o 12:40, Sander Apweiler pisze: >>> We don't run any groovy scripts or API calls here. >>> >>> I will write down when I make online changes and check after >>> reboots if >>> they are still in place. When I can limit the time frame where it >>> happens, I let you know. >>> >>> But at least the loss of attribute classes information happened >>> more >>> than once. >> Sounds reasonable - we need at least some rough hint on when this >> could >> be triggered. >> >> Cheers, >> Krzysztof >> |