unity-idm-discuss — Unity discussion and support

[Unity-idm-discuss] SAML NameID

[Sander A. <sa....@fz...>]

Good morning Krzysztof,
good morning Roman,

I have two short questions about SAML NameID and unity. In past weeks I
got two user tickets because their login with 3rd party IdP failed. In
both cases the log showed that the IdP did not use NameID format. Both
IdP admins said they didn't change it or didn't send it in past. Became
unity here more strikt between 3.3.4 and 3.5.1?

One IdP admin reported the NameIdPolicy in AuthRequest is empty, see
screenshot. Is this intended?

Cheers,
Sander
-- 
Federated Systems and Data
Juelich Supercomputing Centre

phone: +49 2461 61 8847
fax: +49 2461 61 6656
email: sa....@fz...

-----------------------------------------------------------------------
-----------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
-----------------------------------------------------------------------
-----------------------------------------------------------------------

[Unity-idm-discuss] reload of endpoints

[Sander A. <sa....@fz...>]

Good morning Krzysztof,

in the old admin endpoint it was possible to reload endpoints, if there
was a chance in configuration. In the console endpoint this is not
possible and we need to restart unity to load new configuration from
files. The endpoint reload was much faster than a restart of unity (2-3
minutes when you use eduGAIN). Would it be possible to include the
reload function in console endpoint too?

Cheers,
Sander
-- 
Federated Systems and Data
Juelich Supercomputing Centre

phone: +49 2461 61 8847
fax: +49 2461 61 6656
email: sa....@fz...

-----------------------------------------------------------------------
-----------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
-----------------------------------------------------------------------
-----------------------------------------------------------------------

Re: [Unity-idm-discuss] second redirect to upstream IdP

[Krzysztof B. <kb...@un...>]

Hi Sander,

W dniu 17.09.2021 o 13:50, Sander Apweiler pisze:
> Dear Krzysztof,
> In past we had the issue that for some IdPs the authentication failed
> and the user needed to select them a second time. This problem was
> solved and the second redirect is done automatically. User reported
> that this is sometimes very slow and they click a second time on the
> IdP button because they don't saw that the process is already running.
> Clicking a second time crashes the login flow. The users see an error
> about no OAuth context.
>
> Would it possible to highlight the running process?
>
> And does the attached log has something to do with the error? At least
> this is what I saw in the logs at the reported timestamp.

It is very likely that this problem will be completely addressed by the 
upcoming 3.6.0 release.

Cheers,
Krzysztof

[Unity-idm-discuss] second redirect to upstream IdP

[Sander A. <sa....@fz...>]

Dear Krzysztof,
In past we had the issue that for some IdPs the authentication failed
and the user needed to select them a second time. This problem was
solved and the second redirect is done automatically. User reported
that this is sometimes very slow and they click a second time on the
IdP button because they don't saw that the process is already running.
Clicking a second time crashes the login flow. The users see an error
about no OAuth context.

Would it possible to highlight the running process?

And does the attached log has something to do with the error? At least
this is what I saw in the logs at the reported timestamp.


2021-09-17T13:29:39,357 [qtp1186312760-8179] DEBUG unity.server.event.EventProcessor: Fire event: [trigger=methodInvocation.getEntityLabel, invokerEntity=102, timestamp=Fri Sep 17 13:29:39 CEST 2021, contents={"method":"getEntityLabel","interfaceName":"EntityManagement","exception":null,"args":[{"identity":null,"entityId":102}]}]
2021-09-17T13:29:39,367 [qtp1186312760-8179] DEBUG unity.server.event.EventProcessor: Fire event: [trigger=methodInvocation.getEntityLabel, invokerEntity=102, timestamp=Fri Sep 17 13:29:39 CEST 2021, contents={"method":"getEntityLabel","interfaceName":"EntityManagement","exception":null,"args":[{"identity":null,"entityId":102}]}]
2021-09-17T13:29:39,378 [qtp1186312760-8179] DEBUG unity.server.event.EventProcessor: Fire event: [trigger=methodInvocation.getEntityLabel, invokerEntity=102, timestamp=Fri Sep 17 13:29:39 CEST 2021, contents={"method":"getEntityLabel","interfaceName":"EntityManagement","exception":null,"args":[{"identity":null,"entityId":102}]}]
2021-09-17T13:29:39,385 [qtp1186312760-8179] DEBUG unity.server.event.EventProcessor: Fire event: [trigger=methodInvocation.getEntityLabel, invokerEntity=102, timestamp=Fri Sep 17 13:29:39 CEST 2021, contents={"method":"getEntityLabel","interfaceName":"EntityManagement","exception":null,"args":[{"identity":null,"entityId":102}]}]
2021-09-17T13:29:39,393 [qtp1186312760-8179] DEBUG unity.server.event.EventProcessor: Fire event: [trigger=methodInvocation.getEntityLabel, invokerEntity=102, timestamp=Fri Sep 17 13:29:39 CEST 2021, contents={"method":"getEntityLabel","interfaceName":"EntityManagement","exception":null,"args":[{"identity":null,"entityId":102}]}]
2021-09-17T13:29:39,401 [qtp1186312760-8179] DEBUG unity.server.event.EventProcessor: Fire event: [trigger=methodInvocation.getEntityLabel, invokerEntity=102, timestamp=Fri Sep 17 13:29:39 CEST 2021, contents={"method":"getEntityLabel","interfaceName":"EntityManagement","exception":null,"args":[{"identity":null,"entityId":102}]}]
2021-09-17T13:29:39,409 [qtp1186312760-8179] DEBUG unity.server.event.EventProcessor: Fire event: [trigger=methodInvocation.getEntityLabel, invokerEntity=102, timestamp=Fri Sep 17 13:29:39 CEST 2021, contents={"method":"getEntityLabel","interfaceName":"EntityManagement","exception":null,"args":[{"identity":null,"entityId":102}]}]
2021-09-17T13:29:39,417 [qtp1186312760-8179] DEBUG unity.server.event.EventProcessor: Fire event: [trigger=methodInvocation.getEntityLabel, invokerEntity=102, timestamp=Fri Sep 17 13:29:39 CEST 2021, contents={"method":"getEntityLabel","interfaceName":"EntityManagement","exception":null,"args":[{"identity":null,"entityId":102}]}]
2021-09-17T13:29:39,424 [qtp1186312760-8179] DEBUG unity.server.event.EventProcessor: Fire event: [trigger=methodInvocation.getEntityLabel, invokerEntity=102, timestamp=Fri Sep 17 13:29:39 CEST 2021, contents={"method":"getEntityLabel","interfaceName":"EntityManagement","exception":null,"args":[{"identity":null,"entityId":102}]}]
2021-09-17T13:29:39,433 [qtp1186312760-8179] DEBUG unity.server.event.EventProcessor: Fire event: [trigger=methodInvocation.getEntityLabel, invokerEntity=102, timestamp=Fri Sep 17 13:29:39 CEST 2021, contents={"method":"getEntityLabel","interfaceName":"EntityManagement","exception":null,"args":[{"identity":null,"entityId":102}]}]
2021-09-17T13:29:39,440 [qtp1186312760-8179] DEBUG unity.server.event.EventProcessor: Fire event: [trigger=methodInvocation.getEntityLabel, invokerEntity=102, timestamp=Fri Sep 17 13:29:39 CEST 2021, contents={"method":"getEntityLabel","interfaceName":"EntityManagement","exception":null,"args":[{"identity":null,"entityId":102}]}]
2021-09-17T13:29:39,449 [qtp1186312760-8179] DEBUG unity.server.event.EventProcessor: Fire event: [trigger=methodInvocation.getEntityLabel, invokerEntity=102, timestamp=Fri Sep 17 13:29:39 CEST 2021, contents={"method":"getEntityLabel","interfaceName":"EntityManagement","exception":null,"args":[{"identity":null,"entityId":102}]}]

Just a copy of a few lines, repeates much more in the logs with ongoing
timestamps.

Best regards,
Sander
-- 
Federated Systems and Data
Juelich Supercomputing Centre

phone: +49 2461 61 8847
fax: +49 2461 61 6656
email: sa....@fz...

-----------------------------------------------------------------------
-----------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
-----------------------------------------------------------------------
-----------------------------------------------------------------------

Re: [Unity-idm-discuss] Issue with login form autofill in chrome

[Tomasz G. <ymg...@cy...>]

Hi Krzysztof,

Thanks for the update! It's good to know it will be solved soon

Cheers
Tomek


śr., 15 wrz 2021 o 15:49 Krzysztof Benedyczak <kb...@un...> napisał(a):
>
> Hi Tomasz,
>
> W dniu 02.08.2021 o 13:05, Piotr Piernik pisze:
> >  Hi Tomasz
> > We investigating this problem. We are trying to recreate this
> > situation on varius browser and systems. We hope to fix this in the
> > next release.
> >
> A small update on that issue. That's really hard one as the logic/rules
> behind Chrome's password manager site introspection are pretty much
> confusing/undefined/...
>
>
> Anyway we finally managed to build a prototype that fixes this issue. It
> will take bit time to refactor that prototype into final version so it
> won't make it into the upcoming 3.6, but will be soon after either in a
> patch or in 3.7 latest.
>
>
> Cheers,
> Krzysztof
>
>
>

Re: [Unity-idm-discuss] Issue with login form autofill in chrome

[Krzysztof B. <kb...@un...>]

Hi Tomasz,

W dniu 02.08.2021 o 13:05, Piotr Piernik pisze:
>  Hi Tomasz
> We investigating this problem. We are trying to recreate this 
> situation on varius browser and systems. We hope to fix this in the 
> next release.
>
A small update on that issue. That's really hard one as the logic/rules 
behind Chrome's password manager site introspection are pretty much 
confusing/undefined/...


Anyway we finally managed to build a prototype that fixes this issue. It 
will take bit time to refactor that prototype into final version so it 
won't make it into the upcoming 3.6, but will be soon after either in a 
patch or in 3.7 latest.


Cheers,
Krzysztof

Re: [Unity-idm-discuss] configuration lost after restart

[Krzysztof B. <kb...@un...>]

Hi Sander,

W dniu 10.09.2021 o 11:52, Sander Apweiler pisze:
> Hi Krzysztof,
> I inspected one thing and can reproduce it. I thought it was caused by
> the restart but at least one information isn't. When I have added
> attribute classes to a subgroup and configure or even change the
> membership delegation thereafter, the attribute classes are dropped.

Got it. Actually this happens only if you do the above without any 
reset/reload of the groups tree in between the changes.

Will be addressed in 3.6.0 which we are finalizing slowly.

Thank you for precise bug report!
Krzysztof


> Cheers,
> Sander
>
> On Fri, 2021-08-20 at 14:49 +0200, Krzysztof Benedyczak wrote:
>> W dniu 19.08.2021 o 12:40, Sander Apweiler pisze:
>>> We don't run any groovy scripts or API calls here.
>>>
>>> I will write down when I make online changes and check after
>>> reboots if
>>> they are still in place. When I can limit the time frame where it
>>> happens, I let you know.
>>>
>>> But at least the loss of attribute classes information happened
>>> more
>>> than once.
>> Sounds reasonable - we need at least some rough hint on when this
>> could
>> be triggered.
>>
>> Cheers,
>> Krzysztof
>>

Re: [Unity-idm-discuss] configuration lost after restart

[Sander A. <sa....@fz...>]

Hi Krzysztof,
I inspected one thing and can reproduce it. I thought it was caused by
the restart but at least one information isn't. When I have added
attribute classes to a subgroup and configure or even change the
membership delegation thereafter, the attribute classes are dropped.

Cheers,
Sander

On Fri, 2021-08-20 at 14:49 +0200, Krzysztof Benedyczak wrote:
> W dniu 19.08.2021 o 12:40, Sander Apweiler pisze:
> > We don't run any groovy scripts or API calls here.
> > 
> > I will write down when I make online changes and check after
> > reboots if
> > they are still in place. When I can limit the time frame where it
> > happens, I let you know.
> > 
> > But at least the loss of attribute classes information happened
> > more
> > than once.
> 
> Sounds reasonable - we need at least some rough hint on when this
> could 
> be triggered.
> 
> Cheers,
> Krzysztof
> 

-- 
Federated Systems and Data
Juelich Supercomputing Centre

phone: +49 2461 61 8847
fax: +49 2461 61 6656
email: sa....@fz...

-----------------------------------------------------------------------
-----------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
-----------------------------------------------------------------------
-----------------------------------------------------------------------

Re: [Unity-idm-discuss] OpenID connect - Jupyter hub Invalid user name, credential or external authentication failed

[Krzysztof B. <kb...@un...>]

Dear Zoltan,

W dniu 30.08.2021 o 15:26, Zoltan Bakcsa pisze:
> Dear Krzysztof,
>
> First of all: now it works, many thanks for the help.
>
> > One of the screenshots you have shared shows that your OAuth clients 
> are
> > configured to authenticate with the *authenticator* called 'pwd'.
>
> Yes, but that 'pwd' authenticator is under
> Identity provider>Jupyter hub login>Users authentication 
> (https://snipboard.io/rUS3MP.jpg). Since it is under Users 
> authentication I assumed that authenticator is used (only) for 
> checking user's credentials and not the client credentials.

Yes, it is used there, however it was also present on this screenshot: 
https://snipboard.io/pTxEek.jpg

So you have reused the same authenticator for authenticating users as 
well as OAuth client. This, on its own, can be a valid setup.


> > Next check if your client (in Directory browser) has this particular
> > password credential set.
>
> I did not have a password configured there. Once I set it (with Update 
> credential button in the context menu) and adjusted the jupyter hub 
> config it started to work. I must have overlooked the relevant part in 
> the docs.
>
> However, it is still super confusing to me.
> Now I have 2 "passwords" for the client.
> The one that can be set in the directory browser here: 
> https://snipboard.io/BEAplM.jpg
>
> And another one that can be set under Identity Provider>Jupyter hub 
> login> Oauth client> [client ID]>Client secret : 
> https://snipboard.io/YnEado.jpg
>
> Of course, up to now I tried to use the client secret from this latter 
> option, which did not work.
> What is the purpose of the Client secret then?
>
Hmm, that should set up exactly the same credential - you can access 
that from two places. From directory you can set all credentials and 
from IdP -> client you should be only setting the one used for OAuth. 
I'll recheck that, maybe we have some regression there, but most likely 
there was some save click missing. Anyway we should improve the UI there 
to show whether the client secret is set or not.

Best,
Krzysztof


> Br,
> Zoltan
>
> On 8/30/2021 10:25 AM, Krzysztof Benedyczak wrote:
>> Dear Zoltan,
>>
>>
>> W dniu 25.08.2021 o 15:34, ba...@aw... pisze:
>>> Dear Krzysztof,
>>>
>>>> One more thing to check: please ensure that your authenticator used 
>>>> by OAuth token endpoint ('pwd') is linked to a *password 
>>>> credential* that is actually set for the client. It is a common 
>>>> pitfall (as >in Unity you can have multiple password credentials).
>>> Could you please describe how to do this step-by-step? I'm afraid I 
>>> do not speak the Unity language yet.
>>> Also, in my first email I linked screenshots of the whole 
>>> configuration. Can you check whether the authenticator is linked to 
>>> the correct credential?
>>> Perhaps you could point me to the relevant part in the documentation?
>>
>> One of the screenshots you have shared shows that your OAuth clients 
>> are configured to authenticate with the *authenticator* called 'pwd'.
>>
>> Now this authenticator is defining how to check the client's 
>> credential. In Authentication -> Facilities you will find the list of 
>> your authenticators. Locate entry 'pwd' there and check details. It 
>> should be an authenticator of type 'password' (i.e. checking 
>> passwords stored locally). And in its configuration there will be a 
>> password credential selected, which is used by this authenticator.  
>> Note it down.
>>
>> Next check if your client (in Directory browser) has this particular 
>> password credential set. Note that you can define multiple password 
>> credentials for your system (e.g. one for admins with high security 
>> requirements, one for ordinary users with lower requirements). Also 
>> unity defines one by its own (used to for the initial admin's 
>> password). So it is likely you have >1, and make sure the 
>> authenticator is using the correct one.
>>
>> HTH,
>> Krzysztof
>>

Re: [Unity-idm-discuss] OpenID connect - Jupyter hub Invalid user name, credential or external authentication failed

[Zoltan B. <ba...@aw...>]

Dear Krzysztof,

First of all: now it works, many thanks for the help.

 > One of the screenshots you have shared shows that your OAuth clients are
 > configured to authenticate with the *authenticator* called 'pwd'.

Yes, but that 'pwd' authenticator is under
Identity provider>Jupyter hub login>Users authentication 
(https://snipboard.io/rUS3MP.jpg). Since it is under Users 
authentication I assumed that authenticator is used (only) for checking 
user's credentials and not the client credentials.

 > Next check if your client (in Directory browser) has this particular
 > password credential set.

I did not have a password configured there. Once I set it (with Update 
credential button in the context menu) and adjusted the jupyter hub 
config it started to work. I must have overlooked the relevant part in 
the docs.

However, it is still super confusing to me.
Now I have 2 "passwords" for the client.
The one that can be set in the directory browser here: 
https://snipboard.io/BEAplM.jpg

And another one that can be set under Identity Provider>Jupyter hub 
login> Oauth client> [client ID]>Client secret : 
https://snipboard.io/YnEado.jpg

Of course, up to now I tried to use the client secret from this latter 
option, which did not work.
What is the purpose of the Client secret then?

Br,
Zoltan

On 8/30/2021 10:25 AM, Krzysztof Benedyczak wrote:
> Dear Zoltan,
> 
> 
> W dniu 25.08.2021 o 15:34, ba...@aw... pisze:
>> Dear Krzysztof,
>>
>>> One more thing to check: please ensure that your authenticator used 
>>> by OAuth token endpoint ('pwd') is linked to a *password credential* 
>>> that is actually set for the client. It is a common pitfall (as >in 
>>> Unity you can have multiple password credentials).
>> Could you please describe how to do this step-by-step? I'm afraid I do 
>> not speak the Unity language yet.
>> Also, in my first email I linked screenshots of the whole 
>> configuration. Can you check whether the authenticator is linked to 
>> the correct credential?
>> Perhaps you could point me to the relevant part in the documentation?
> 
> One of the screenshots you have shared shows that your OAuth clients are 
> configured to authenticate with the *authenticator* called 'pwd'.
> 
> Now this authenticator is defining how to check the client's credential. 
> In Authentication -> Facilities you will find the list of your 
> authenticators. Locate entry 'pwd' there and check details. It should be 
> an authenticator of type 'password' (i.e. checking passwords stored 
> locally). And in its configuration there will be a password credential 
> selected, which is used by this authenticator.  Note it down.
> 
> Next check if your client (in Directory browser) has this particular 
> password credential set. Note that you can define multiple password 
> credentials for your system (e.g. one for admins with high security 
> requirements, one for ordinary users with lower requirements). Also 
> unity defines one by its own (used to for the initial admin's password). 
> So it is likely you have >1, and make sure the authenticator is using 
> the correct one.
> 
> HTH,
> Krzysztof
> 

Re: [Unity-idm-discuss] user registration successful although mandatory attributes not provided

[Krzysztof B. <kb...@un...>]

Hi Sander,

W dniu 23.08.2021 o 09:30, Krzysztof Benedyczak pisze:
> W dniu 23.08.2021 o 07:29, Sander Apweiler pisze:
>> Good morning Krzysztof,
>> I'm not sure for all, but atleast for some I know what they did:
>> - Browse to /home endpoint
>> - Select external IdP
>> - Authenticate at external IdP
>> - Got register pop-up
>> - Register account
>>
>> I guess, but I have no hints for it, that some other users went to SAML
>> or OAuth endpoint instead of userhome endpoint.
>
> OK, so that's the unknown remote user flow. We will re-check that for 
> any regressions. 

I've retested that scenario manually and it seems to block such 
situation as before. I.e. user who has no remote attribute, and this 
attribute is required in the form as a remote provided, can't even see 
the form.

Maybe you have hit some edge case here? Can you please check the exact 
data that came from the remote IdP (or more precisely - to what it was 
mapped by the input profile) and then compare it against form config?

Cheers,
Krzysztof

Re: [Unity-idm-discuss] OpenID connect - Jupyter hub Invalid user name, credential or external authentication failed

[Krzysztof B. <kb...@un...>]

Dear Zoltan,


W dniu 25.08.2021 o 15:34, ba...@aw... pisze:
> Dear Krzysztof,
>
>> One more thing to check: please ensure that your authenticator used by OAuth token endpoint ('pwd') is linked to a *password credential* that is actually set for the client. It is a common pitfall (as >in Unity you can have multiple password credentials).
> Could you please describe how to do this step-by-step? I'm afraid I do not speak the Unity language yet.
> Also, in my first email I linked screenshots of the whole configuration. Can you check whether the authenticator is linked to the correct credential?
> Perhaps you could point me to the relevant part in the documentation?

One of the screenshots you have shared shows that your OAuth clients are 
configured to authenticate with the *authenticator* called 'pwd'.

Now this authenticator is defining how to check the client's credential. 
In Authentication -> Facilities you will find the list of your 
authenticators. Locate entry 'pwd' there and check details. It should be 
an authenticator of type 'password' (i.e. checking passwords stored 
locally). And in its configuration there will be a password credential 
selected, which is used by this authenticator.  Note it down.

Next check if your client (in Directory browser) has this particular  
password credential set. Note that you can define multiple password 
credentials for your system (e.g. one for admins with high security 
requirements, one for ordinary users with lower requirements). Also 
unity defines one by its own (used to for the initial admin's password). 
So it is likely you have >1, and make sure the authenticator is using 
the correct one.

HTH,
Krzysztof

Re: [Unity-idm-discuss] OpenID connect - Jupyter hub Invalid user name, credential or external authentication failed

[<ba...@aw...>]

Dear Krzysztof,

>One more thing to check: please ensure that your authenticator used by OAuth token endpoint ('pwd') is linked to a *password credential* that is actually set for the client. It is a common pitfall (as >in Unity you can have multiple password credentials).

Could you please describe how to do this step-by-step? I'm afraid I do not speak the Unity language yet.
Also, in my first email I linked screenshots of the whole configuration. Can you check whether the authenticator is linked to the correct credential? 
Perhaps you could point me to the relevant part in the documentation?


-----Original Message-----
From: Krzysztof Benedyczak <kb...@un...> 
Sent: Tuesday, August 17, 2021 2:36 PM
To: Roman Krysiński <ro...@un...>; ba...@aw...
Cc: Unity ML <uni...@li...>
Subject: *****SPAM***** Re: [Unity-idm-discuss] OpenID connect - Jupyter hub Invalid user name, credential or external authentication failed

Hi,

W dniu 17.08.2021 o 14:08, Roman Krysiński pisze:
> Hi Zoltan,
>
> > In the meantime, ideas about what could be possible misconfigured
> and/or working configuration examples (both Unity and Jupyter side) 
> are welcomed.
> Note that I was not using Jupyter for my tests, I just configured 
> unity according to your screenshots and used https://oauth.tools/ 
> <https://oauth.tools/> for testing, Please check whether clientId and 
> secret configured in jupyterhub_config.py are the same with those 
> generated by Unity, or regenerate client credentials in Unity and 
> update Jupyter config file.
>
> As an aside, I noticed that Jupyter under the hood is using Tornado as 
> a networking library, consider enabling the Tornado lib logging to see 
> more details in the Jupyter log:
> https://www.tornadoweb.org/en/stable/log.html
> <https://www.tornadoweb.org/en/stable/log.html>.
>

One more thing to check: please ensure that your authenticator used by OAuth token endpoint ('pwd') is linked to a *password credential* that is actually set for the client. It is a common pitfall (as in Unity you can have multiple password credentials).

You can also try to use command line tool as curl to make a request to the token endpoint in unity. Perhaps you won't be able to easily provide proper token, but at least you should be able to authenticate and get some OAuth-level error instead of an early authN error.  This would confirm that correct credential is configured on Unity side.

Best,
Krzysztof

[Unity-idm-discuss] 3.5.4 release, 3.5.3 skipped

[Krzysztof B. <kb...@un...>]

Dear Subscribers,

Two news. First of all the release number 3.5.3 was skipped, due to 
problems with our build process.

The next release after 3.5.2 is therefore v3.5.4. It was already made 
public, and contains two bug fixes in UpMan endpoint.

See https://www.unity-idm.eu/downloads/ for more details.

Best regards,
Krzysztof

Re: [Unity-idm-discuss] Emails to administrators in enquiry forms

[Sander A. <sa....@fz...>]

Hi Krzysztof,
thanks for the swift reply. I already did this for a quick workaround.
So now it is the solution ;)

Cheers,
Sander

On Mon, 2021-08-23 at 09:25 +0200, Krzysztof Benedyczak wrote:
> Hi Sander,
> 
> W dniu 23.08.2021 o 07:36, Sander Apweiler pisze:
> > Good morning Krzysztof,
> > end of last week we tested the emails to administrators about new
> > filled enquiries in the group membership update enquiry. We
> > encountered, that the email was send to all users from the group
> > and
> > not only to users with some administrative role. Of course there is
> > no
> > adminsitrator/admin role in unity but a set of roles with
> > administrativ
> > roles (System manager, Contents manager, Manager, projectsAdmin).
> > 
> > Is it intended that the email is send to all users in the entered
> > group
> > and not only to users with an administrativ role?
> > 
> Yes, it is by design. It may be the case that user w/o any special 
> rights should get emails (e.g. an unity entity representing some
> mailing 
> list address).
> 
> So the best is to have a dedicated group with people to be notified.
> 
> Cheers,
> Krzysztof
> 

-- 
Federated Systems and Data
Juelich Supercomputing Centre

phone: +49 2461 61 8847
fax: +49 2461 61 6656
email: sa....@fz...

-----------------------------------------------------------------------
-----------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
-----------------------------------------------------------------------
-----------------------------------------------------------------------

Re: [Unity-idm-discuss] user registration successful although mandatory attributes not provided

[Krzysztof B. <kb...@un...>]

W dniu 23.08.2021 o 07:29, Sander Apweiler pisze:
> Good morning Krzysztof,
> I'm not sure for all, but atleast for some I know what they did:
> - Browse to /home endpoint
> - Select external IdP
> - Authenticate at external IdP
> - Got register pop-up
> - Register account
>
> I guess, but I have no hints for it, that some other users went to SAML
> or OAuth endpoint instead of userhome endpoint.

OK, so that's the unknown remote user flow. We will re-check that for 
any regressions.

Thanks,
Krzysztof

Re: [Unity-idm-discuss] Emails to administrators in enquiry forms

[Krzysztof B. <kb...@un...>]

Hi Sander,

W dniu 23.08.2021 o 07:36, Sander Apweiler pisze:
> Good morning Krzysztof,
> end of last week we tested the emails to administrators about new
> filled enquiries in the group membership update enquiry. We
> encountered, that the email was send to all users from the group and
> not only to users with some administrative role. Of course there is no
> adminsitrator/admin role in unity but a set of roles with administrativ
> roles (System manager, Contents manager, Manager, projectsAdmin).
>
> Is it intended that the email is send to all users in the entered group
> and not only to users with an administrativ role?
>
Yes, it is by design. It may be the case that user w/o any special 
rights should get emails (e.g. an unity entity representing some mailing 
list address).

So the best is to have a dedicated group with people to be notified.

Cheers,
Krzysztof

[Unity-idm-discuss] Emails to administrators in enquiry forms

[Sander A. <sa....@fz...>]

Good morning Krzysztof,
end of last week we tested the emails to administrators about new
filled enquiries in the group membership update enquiry. We
encountered, that the email was send to all users from the group and
not only to users with some administrative role. Of course there is no
adminsitrator/admin role in unity but a set of roles with administrativ
roles (System manager, Contents manager, Manager, projectsAdmin).

Is it intended that the email is send to all users in the entered group
and not only to users with an administrativ role?

Best regards,
Sander
-- 
Federated Systems and Data
Juelich Supercomputing Centre

phone: +49 2461 61 8847
fax: +49 2461 61 6656
email: sa....@fz...

-----------------------------------------------------------------------
-----------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
-----------------------------------------------------------------------
-----------------------------------------------------------------------

Re: [Unity-idm-discuss] user registration successful although mandatory attributes not provided

[Sander A. <sa....@fz...>]

Good morning Krzysztof,
I'm not sure for all, but atleast for some I know what they did:
- Browse to /home endpoint
- Select external IdP
- Authenticate at external IdP
- Got register pop-up
- Register account

I guess, but I have no hints for it, that some other users went to SAML
or OAuth endpoint instead of userhome endpoint.

Cheers,
Sander

On Fri, 2021-08-20 at 14:50 +0200, Krzysztof Benedyczak wrote:
> Hi Sander,
> 
> W dniu 20.08.2021 o 14:07, Sander Apweiler pisze:
> > Hi Krzysztof,
> > 
> > sorry for bothering you again, but we encountered another problem.
> > In
> > registration forms we have some mandatory attributes, which must
> > provided by the remote IdP (config in screenshot). Is it intended,
> > that
> > the registration is succesful, although mandatory attributes are
> > missing? If I remember correctly in past this was not the case.
> > 
> AFAIR nothing has changed wrt that, so your assumption looks correct.
> We 
> verify that.
> 
> Can you please write in what flow this form is used? I.e. by
> invitation, 
> shown to unknown remote users, users enter it using well-known link,
> ...?
> 
> Thanks,
> Krzysztof
> 

-- 
Federated Systems and Data
Juelich Supercomputing Centre

phone: +49 2461 61 8847
fax: +49 2461 61 6656
email: sa....@fz...

-----------------------------------------------------------------------
-----------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
-----------------------------------------------------------------------
-----------------------------------------------------------------------

Re: [Unity-idm-discuss] user registration successful although mandatory attributes not provided

[Krzysztof B. <kb...@un...>]

Hi Sander,

W dniu 20.08.2021 o 14:07, Sander Apweiler pisze:
> Hi Krzysztof,
>
> sorry for bothering you again, but we encountered another problem. In
> registration forms we have some mandatory attributes, which must
> provided by the remote IdP (config in screenshot). Is it intended, that
> the registration is succesful, although mandatory attributes are
> missing? If I remember correctly in past this was not the case.
>
AFAIR nothing has changed wrt that, so your assumption looks correct. We 
verify that.

Can you please write in what flow this form is used? I.e. by invitation, 
shown to unknown remote users, users enter it using well-known link, ...?

Thanks,
Krzysztof

Re: [Unity-idm-discuss] configuration lost after restart

[Krzysztof B. <kb...@un...>]

W dniu 19.08.2021 o 12:40, Sander Apweiler pisze:
> We don't run any groovy scripts or API calls here.
>
> I will write down when I make online changes and check after reboots if
> they are still in place. When I can limit the time frame where it
> happens, I let you know.
>
> But at least the loss of attribute classes information happened more
> than once.

Sounds reasonable - we need at least some rough hint on when this could 
be triggered.

Cheers,
Krzysztof

[Unity-idm-discuss] user registration successful although mandatory attributes not provided

[Sander A. <sa....@fz...>]

Hi Krzysztof,

sorry for bothering you again, but we encountered another problem. In
registration forms we have some mandatory attributes, which must
provided by the remote IdP (config in screenshot). Is it intended, that
the registration is succesful, although mandatory attributes are
missing? If I remember correctly in past this was not the case.

Cheers,
Sander
-- 
Federated Systems and Data
Juelich Supercomputing Centre

phone: +49 2461 61 8847
fax: +49 2461 61 6656
email: sa....@fz...

-----------------------------------------------------------------------
-----------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
-----------------------------------------------------------------------
-----------------------------------------------------------------------

Re: [Unity-idm-discuss] configuration lost after restart

[Sander A. <sa....@fz...>]

Hi Krzysztof,

On Thu, 2021-08-19 at 12:28 +0200, Krzysztof Benedyczak wrote:
> Hi Sander,
> 
> W dniu 19.08.2021 o 08:32, Sander Apweiler pisze:
> > Good morning Krzysztof, all,
> > 
> > we encountered a problem with configuration loss after restarts. We
> > are
> > using the configuration files everywhere where it is possible
> > because
> > we are using puppet as configuration management service.
> > 
> > The configuration loss we encountered is e.g.
> > - attached attribute classes
> > - attribute statements
> > 
> > If there is a large timeframe between changes and restart, they are
> > kept. So it is difficult to reproduce this problem.
> 
> That sounds as a very serious problem, however doesn't ring any bell.
> Attribute classes and statements as attached to groups can be only 
> stored to and loaded from DB. So I don't think that configuration
> files 
> matter here. There is also no write-cache that could trigger such 
> situation. I would be less confident in case of objects that are
> stored 
> in DB but can be also reloaded from config files, but that's not the 
> case here.
> 
> I'd investigate whether perhaps you have some DB migration policy
> which 
> looses some data written recently? 
I don't think so. We use a local mariadb instance only for unity with
nightly db dumps as backup.
> Or maybe some of the data (e.g. 
> groups) are re-initialized on each restart with either groovy script
> or 
> via REST? In such case bugs in such automation may overwrite what is
> in DB.
We don't run any groovy scripts or API calls here.

I will write down when I make online changes and check after reboots if
they are still in place. When I can limit the time frame where it
happens, I let you know.

But at least the loss of attribute classes information happened more
than once. 

Cheers,
Sander
> 
> HTHm
> Krzysztof
> 
> 

-- 
Federated Systems and Data
Juelich Supercomputing Centre

phone: +49 2461 61 8847
fax: +49 2461 61 6656
email: sa....@fz...

-----------------------------------------------------------------------
-----------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
-----------------------------------------------------------------------
-----------------------------------------------------------------------

Re: [Unity-idm-discuss] configuration lost after restart

[Krzysztof B. <kb...@un...>]

Hi Sander,

W dniu 19.08.2021 o 08:32, Sander Apweiler pisze:
> Good morning Krzysztof, all,
>
> we encountered a problem with configuration loss after restarts. We are
> using the configuration files everywhere where it is possible because
> we are using puppet as configuration management service.
>
> The configuration loss we encountered is e.g.
> - attached attribute classes
> - attribute statements
>
> If there is a large timeframe between changes and restart, they are
> kept. So it is difficult to reproduce this problem.

That sounds as a very serious problem, however doesn't ring any bell. 
Attribute classes and statements as attached to groups can be only 
stored to and loaded from DB. So I don't think that configuration files 
matter here. There is also no write-cache that could trigger such 
situation. I would be less confident in case of objects that are stored 
in DB but can be also reloaded from config files, but that's not the 
case here.

I'd investigate whether perhaps you have some DB migration policy which 
looses some data written recently? Or maybe some of the data (e.g. 
groups) are re-initialized on each restart with either groovy script or 
via REST? In such case bugs in such automation may overwrite what is in DB.

HTHm
Krzysztof

[Unity-idm-discuss] configuration lost after restart

[Sander A. <sa....@fz...>]

Good morning Krzysztof, all,

we encountered a problem with configuration loss after restarts. We are
using the configuration files everywhere where it is possible because
we are using puppet as configuration management service.

The configuration loss we encountered is e.g.
- attached attribute classes
- attribute statements

If there is a large timeframe between changes and restart, they are
kept. So it is difficult to reproduce this problem.

Best regards,
Sander
-- 
Federated Systems and Data
Juelich Supercomputing Centre

phone: +49 2461 61 8847
fax: +49 2461 61 6656
email: sa....@fz...

-----------------------------------------------------------------------
-----------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Volker Rieke
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Dr. Astrid Lambrecht,
Prof. Dr. Frauke Melchior
-----------------------------------------------------------------------
-----------------------------------------------------------------------