Re: [Opensc-devel] mutual authentication fails when FIPS enabled

[Petr P. <pet...@at...>]

V Mon, Feb 01, 2021 at 10:15:43PM -0500, George napsal(a):
>    I would like to clarify my original problem. The mutual 
> authentication is between my application using OpenSSL and another 
> server. I am using LIBP11 to with OpenSSL to allow OpenSSL to access the 
> certificate and private key on my smart card. i.e. OpenSSL is using 
> LIBP11 to get the private key and certificate.
> 
> When FIPS is enabled, I see the OpenSSL error:
> 
>     *error:0409909D:rsa routines:RSA_public_encrypt:non fips rsa method*
> 
> 
> Since I am not making any calls directly to LIBP11, how can I fix this 
> in my code? i.e. OpenSSL is using LIBP11, not my application code.
> 
There can be many places which violate FIPS. Your (server or client) X.509
certificate can use a weak digest algorithm. The TLS cipher suite can use
a weak algorithm. Your PKCS11 driver can use a weak algorithm when loging into
a smart card. And probably other places. I recommend you to split your code
into smaller pieces: Establishing a TLS connection without the smart card.
Logging into the smart card without making a TLS connection. Making TLS
connection using keys on the card without libp11 library (libp11 is only an
abstraction layer. You can use OpenSSL engines directly.) And find out where
the problem exactly is.

> Is the problem related to the actual PIN? If so, how can I fix that?
> 
The problem is not a value of the PIN. It might be a way how the card driver
logs into the card with the PIN. If it is so, you would have to look into the
PKCS11 driver of your card.

-- Petr