[Opensc-devel] pam_pkcs11 -- Pull requests and advice for another bug fix

[Logan G. <lo...@ie...>]

Hi all,

I've been starting to use OpenSC and pam_pkcs11 personally and
professionally and have discovered a couple issues in `pam_pkcs11`. It
looks like the mailing list is the proper avenue for raising some of these
issues so here I go.

1) I've put up a pull request for one bug that has been verified by another
user here, this seems to be primarily an issue with OpenSSL versions and
how they are handled:

https://github.com/OpenSC/pam_pkcs11/pull/45

2) I've found what appears to be a much trickier issue in the ECDSA
signature verification of cert_vfy.c (
https://github.com/OpenSC/pam_pkcs11/issues/44). I'll reproduce the salient
parts of the issue below:

When using signature verification in pam_pkcs11, I immediately run into the
> following verification issue on ECDSA certs/keys in verify_signature():
>
verify_signature() failed: EVP_VerifyFinal() failed:
error:25066067:DSO support routines:dlfcn_load
>
> The primary issues seems to be "Signed big-endian encoding of minimal
> length", from the stackexchange post How can I convert a DER ECDSA
> signature to ASN.1?
> <https://crypto.stackexchange.com/questions/1795/how-can-i-convert-a-der-ecdsa-signature-to-asn-1/1797>.
> The current upstream code doesn't seem to even yield a properly terminated
> ASN1 signatures on modern OpenSSL versions (1.1.0) and even if it did lacks
> handling of BIGNUMs that may be interpreted as signed integers if not
> properly padded (see above stack exchange).
>

I have attached a hacky demo patch file that along with the linked stack
exchange posts clarifies my issue and what appears to be the necessary
solution.

This leads me into my primary question, as someone who hasn't had too much
exposure to OpenSC and is certainly not an expert in the OpenSSL API, what
is the best way to contribute?

It seems like EVP_Verify is a much older API and EVP_DigestVerify is now
recommended, has better documentation, and has better support for ECDSA
signatures (https://wiki.openssl.org/index.php/EVP_Signing_and_Verifying).
However, this would be a much bigger change than trying to reassemble the
ASN1 encoding using BN_bin2bn and ECDSA_SIG_get0_*.

Best,
Logan