Re: [Opensc-devel] PKCS#11 Test suite (PIV)

[Douglas E E. <dee...@gm...>]

On 5/16/2016 10:04 AM, Jakub Jelen wrote:
> Hello OpenSC devels,
>
> I didn't find any test suite or unit tests for OpenSC project. As I
> noticed, there is a lot of hand-testing work on pull requests for
> various cards and users. I believe everyone has some use cases to verify
> basic functionality of their cards.
> I understand that this fields is very divergent, there is a lot of card
> variants and it is almost impossible to build automatic test suite that
> would run in cloud with every build. But would it make sense to have
> something that devels (or users) can simply run and what would verify
> basic functionality and possible regressions?

In-cloud testing would require the cloud test machine to have physical cards.
(Unless you are suggesting some RDC access to cards.)

I would say the closest tool we have is: pkcs11-tool -t -l
It does some basic tests, but as you may have noted if you try and run it
with a PIV card, it has some problems, especially with the decryption, as it
says the user is not logged in when trying to use the Sign key. The key usage
says it should not be used for decryption. With other cards it may have different problems.

>
> I went to the directory src/tests/ and fixed the tests that are
> available now (see pull request [1], broken for 6 years), but they are
> far away from complete test suite.
>
> I also started with the idea from PKCS#11 API and put together basic
> test suite and inspector for OpenSC, which is currently in my OpenSC
> fork [2]. It is by no mean complete test suite of all the use cases, but
> I tried to catch most common cases, represent results in understandable
> form (currently tested with PIV cards) and add regression test for
> recent pull request [3].

Good choice of card :-)

Are you using the the NIST set of 16 demo/test cards?

>
> And there is the twist. What would you expect from PKCS#11/Smartcard
> testsuite? Would it make sense to have something like this upstream?
> What use cases would you expect from that to check?

Enhancing the functionality of the pkcs11-tool -t would be a good start.


>
>
> [1] https://github.com/OpenSC/OpenSC/pull/759
> [2] https://github.com/Jakuje/OpenSC/tree/jjelen-testsuite/src/tests
> [3] https://github.com/OpenSC/OpenSC/pull/743
>
> Regards,
>

-- 

  Douglas E. Engert  <DEE...@gm...>