Re: [Opensc-devel] PKCS#11 Test suite (PIV)

[Nikos M. <n.m...@gm...>]

On Mon, May 16, 2016 at 5:40 PM, Andreas Schwier
<and...@ca...> wrote:
> Dear Jakub,
> having a central regression test set-up would be great thing.
> Internally we use a PKCS#11 test suite similar to [1] that validates the
> PKCS#11 interface in combination with our SmartCard-HSM. We also have a
> smoke-test that runs a couple of tests on the command line.
[...]
> I guess the biggest issue is, that you need a specific configuration for
> each card. If there was a common testing framework, I'd be happy to
> provide tests for our card.

The test suite Jakub provided is read-only in nature and can be shared
by several cards.
Would it make sense to split the testing to:
(initialization part) - (write params part) - (read part)

Where the write/initialization part can be unique per card and can
simply be a script calling other tools. The write part generates some
test parameters (certificates and keys) under some criteria we agree
on and is shared between families of cards (ECDSA/RSA). Finally, the
read only part verifies the correct operation with the parameters
written.

Each part can be a different tool, and can even be a pkcs11-tool
option, and can be combined under a per-card specific script (the
initialization). For example:
mycard.sh:
  initialize_that_card_with_pin 1234
  run_write_params_test_with_pin 1234
  run_read_params_test_with_pin 1234
  clear_card

For the PIV cards the test suite is written on it would be simply a
run_read_params_test_with_pin call.

Would that be some framework you'd like to provide tests under? For
your cards for example, the initialization part needs to be written,
and followed up by the read-only check.

regards,
Nikos