The testsuite fails on architectures where 32-bit accesses must be 32-bit aligned because test/srtp_driver.c hands unaligned data to srtp_protect() and srtp_unprotect(). This makes the testsuite fail on arm.
There are #ifdef ALIGNMENT_32BIT_REQUIRED macros for word access in crypto/include/integers.h, but these are never used anywhere. Instead the srtp_*protect() routines say they require 32bit-aligned data in the commentary.
As a quick fix for the Debian arm variants, I've made the minimal change (patch attached), simply aligning the test character arrays used in the test program, which solves the problem.
A more robust solution would be to lift the alignment requirement by using the GET_32 and PUT_32 macros and enabling ALIGNMENT_32BIT_REQUIRED in the header #ifdef __arm__ - that way other people's unaligned code using these functions that seems to work for them would not break mysteriously on arm.