On OS X (10.6.8 -- haven't yet tried it with 10.7 or 10.8), ./configure succeeds, but when the libmad library is being built, it fails with:
version.c:1: error: CPU you selected does not support x86-64 instruction set
The Stackoverflow question at http://stackoverflow.com/questions/9284698/ provides an explanation and a workaround. The accepted answer there doesn't seem to work, but the alternative suggestion (at http://stackoverflow.com/a/12864879/375147\) does, in the sense that the library does build. However it involves hacking a change to the ./configure script itself, so obviously isn't ideal.
I'm guessing you know about this already, but I don't see it in the issue tracker, so I'm posting here to make sure it doesn't get lost.
This is possibly related to bug 3398680