I do see that the c routine build_xsb_backtrace does generate an empty list when the flags say no backtrace is desired. This could be a bug, and where the [] that you see is coming from. I guess the question is whether we should fix it there to return [[]], or add a clause for print_backtrace([]) as you suggest. I notice that default_sys_error_handler_main in error_handler.P checks if backtrace is [] and doesn't call print_backtrace if so. So someone saw this before. My feeling would be to fix the build_xsb_backtrace to return [[]], which seems to have been the intention, and see if this fixes things.
Thoughts?