Well, as usual, the minute I post about it I figure it out... It seems you can specify the secret key you would like to use on the 2fa page and just use the same key on all the servers. It is not automagic, but easy enough. Now on to the next issue, why do the 2fa tokens not work. :/
I've done a fair bit of looking and it appears that there is not a way, at least in the webui, to sync the 2fa login information across multiple servers. Am I missing something? Would there be a manual way to do this?