I'm getting ready to poke about in that a bit, but I think I have an idea of one of the problems.
I know a list using markup was a problem in GTK 2.x... might be for 3 also.
I recall us discussing this in relation to the customizable keyboard shortcuts in the preferences dialog. Would you mind having a look there too? The difference here though is that I don't believe the startup time was affected by the list with markup in the prefs dialog... only major performance hit is when it's being fully populated/updated.
As an aside, is it possible to make it run the font list "population routine" when the text tool is first invoked rather than when you first try to bring up the list? If we could eliminate that delay the first time you try to bring up the list of fonts, it would be a huge user-perceived win.