It is important to keep them in place since in XWiki we're passing the cleaned HTML to an XSLT processor, then to Apache FOP, which further passes each embedded SVG images to Apache Batik, so it's important that the SVGs are fully self-contained, without any dependencies on the outer HTML, including styles.
Sample HTML+SVG file.
As far as I can tell this passes fine with trunk.