Re: [Simple-support] any performance tricks?
Brought to you by:
niallg
Menu
▾
▴
Re: [Simple-support] any performance tricks?
|
From: Kevin D. <ke...@tr...> - 2009-03-03 17:32:15
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <META http-equiv=Content-Type content=text/html;charset=ISO-8859-1> <META content="MSHTML 6.00.6000.16788" name=GENERATOR></HEAD> <BODY text=#000000 bgColor=#ffffff leftMargin=1 topMargin=1 rightMargin=1><FONT face=Arial size=2> <DIV>At the risk of starting a war here (not my intent!):</DIV> <DIV> </DIV> <DIV>This is a case where text based, xml files are always going to be many orders of magnitude less efficient than binary representations. Protocol Buffers were designed for very high speed (and cross language compatability). XML is really just not going to be fast (even formats like JSON which were developed to address some of XML's verboseness for wire-protocol use will be considerably slower than binary representations).</DIV> <DIV> </DIV> <DIV>The cost of generating and parsing the text in XML is just always going to be several orders of magnitude higher. If you took the protocol buffer output and converted it to XML, you'd probably find that the overall performance is equivalent to what you are seeing from Simple. Protobuffers may still be a bit faster because they don't use reflection, but I'm pretty sure that the bottleneck here is the text processing...</DIV> <DIV> </DIV> <DIV> </DIV> <DIV>XML is a *heck* of a lot easier to read, perform transformations on and troubleshoot than binary data, though. So the substantial inneficienies inherent to XML (and any text based data transfer format) are overcome by ease of development and troubleshooting.</DIV> <DIV> </DIV> <DIV> </DIV> <DIV>What Jim might be interesed in doing is seeing if there is any way to generate protocol buffer code from Java annotations. I kind of doubt this will be the case (if I recall correctly, the idea behind protocol buffers was to have a language neutral data transfer definition that then compiles to objects in different languages) - but Google is into annotations these days, so maybe they have something like that in the works.</DIV> <DIV> </DIV> <DIV>- K</DIV> <DIV> </DIV> <DIV> </DIV> <DIV> </DIV> <DIV></DIV> <DIV> </DIV></FONT> <DIV style="FONT-SIZE: x-small; FONT-FAMILY: Tahoma"> <DIV>----------------------- <B>Original Message</B> -----------------------</DIV> <DIV> </DIV> <DIV><B>From:</B> Jim Sermersheim <A href="mailto:ji...@fu..."><FONT color=#0000ff><ji...@fu...></FONT></A></DIV> <DIV><B>To:</B> <A href="mailto:Nia...@ub..."><FONT color=#0000ff>Nia...@ub...</FONT></A></DIV> <DIV><B>Cc:</B> <A href="mailto:sim...@li..."><FONT color=#0000ff>sim...@li...</FONT></A></DIV> <DIV><B>Date:</B> Tue, 03 Mar 2009 09:24:59 -0700</DIV> <DIV><B>Subject: <U>Re: [Simple-support] any performance tricks?</U></B></DIV> <DIV> </DIV></DIV>Thanks Niall,<BR><BR>No, by default protobuf serializes into a binary format (<A class=moz-txt-link-freetext href="http://code.google.com/apis/protocolbuffers/docs/encoding.html">http://code.google.com/apis/protocolbuffers/docs/encoding.html</A>). One could implement a formatter callback and cause the output to be XML, but for my needs binary is fine. I know that creates an unfair apples to oranges comparison in looking at Simple vs protobuf. Mostly I was just wondering if there were any known performance tricks I may have overlooked. FWIW, I'm transferring the resulting data over the wire and even if I serialized to XML, I'd end up zipping it to reduce the payload size.<BR><BR>Jim<BR><BR><A class=moz-txt-link-abbreviated href="mailto:Nia...@ub...">Nia...@ub...</A> wrote: <BLOCKQUOTE cite=mid:9BB...@NL... type="cite"><PRE wrap="">Hi, Does protobuf generate XML as a result of the serialization process? There is definitely room for performance improvements in Simple. Its been a while since I have done a release focused on such enhancements. Since the introduction of namespaces there has been a slight lag in performance. However, I have already begun a scanner framework based around Javassist which will hopefully provide a signifigant performance improvement over reflection. I think though it will be weeks rather than days before any signifigant improvements in this area are released. Niall -----Original Message----- From: Jim Sermersheim [<A class=moz-txt-link-freetext href="mailto:ji...@fu...">mailto:ji...@fu...</A>] Sent: 03 March 2009 15:53 To: Gallagher, Niall-N Cc: <A class=moz-txt-link-abbreviated href="mailto:sim...@li...">sim...@li...</A> Subject: Re: [Simple-support] any performance tricks? You're right, the first serialization takes hundreds of ms. I should have mentioned that my test serializes the object a few hundred times (just to make sure *everything* is primed -- cache, hotspot, etc.), then starts the timer and serializes it 10000 more times and calculates an average time per serialization. protobuf is a different beast in that you build a template and it generates your code for you. I prefer the more flexible annotation-based method of Simple, but might have to go with the speed of protobuf :( <A class=moz-txt-link-abbreviated href="mailto:Nia...@ub...">Nia...@ub...</A> wrote: </PRE> <BLOCKQUOTE type="cite"><PRE wrap="">Hi, The first time you serialize or deserialize an object will be much slower that subsequent times. The best thing to do is write the object </PRE></BLOCKQUOTE><PRE wrap=""><!----> </PRE> <BLOCKQUOTE type="cite"><PRE wrap="">once, then start your timer and write it again. You will find the second time is much faster than the first. There is also a planned performance improvement for serialization, where I intend to reduce the impact of namespaces, which have added about 20% more overhead than needed. With regard to protobuf I am not familiar with it. However you will find Simple as fast as JAXB or </PRE></BLOCKQUOTE><PRE wrap=""><!---->Xstream. </PRE> <BLOCKQUOTE type="cite"><PRE wrap="">Niall -----Original Message----- From: Jim Sermersheim [<A class=moz-txt-link-freetext href="mailto:ji...@fu...">mailto:ji...@fu...</A>] Sent: 02 March 2009 19:07 To: <A class=moz-txt-link-abbreviated href="mailto:sim...@li...">sim...@li...</A> Subject: [Simple-support] any performance tricks? Hi, I'm evaluating Simple along with some other tools that will serialize data for purposes of data transfer across a network. I much prefer the </PRE></BLOCKQUOTE><PRE wrap=""><!----> </PRE> <BLOCKQUOTE type="cite"><PRE wrap="">annotations-based way that Simple lets me prepare data for serialization to the other tool I'm evaluating (Google protobuf), but the performance is becoming a determining factor. For example, a sample data structure which takes 1.53ms to encode with Simple only takes 0.048ms to encode with protobuf. I'm just doing a very simple standard annotation markup with @Attribute and @Element (the outer structure has a dozen fields and a list of two inner structures, each having about a half-dozen fields). Then I just use persister.write to serialize. I'm guessing maybe the reflection is taking a lot of time? Not sure. But are there any known performance tips/tricks to know about with Simple? Thanks, Jim CONFIDENTIAL This document and attachments contain information from Fusion-io, Inc. which is confidential and/or legally privileged. The information is intended only for the use of the individual or entity named on this transmission. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking of any action in reliance on the contents of this emailed information is strictly prohibited, and that the documents should be returned to Fusion-io, </PRE></BLOCKQUOTE><PRE wrap=""><!---->Inc. immediately. </PRE> <BLOCKQUOTE type="cite"><PRE wrap="">In this regard, if you have received this email in error, please notify us by return email immediately. ---------------------------------------------------------------------- -- ------ Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD <A class=moz-txt-link-freetext href="http://p.sf.net/sfu/XcvMzF8H">http://p.sf.net/sfu/XcvMzF8H</A> _______________________________________________ Simple-support mailing list <A class=moz-txt-link-abbreviated href="mailto:Sim...@li...">Sim...@li...</A> <A class=moz-txt-link-freetext href="https://lists.sourceforge.net/lists/listinfo/simple-support">https://lists.sourceforge.net/lists/listinfo/simple-support</A> Visit our website at <A class=moz-txt-link-freetext href="http://www.ubs.com">http://www.ubs.com</A> This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify </PRE></BLOCKQUOTE><PRE wrap=""><!----> </PRE> <BLOCKQUOTE type="cite"><PRE wrap="">the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mails are not encrypted and cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the </PRE></BLOCKQUOTE><PRE wrap=""><!----> </PRE> <BLOCKQUOTE type="cite"><PRE wrap="">contents of this message which arise as a result of e-mail </PRE></BLOCKQUOTE><PRE wrap=""><!---->transmission. </PRE> <BLOCKQUOTE type="cite"><PRE wrap="">If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. UBS Limited is a company registered in England & Wales under company number 2035362, whose registered office is at 1 Finsbury Avenue, London, EC2M 2PP, United Kingdom. UBS AG (London Branch) is registered as a branch of a foreign company under number BR004507, whose registered office is at 1 Finsbury Avenue, London, EC2M 2PP, United Kingdom. UBS Clearing and Execution Services Limited is a company registered in </PRE></BLOCKQUOTE><PRE wrap=""><!----> </PRE> <BLOCKQUOTE type="cite"><PRE wrap="">England & Wales under company number 03123037, whose registered office </PRE></BLOCKQUOTE><PRE wrap=""><!----> </PRE> <BLOCKQUOTE type="cite"><PRE wrap="">is at 1 Finsbury Avenue, London, EC2M 2PP, United Kingdom. </PRE></BLOCKQUOTE><PRE wrap=""><!----> CONFIDENTIAL This document and attachments contain information from Fusion-io, Inc. which is confidential and/or legally privileged. The information is intended only for the use of the individual or entity named on this transmission. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking of any action in reliance on the contents of this emailed information is strictly prohibited, and that the documents should be returned to Fusion-io, Inc. immediately. In this regard, if you have received this email in error, please notify us by return email immediately. Visit our website at <A class=moz-txt-link-freetext href="http://www.ubs.com">http://www.ubs.com</A> This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mails are not encrypted and cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments. UBS Limited is a company registered in England & Wales under company number 2035362, whose registered office is at 1 Finsbury Avenue, London, EC2M 2PP, United Kingdom. UBS AG (London Branch) is registered as a branch of a foreign company under number BR004507, whose registered office is at 1 Finsbury Avenue, London, EC2M 2PP, United Kingdom. UBS Clearing and Execution Services Limited is a company registered in England & Wales under company number 03123037, whose registered office is at 1 Finsbury Avenue, London, EC2M 2PP, United Kingdom. </PRE></BLOCKQUOTE><BR><BR>CONFIDENTIAL<BR><BR>This document and attachments contain information from Fusion-io, Inc. which is confidential and/or legally privileged.<BR>The information is intended only for the use of the individual or entity named on this transmission.<BR>If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking of any<BR>action in reliance on the contents of this emailed information is strictly prohibited, and that the documents should be returned to Fusion-io, Inc. immediately.<BR>In this regard, if you have received this email in error, please notify us by return email immediately.<BR><BR> <STYLE type=text/css> P, UL, OL, DL, DIR, MENU, PRE { margin: 0 auto;}</STYLE> <FONT face=Tahoma size=2> <DIV>------------------------------------------------------------------------------<BR>Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA<BR>-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise<BR>-Strategies to boost innovation and cut costs with open source participation<BR>-Receive a $600 discount off the registration fee with the source code: SFAD<BR>http://p.sf.net/sfu/XcvMzF8H<BR><BR></DIV></FONT> <STYLE type=text/css> P, UL, OL, DL, DIR, MENU, PRE { margin: 0 auto;}</STYLE> <FONT face=Tahoma size=2> <DIV>_______________________________________________<BR>Simple-support mailing list<BR>Sim...@li...<BR>https://lists.sourceforge.net/lists/listinfo/simple-support<BR><BR><BR></DIV></FONT></BODY></HTML> |
