Re: [Soaplite-devel] huge memory consumption on transferring big files...

[Thorsten S. <tsc...@am...>]

Guten Tag Martin Kutter,
am Samstag, 17. Juli 2010 um 20:41 schrieben Sie:

> Eliminating all (really) unnecessary copies would probably reduce memory
> usage by a factor of between 2 and 10. Unfortunately, even this is
> unlikely to happen: SOAP::Lite's test suite is far from being perfect,
> and memory optimizations (and testing they don't break anything) would
> require quite some effort - and it's not even sure memory usage would
> drop below a level acceptable for you.

Hi Martin,

thanks for your answer. With some optimizations we at least can gain
enough time until everyone runs 64 Bit systems. ;-)

Today I reimplemented SOAP::Serializer::tag and ::xmlize in my own
serializer and could drop memory usage about some hundred MBs with
using array refs and string refs als elements instead of strings which
are often copied twice or more. One approach I thought of was using
our own data types which hold references to their data and recognizing
them everywhere where SOAP::Lite needs to handle this data. Especially
tag and xmlize are places where thos data type maybe recognized and
changed to the standard ones, but that's now problem because
SOAP::Lite gives one the possibility to override them very easy.

If I find some of those places in the deserializer and maybe the
transport layer, too, I'm optimistic to decrease memory usage to an
acceptable level without breaking to much.

> Unfortunately, SOAP::Lite doesn't have any means for streaming requests.
> To my knowledge, there's also no other SOAP library in perl which has
> direct support for streaming request, so you may need to roll your own.

I don't think this will happen, it's more likely to just use 64 Bit
systems and optimize serializer and deserializer to fit our needs.
It's not even clear if we stick with perl on the server side. We did a
project with Java and Axis2, this could be an option for new things,
too.

Mit freundlichen Grüßen,

Thorsten Schöning

-- 
Thorsten Schöning
AM-SoFT IT-Systeme - Hameln | Potsdam | Leipzig
 
Telefon: Potsdam: 0331-743881-0
E-Mail:  tsc...@am...
Web:     http://www.am-soft.de

AM-SoFT GmbH IT-Systeme, Konsumhof 1-5, 14482 Potsdam
Amtsgericht Potsdam HRB 21278 P, Geschäftsführer: Andreas Muchow