Re: [Jfilesync-news] memory usage

[Jens H. <hei...@in...>]

Hi Stefan:

Unfortunately, there is currently no solution to this problem. As Erich
already said, you may try to decrease the amount of files compared, or
increase the memory, Java is allowed to use, as you did.

The problem is, that the complete hierarchy of files on source and
target side is held in memory after comparison. It future, it would be
better to store this information somewhere on the hard disk and stream
to a file during comparison and read from the file when browsing the
compared files and when doing the synchronization. Moreover, it will
help avoiding referencing the Java File object and keeping it for all
files compared. However, all this will cause significant refactoring and
is no short-term task. I'll try to have a closer look at it beginning of
next year.

As to your second question, you may specify an exclude filter for your
directory before comparison, or alternatively, de-select the directory
pair after comparison from the comparison table (this will of course
have no effect in terms of memory usage). It is not possible to
de-select the file pair from the profile manager.


Cheers,

Jens.


Erich Musick wrote:
> I believe I've encountered a similar problem in the past, though I can't
> recall if I got the Out of Memory error or whether the program just locked
> up. I know with certainty that I've observed the latter.
> 
> I was syncing several directories that were each fairly large and contained
> a lot of files. My solution was to make a couple of different profiles and
> run each separately. It's kind of annoying, but it gets the job done.
> 
> Interestingly, it seemed to have more of a tendency to lock up on OSX than
> on my XP Home install. Both PCs have 1GB ram. I think the cause of this
> might have been that I was backing up my entire Library folder, which
> consists of far too many files.
> 
> Hope this information helps.
> 
> Erich Musick
> http://erichmusick.com
> 
> 
> -----Original Message-----
> From: jfi...@li...
> [mailto:jfi...@li...] On Behalf Of Stefan
> Hermanek
> Sent: Wednesday, December 12, 2007 11:52 AM
> To: jfi...@li...
> Subject: [Jfilesync-news] memory usage
> 
> i tried JFileSync last day and this is what i experienced:
> 
> windows2000, sp4, 1gb ram, java 1.5
> 
> source directory: 167.000 files
> target directory: already 164.000 files from an old backup of the source
> 
> i started JFileSync with java -Xmx256m -jar \lib\jfsync.jar
> 
> aprox. 4000 files were deleted in the target directory and 5000 new files
> copied to it.
> one second after the last copied file JFileSyn termineated with a
> OutOfMemoryError: java heap space.
> the task manager showed 370mb of memory for the java process.
> 
> then i tried to synchronize a directory of aprx. 36.000 files with an
> empty target directory: java used 370mb of memory. why is so much memory
> needed just for copying files?
> 
> then i did another synchronization with 10.000 files to an empty
> directory: used memory raised to 600mb (i don't know how java can do this
> with the -Xmx256m option?).
> after some repeated compare actions of the just synchronized directories
> java ended up with 656mb and another OutOfMemoryError.
> 
> i like JFileSync because of its clear user interface, but is there a way
> to get along with such a number of files and a reasonable memory usage?
> 
> btw, is it possible to exclude a directory pair from beeing
> compared/synchronized in a profile without deleting it?
> 
> thanx, stefan
> 
> 
> 
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> JFileSync-News mailing list
> JFi...@li...
> https://lists.sourceforge.net/lists/listinfo/jfilesync-news
> 
> 
> -------------------------------------------------------------------------
> SF.Net email is sponsored by:
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services
> for just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> JFileSync-News mailing list
> JFi...@li...
> https://lists.sourceforge.net/lists/listinfo/jfilesync-news