[gphoto-features] [ gphoto-Feature Requests-1461148 ] [gphoto2] Downloading huge files should not require huge mem

[SourceForge.net <no...@so...>]

Feature Requests item #1461148, was opened at 2006-03-30 04:05
Message generated for change (Comment added) made by barrett9h
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=358874&aid=1461148&group_id=8874

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: CLI
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Aleksey Nogin (nogin)
Assigned to: Nobody/Anonymous (nobody)
Summary: [gphoto2] Downloading huge files should not require huge mem

Initial Comment:
If I try to download a huge file (such as a several
hundred meg .AVI - my Canon PowerShot A2 IS allows .AVI
of up to 1GB in size and I do take advantage of that),
then gallery2 (I am using 2.1.6 on Fedora Core 3) tries
to fit the whole thing in memory. As I usually do not
have enough physical memory, it starts trashing and the
whole process becomes unnecessarily slow :-(

It would be very nice if gphoto2 knew how to "stream"
the files from camera directly to disk without trying
to cache them in between. This would have a number of
advantages:

- Reducing the memory footprint (especially important
for large files)
- Allowing the camera I/O and disk I/O to happen in
parallel (further speeding things up, especially for
large files).

TIA for considering this.

P.S. Unfortunately, this issue makes gphoto2 nearly
unusable for me, forcing me to use a separate card
reader instead. :-(

----------------------------------------------------------------------

Comment By: rodolfo borges (barrett9h)
Date: 2006-08-27 12:18

Message:
Logged In: YES 
user_id=311242

Marcus, your suggestion does not work for a single big file.
I already made a script that lists the files `gphoto2 -L`,
then group files by size, not exceding half system memory,
and issue various `gphoto -p` for each range of files.

That helps a lot for when the camera is filled with various
photos, or small videos, but when you have a big video
there's no way around. Now I upgraded the memory card to
2GB, but having "only" 1GB on my computer, so I just can't
transfer a full video to the PC.

We really need the download_to_file() thing. There's no need
to allocate much memory, only a transfer buffer.

----------------------------------------------------------------------

Comment By: Hubert Figuiere (hfiguiere)
Date: 2006-05-08 05:34

Message:
Logged In: YES 
user_id=27487

The problem is that we would need to have an API call
"download_to_file()"

----------------------------------------------------------------------

Comment By: Marcus Meissner (marcusmeissner)
Date: 2006-03-30 09:27

Message:
Logged In: YES 
user_id=48092

libgphoto2 problem actually, it cached too much files. 
 
it now only caches the last 2 (can be reduced to 0), which 
should help. 
 
As for streaming from the camera ... well .... this is even 
more difficult in our current framework. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=358874&aid=1461148&group_id=8874