Commandline Version is faster than Objekt
Swiss army knife of image processing
Brought to you by:
bfriesen
Menu
▾
▴
Hello!
I am happy that gm is much faster then im. I remember that the command line version is faster than the object modell when atiff graphic is greater 13244 x 9354 G4.
But if I use 20551 x 9933 G4 my speed also goes down. The harddisk is working. I think gm is swapping memory.
How could I manipulate big pictures very fast. If I try to resize to 50% is also need some time.
Is there a fast and simply function to reduce the pixel size? Maybe with others commandline tools?
I use windows xp.
Unfortunately, GraphicsMagick still uses the same simple "one size fits all" image model that ImageMagick uses. The first thing to check is to make sure that you are using a "Q8" version of GraphicsMagick. As mentioned in the FAQ, The amount of memory (in bytes) required to store a single image in memory may be calculated via the equation (QuantumDepth*Rows*Columns*5)/8. This means your huge image is taking (8*20551*9933*5)/8 = 1,020,665,415 bytes, or 973MB of RAM. Clearly GraphicsMagick is not very memory efficient at processing bilevel images.
There are other packages which are much more memory efficient than GraphicsMagick. One of these is VIPS
(http://www.vips.ecs.soton.ac.uk/index.php?title=VIPS). I don't think that VIPS is as easy to use since it caters more to the scientific/research community, but its "pull through" based design with multiple image models allows it to use much less memory than GraphicsMagick.
Hello Bob,
thank you very much for your detailed answer! Yes I use the Q8 version and when I redused the dpi of the tif image from 400 to 300, then gm works perfectly very fast!
My problem was that also Irfanview or other programs have problems with this resolution to convert it down. But I write a batch file to find all the 400 dpi images and then I regenerate this pictures (2.500 meters x 0.841 meters). At this time my computer has 2GB Ram. The problem is solved :-)
Thanks again Bob!
I have had a lookat vips, but I did not find a commandline exe for windows, only a dll.
It would be interestig to use this dll with vbs script's, but I did not find an example.
I never tried 20551 x 9933 in commandline but with GM C++ APIs, under Pentium 4 3.0, 8GB of ram, Linux 2.6.x, so far I have no problem dealing with 10000x10000 pixels bitmaps.
Works usually done in less than 10 seconds( lots of things happens inside the code: read, adding text, cut, zoom, quantize etc...).
Please keep in mind that I am using GM C++ APIs. Not commandlines like "gm convert" or something like that.
I think GM C APIs can do it well too.
If you can do programming, then switching from a batch of gm commands to GM C++ code will not so difficult.
Thanks for your answer!
Do you use a 64bit Linux, because of the 8GB ram?
Normally it should be no problem, but I did not need a c++ compiler during my work.
Only for the 1 project. But mabye I should really try. Do you know a good free compiler for windows?
It is an 32 bit IA based Redhat one.
To get the work done, I don't think you should mind the compiler or OS. Just get it done.
For Windows GM binary, you can download it here:
ftp://ftp.graphicsmagick.org/pub/GraphicsMagick/windows/
http://www.graphicsmagick.org/www/INSTALL-windows.html
http://www.graphicsmagick.org/www/Magick++/Install.html
I think it should work fine out of the box.
I don't use GM in Windows so I can not say anything about it.
If you want the taste of GM on Windows, with a Unix favor:
1. Install cygwin!
2. Install gcc-g++ ( this is the compiler you want )
3. Install GM from source.
4. Compile your GM C++ code with gcc-g++.
While I still think that the C++ interface is great (ok, I am biased), there is also the Perl scripting interface. GraphicsMagick DLL-based builds come prepared to use ActiveState Perl. Download and install ActiveState Perl (free) and then install GraphicsMagick. PerlMagick is a good programming alternative under Windows. If you will be performing multiple complex operations on images and can't quite get it done in one 'convert' or 'mogrify' command, then using the Perl, C++, Tcl, or Ruby interfaces will speed things quite a lot by eliminating redundant loads and stores.
Quite a lot of the GraphicsMagick test suite is written in Perl, and there are some Perl-based demos to use as examples.
Microsoft offers Visual Studio as a free download as an "Express" version. However, it is set up for compiling for .net only so if you want to build C and C++ code you have to follow some tedious instructions to obtain the required development headers and libraries ("SDK") and get them installed correctly. I successfully did this for Visual Studio Express 2005. The GUI and ImageMagick object rely on proprietary Microsoft libraries but gm.exe and Magick++ do build correctly.
Another alternative is MinGW (http://www.mingw.org/) which is a GCC-based tool-chain for building native Windows applications. This is free but it can be time-consuming to build the additional libraries needed.
I see that someone mentioned Cygwin. GraphicsMagick builds easily under Cygwin and Cygwin does offer most of the useful add-on libraries that GraphicsMagick needs. Run-time performance is a bit reduced on Cygwin as compared with MinGW or Visual Studio.
It is good to hear that you find performance to be good under Linux. Do you have GraphicsMagick built as a 64-bit app, or is it 32-bit? 64-bits with a lot of RAM makes problems just go away. :-)
Hi, it is 32 bit.
Can you explain why 64 bit with alot of RAM will be better?
Is there any perf. test for GM 64 bit?
There are a few blessings for 64-bit. On current Intel 64-bit CPUs, and definitely on AMD Opteron, the code often runs faster for 64-bit since it offers lots more registers. It is pretty typical to see a 30% speed improvement for compute-bound code just by compiling as 64-bit. This not the case for all CPU types. For example 64-bit SPARC and PowerPC code runs at about the same speed as 32-bit code (without the 64-bit "penalty" that some people like to claim).
If an image is very large, it may require more than the 2GB (or sometimes 3GB) that the 32-bit application model can support. In that case, memory allocations start failing and GraphicsMagick switches to putting the working image pixels on disk, which is a whole lot slower (100MB/second vs 10GB/second). Due to 32-bit address restrictions the pixel cache file can't be memory mapped. The slower read/write APIs need to be used instead.
A 64-bit application can access a ton of memory. It can access all of the RAM on the machine. It can memory map large files so that access is more efficient than read/write. The memory allocator does not need to work as hard in 64-bit applications as it does in 32-bit applications since address space is so much cheaper. The memory management unit (MMU) is responsible for assigning actual RAM to address space so the memory allocator only has to manage units of address space.
The output of 'gm convert -list resource' can be illuminating (try it on your machine!):
32-bit GraphicsMagick:
% gm convert -list resource
Resource Limits
----------------------------------------
Disk: Unlimited (MAGICK_LIMIT_DISK)
File: 128 (MAGICK_LIMIT_FILE)
Map: 1.9GB (MAGICK_LIMIT_MAP)
Memory: 1.9GB (MAGICK_LIMIT_MEMORY)
Pixels: 409.6MP (MAGICK_LIMIT_PIXELS)
64-bit GraphicsMagick:
% gm convert -list resource
Resource Limits
----------------------------------------
Disk: Unlimited (MAGICK_LIMIT_DISK)
File: 128 (MAGICK_LIMIT_FILE)
Map: 40.0GB (MAGICK_LIMIT_MAP)
Memory: 40.0GB (MAGICK_LIMIT_MEMORY)
Pixels: Unlimited (MAGICK_LIMIT_PIXELS)
Bob
Thanks for sharing such a good know-how.
I have no experience with 64bit so I won't say anything
regarding 64bit vs 32bit.
Hope one day I have a chance :)