Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

Force GM not to generate temp-files

Help
2014-04-14
2014-04-14
  • Hi,
    is there a way that I can force GM not to generate any temporary files at all?
    My goal is to pipe image files to and from GM (via stdio) as fast as possible without putting load onto the hard drive.
    Thanks in advance

     
    • On Mon, 14 Apr 2014, KnownIdentifier wrote:

      Hi,
      is there a way that I can force GM not to generate any temporary files at all?
      My goal is to pipe image files to and from GM (via stdio) as fast as possible without putting load onto the
      hard drive.

      Temporary files are used for several purposes. For huge images where
      the decoded representation can't fit entirely in memory, disk files
      are used to support the "pixel cache". When external programs are
      used (e.g. Ghostscript) these programs may require use of temporary
      files to pass input data and obtain output data. When input is piped
      from stdio then if the coder (reader) for the input format is declared
      to need to be able to seek in the input stream (MagickInfo
      seekable_stream is set to True), then it is necessary to buffer the
      input from stdio to a temporary file since pipes do not support
      seek().

      For some formats (e.g. TIFF), using seek is unavoidable. For some
      other formats, seek was used due to the programmer being lazy or due
      to some notion of efficiency (skip over rather than read).

      These formats are currently declared as requiring seek:

      BMP, CALS, CUT, DCM, FITS, FPX, ICON, JP2, MAC, MAT, META, MIFF, MVG,
      PALM, PCD, PCX, PDF, PICT, PNG, PS2, PS3, RLA, SGI, TIFF, TOPOL, TXT,
      WMF, WPG, and XCF.

      In order to support automatic file type identification (in case it is
      not already specified), GraphicsMagick reads the first part of the
      file data to determine the file format and then rewinds the file
      descriptor to the beginning of the file before invoking the correct
      decoder. It is not possible to rewind the file descriptor on a pipe.
      It is possible to buffer that input for subsequent re-use but this has
      not yet been implemented. For these cases, explicitly specifying the
      file format will avoid use of a temporary file:

      For example, without specifying the input format, '-debug
      TemporaryFile' reveals that a temporary file is used:

      % cat Sunrise.jpg | gm convert -debug TemporaryFile - null:
      09:01:24 0:01 0.000u 18358 tempfile.c/unknown/60/TemporaryFile:
         Allocating temporary file "/scratch/bfriesen/tempdir-1.4/gm6gw57B"
      09:01:25 0:01 0.310u 18358 tempfile.c/unknown/93/TemporaryFile:
         Deallocating temporary file "/scratch/bfriesen/tempdir-1.4/gm6gw57B"
      

      but if we specify the format of the input file, then there is no more
      use of a temporary file:

      % cat Sunrise.jpg | gm convert -debug TemporaryFile jpeg:- null:
      

      To summarize, if you are able to specify the format of the stream
      input and the coder does not require seek, then a temporary file will
      usually be avoided. Use '-debug TemporaryFile' to learn when
      temporary files are being used.

      Ultimately, you may find it easiest to simply redirect (MAGICK_TMPDIR
      environment variable) the temporary file location that GraphicsMagick
      uses to a RAM-backed temporary file directory.

      Bob

      Bob Friesenhahn
      bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
      GraphicsMagick Maintainer, http://www.GraphicsMagick.org/