IIPImage dropping out when serving some JP2 images

2012-11-06
2015-05-14
  • Hi

    We're experiencing some puzzling behaviour. Our apache hosted IIPImage server drops out when serving certain JP2 images, but we're not clear why, or what is the critical factor which is different about the particular images involved.

    The behaviour happens with specific images, and is reliably reproducable on our web service. It doesnt appear to be size which is the critical factor, as I'v identified larger JP2s in the same image collection which display just fine. The JP2 images were all created in the same way from .tif images using kakadu. I'v examined the JP2 images using a JP2 analalyzer, which reports them to be valid, and looked at the header data (nothing leaps out at me as obviously wrong), and I'v looked at the JP2 images themselves in Corel Paintshop Pro Photo x2 - the images look fine. I would like to be able to identify what is causing the problem so I can identify all the problem images and recreate them in a way which doesnt produce this problem - has anyone encountered similar issues or can anyone point me towards the sort of thing I should be looking for in the images themselves, or in the server?

     
  • Ruven
    Ruven
    2012-11-06

    What do you mean by "drops out"? It crashes the server or is the image tile corrupted?

    And what version of the server are you running? Pre-built or compiled yourself? What OS and version are you running this on?

     
  • the image tile is not corrupted, IIPImage crashes. The apache server hosting it does not crash. its running on linux Ubuntu. IIP Image was installed from a binary download iipimage-0.9.9-jp2_i386.deb, and this set up has until recently not been giving us any problems - I believe the problem is something particular to certain images, but what is bewildering is it is only certain images from within this set which produce the behaviour

     
  • Ruven
    Ruven
    2012-11-06

    It's most probably a problem with the server. Do you think you could send me a copy of the image causing you problems?

     
  • sure can! in fact I can send two similar images of the same ubject, one of which causes dropout, the other displays as expected. But how do i send you the image? Sorry to aska daft question, but I'v never actually sent a file via sourceforge before

     
  • got an error message in the apache log as below (this was for another of the images)

    [Tue Nov 06 08:06:23 2012] [error] [client 86.137.15.135] FastCGI: incomplete headers (0 bytes) received from server "/usr/lib/cgi-bin/iipsrv.fcgi", referer: https://dlib.york.ac.uk/yodl/app/image/detail?id=york%3A800430&ref=browse
    [Tue Nov 06 08:06:23 2012] [warn] FastCGI: server "/usr/lib/cgi-bin/iipsrv.fcgi" (pid 30686) terminated due to uncaught signal '11' (Segmentation fault)
    [Tue Nov 06 08:06:23 2012] [warn] FastCGI: server "/usr/lib/cgi-bin/iipsrv.fcgi" restarted (pid 9055)

    We havnt got our IIP logging set to a high level because with the logging set high we were getting unmanageably large logs

     
  • Ruven
    Ruven
    2012-11-06

    OK, I'll take a look at this. Is there a particular request for an individual tile which can cause the crash?

     
  • hmm, just emailed the developer who set up the service on this one..I'll get back when I know!

     
  • ok. a typical request part would look like
    https://dlib.york.ac.uk/cgi-bin/iipsrv.fcgi?DeepZoom=/usr/digilib-webdocs/digilibImages/HOA/current/F/20120607/1821b707-52f2-4083-90af-72e8b28ce7d2_Fig_22Gallery_2William_Etty__A_bather.jp2_files/10/1_0.jpg

    looking in firebug at a request for one of the images that consistently fails, it appears that NONE of the constituent tiles are returned successfully.
    And yet diffing the analyzer output from a file which works and a file that doesnt, I cant see anything that stands out.

    here,for example, is the codestream data for both
    1) working image
    <contiguousCodestreamBox><siz><lsiz>41</lsiz><rsiz>ISO/IEC 15444-1</rsiz><xsiz>3173</xsiz><ysiz>2344</ysiz><xOsiz>0</xOsiz><yOsiz>0</yOsiz><xTsiz>3173</xTsiz><yTsiz>2344</yTsiz><xTOsiz>0</xTOsiz><yTOsiz>0</yTOsiz>
    <numberOfTiles>1</numberOfTiles>
    <csiz>1</csiz><ssizSign>unsigned</ssizSign><ssizDepth>8</ssizDepth><xRsiz>1</xRsiz><yRsiz>1</yRsiz></siz><cod><lcod>20</lcod><precincts>yes</precincts><sop>yes</sop><eph>no</eph><order>RPCL</order><layers>1</layers>

    <multipleComponentTransformation>no</multipleComponentTransformation>
    <levels>7</levels>
    <codeBlockWidth>64</codeBlockWidth>
    <codeBlockHeight>64</codeBlockHeight>
    <codingBypass>no</codingBypass>
    <resetOnBoundaries>no</resetOnBoundaries>
    <termOnEachPass>no</termOnEachPass>
    <vertCausalContext>no</vertCausalContext>
    <predTermination>no</predTermination><segmentationSymbols>no</segmentationSymbols>
    <transformation>9-7 irreversible</transformation>

    <precinctSizeX>32</precinctSizeX><precinctSizeY>32</precinctSizeY><precinctSizeX>64</precinctSizeX><precinctSizeY>64</precinctSizeY><precinctSizeX>64</precinctSizeX><precinctSizeY>64</precinctSizeY><precinctSizeX>128</precinctSizeX><precinctSizeY>128</precinctSizeY><precinctSizeX>128</precinctSizeX><precinctSizeY>128</precinctSizeY><precinctSizeX>256</precinctSizeX><precinctSizeY>256</precinctSizeY><precinctSizeX>256</precinctSizeX><precinctSizeY>256</precinctSizeY><precinctSizeX>256</precinctSizeX><precinctSizeY>256</precinctSizeY></cod><qcd><lqcd>47</lqcd><qStyle>scalar expounded</qStyle><guardBits>1</guardBits><mu>1822</mu><epsilon>16</epsilon><mu>1770</mu><epsilon>16</epsilon><mu>1770</mu><epsilon>16</epsilon><mu>1724</mu><epsilon>16</epsilon><mu>1770</mu><epsilon>15</epsilon><mu>1770</mu><epsilon>15</epsilon><mu>1724</mu><epsilon>15</epsilon></qcd><com><lcom>17</lcom><rcom>ISO/IEC 8859-15 (Latin)</rcom><comment>Kakadu-v6.4.1</comment></com><com><lcom>92</lcom><rcom>ISO/IEC 8859-15 (Latin)</rcom><comment>Kdu-Layer-Info: log_2{Delta-D(squared-error)/Delta-L(bytes)}, L(bytes) -23.1, 4.6e+05</comment></com>

    <tileParts><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>617</psot><tpsot>0</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>1309</psot><tpsot>1</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>4330</psot><tpsot>2</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>14254</psot><tpsot>3</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>41309</psot><tpsot>4</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>99949</psot><tpsot>5</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>158288</psot><tpsot>6</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>145020</psot><tpsot>7</tpsot><tnsot>8</tnsot></sot></tilePart>
    </tileParts>
    </contiguousCodestreamBox>

    <compressionRatio>15.47</compressionRatio>

    failing image
    <contiguousCodestreamBox><siz><lsiz>41</lsiz><rsiz>ISO/IEC 15444-1</rsiz><xsiz>1435</xsiz><ysiz>2007</ysiz><xOsiz>0</xOsiz><yOsiz>0</yOsiz><xTsiz>1435</xTsiz><yTsiz>2007</yTsiz><xTOsiz>0</xTOsiz><yTOsiz>0</yTOsiz>
    <numberOfTiles>1</numberOfTiles>
    <csiz>1</csiz><ssizSign>unsigned</ssizSign><ssizDepth>8</ssizDepth><xRsiz>1</xRsiz><yRsiz>1</yRsiz></siz><cod><lcod>20</lcod><precincts>yes</precincts><sop>yes</sop><eph>no</eph><order>RPCL</order><layers>1</layers>

    <multipleComponentTransformation>no</multipleComponentTransformation>
    <levels>7</levels>
    <codeBlockWidth>64</codeBlockWidth>
    <codeBlockHeight>64</codeBlockHeight>
    <codingBypass>no</codingBypass>
    <resetOnBoundaries>no</resetOnBoundaries>
    <termOnEachPass>no</termOnEachPass>
    <vertCausalContext>no</vertCausalContext>
    <predTermination>no</predTermination><segmentationSymbols>no</segmentationSymbols>
    <transformation>9-7 irreversible</transformation>

    <precinctSizeX>32</precinctSizeX><precinctSizeY>32</precinctSizeY><precinctSizeX>64</precinctSizeX><precinctSizeY>64</precinctSizeY><precinctSizeX>64</precinctSizeX><precinctSizeY>64</precinctSizeY><precinctSizeX>128</precinctSizeX><precinctSizeY>128</precinctSizeY><precinctSizeX>128</precinctSizeX><precinctSizeY>128</precinctSizeY><precinctSizeX>256</precinctSizeX><precinctSizeY>256</precinctSizeY><precinctSizeX>256</precinctSizeX><precinctSizeY>256</precinctSizeY><precinctSizeX>256</precinctSizeX><precinctSizeY>256</precinctSizeY>
    </cod><qcd><lqcd>47</lqcd><qStyle>scalar expounded</qStyle><guardBits>1</guardBits><mu>1822</mu><epsilon>16</epsilon><mu>1770</mu><epsilon>16</epsilon><mu>1770</mu><epsilon>16</epsilon><mu>1724</mu><epsilon>16</epsilon><mu>1770</mu><epsilon>15</epsilon><mu>1770</mu><epsilon>15</epsilon><mu>1724</mu><epsilon>15</epsilon></qcd><com><lcom>17</lcom><rcom>ISO/IEC 8859-15 (Latin)</rcom><comment>Kakadu-v6.4.1</comment></com><com><lcom>92</lcom><rcom>ISO/IEC 8859-15 (Latin)</rcom><comment>Kdu-Layer-Info: log_2{Delta-D(squared-error)/Delta-L(bytes)}, L(bytes) -25.0, 1.8e+05</comment></com>

    <tileParts><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>270</psot><tpsot>0</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>569</psot><tpsot>1</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>1637</psot><tpsot>2</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>4548</psot><tpsot>3</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>12745</psot><tpsot>4</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>32151</psot><tpsot>5</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>60663</psot><tpsot>6</tpsot><tnsot>0</tnsot></sot></tilePart><tilePart><sot><lsot>10</lsot><isot>0</isot><psot>67457</psot><tpsot>7</tpsot><tnsot>8</tnsot></sot></tilePart>
    </tileParts>
    </contiguousCodestreamBox>

    <compressionRatio>14.72</compressionRatio>

     
  • OK.
    A typical request part looks like this:https://<our host="">/cgi-bin/iipsrv.fcgi?DeepZoom=/<our image="" file="" path="">/1821b707-52f2-4083-90af-72e8b28ce7d2_Fig_22Gallery_2William_Etty__A_bather.jp2_files/10/1_0.jpg

    looking at the firebug information for a failing image, ALL of the constituent tile requests seem to be failing, each ultimately returns a 500 error.

    Its really odd, because running at diff at JP" analyzer data from an image which always works, and one which always fails, they appear very similar.

    for instance the codestream data - see attached

     
    Attachments
  • Ruven
    Ruven
    2012-11-16

    Do you know the encoding parameters you used for each of these images?

     


Anonymous


Cancel   Add attachments