Cannot process JP2

Help
2010-11-26
2012-10-06
  • Leoš Junek
    Leoš Junek
    2010-11-26

    Subject: Cannot process JP2 - Unexpected response from server /fcgi-bin/iipsrv.fcgi

    Hello!

    My task is to process JP2 with IIP Image server. Because I could use openSUSE
    11.3 (x86_64), not Debian, I got latest version of IIP server via SVN (latest
    version is reported to support JP2). Building IIP server and IIPMooViewer was
    successfull, example image PalaisDuLouvre.tif was processed corectly. But as
    mentioned, my task is to process JPEG2000. When I tried to process it, I got
    message "Unexpected response from server /fcgi-bin/iipsrv.fcgi".

    I also tried

    ./configure --with-kakadu=/path/to/kakadu/v6_4-01093N
    

    and that made iipsrv.fcgi about 0.5 MB bigger

    than iipsrv.fcgi configured with no option.

    Here are specification of JP2 I tried to process and some related snippets
    from iipsrv.log

    === Group 1 JP2 pictures ===
    Lossy or lossless: lossy
    Compression: 20:1 to 30:1
    Tilling: 1024x1024
    Progression order: RPCL
    Number of decomposition levels: 5
    Number of quality layers: 12 /logarithmic/
    Code block size (xcb = ycb): 6
    Transformation: 9-7 irreversible filter
    Precinct size: 256x256 for first two decomp. levels, 128 by 128 for lower
    levels
    Code block size: 64x64
    TLM markers: Yes “R”

    == Log snippets for Group 1 pics ==

    CVT maximum viewport size set to 3000
    Full Request is FIF=/tmp/test6/test_bedrich_1_ku_20.jp2&obj=IIP,1.0&obj=Max-
    size&obj=Tile-size&obj=Resolution-

    number
    : Command / Argument is FIF : /tmp/test6/test_bedrich_1_ku_20.jp2
    FIF handler reached
    FIF :: URL decoding/filtering: /tmp/test6/test_bedrich_1_ku_20.jp2 =>
    /tmp/test6/test_bedrich_1_ku_20.jp2
    FIF :: Image cache miss
    Unsupported image type: jp2


    Server: iipsrv/0.9.9
    Cache-Control: no-cache
    Content-Type: application/vnd.netfpx
    Content-Disposition:
    inline;filename="IIPisAMadGameClosedToOurUnderstanding.netfpx"

    Error/7:1 3 FIF


    Total Request Time: 289 microseconds
    image closed and deleted
    Server count is 4

    === Group 2 JP2 pictures ===
    Pictures converted with Kakadu with command

    kdu_compress -i input.tif -o output.jp2 -rate 0.5 Clayers=1 Clevels=7 "Cprecincts={256,256},{256,256},{256,256},
    
    {128,128},{128,128},{64,64},{64,64},{32,32},{16,16}" "Corder=RPCL" "ORGgen_plt=yes" "ORGtparts=R" "Cblk={64,64}"
    
    Cuse_sop=yes
    

    == Log snippets for Group 2 pics ==
    CVT maximum viewport size set to 3000
    Full Request is FIF=/tmp/JP2_for_IIP/zabicka.jp2&obj=IIP,1.0&obj=Max-size&obj
    =Tile-size&obj=Resolution-number
    : Command / Argument is FIF : /tmp/JP2_for_IIP/zabicka.jp2
    FIF handler reached
    FIF :: URL decoding/filtering: /tmp/JP2_for_IIP/zabicka.jp2 =>
    /tmp/JP2_for_IIP/zabicka.jp2
    FIF :: Image cache initialisation
    Unsupported image type: jp2


    Server: iipsrv/0.9.9
    Cache-Control: no-cache
    Content-Type: application/vnd.netfpx
    Content-Disposition:
    inline;filename="IIPisAMadGameClosedToOurUnderstanding.netfpx"

    Error/7:1 3 FIF


    Total Request Time: 596 microseconds
    image closed and deleted
    Server count is 4

    =============================

    I am not graphics formats geek. My colleagues told me the pictures contain
    several layers as does the example picture PalaisDuLouvre.tif. The problem
    with server can be caused with missing JP2 support in IIP Image Server or
    invalid format of tested JP2´s.

    Can anyone send a link to a JP2 that is in appropriate format for IIP Image
    server? With excellent JP2 I can test, whether the error is caused by missing
    JP2 support or wrong configuration.

    I would appreciate any help!

    Thanks in advance!

    Bye

    Leos

     
  • Ruven
    Ruven
    2010-11-26

    IIPImage supports JPEG2000 images even if they are not optimally encoded. From
    your iipsrv.log errors it looks like you didn't correctly include the kakadu
    sources in your compilation.

    During the configure stage, can you tell me whether it prints something about
    Kakadu? Make sure it correctly finds the sources. Then check that when you
    start the iipsrv process that it mentions Kakadu in the logfile during
    startup.

     
  • Leoš Junek
    Leoš Junek
    2010-11-26

    Good tip, ruven! Thanks!

    I forgot to say I use IIPMooViewer.

    I tried

    # ./configure --with-kakadu=/opt/kakadu/v6_4-01093N > config.log
    configure: WARNING: optional dynamic module loading not enabled
    # grep -i kakadu config.log
    checking for /opt/kakadu/v6_4-01093N/managed/all_includes/jpx.h... yes
    configure: Found Kakadu sources. Will compile JPEG2000 support
    configure:15677: checking for /opt/kakadu/v6_4-01093N/managed/all_includes/jpx.h
    configure:15695: result: configure: Found Kakadu sources. Will compile JPEG2000 support
    configure:15748: gcc -o conftest -g -O2   conftest.c -lc  -lz -lm  /opt/kakadu/v6_4-01093N/apps/make/jpx.o /opt/kakadu/v6_4-01093N/apps/make/jp2.o /opt/kakadu/v6_4-01093N/apps/make/kdu_stripe_decompressor.o /opt/kakadu/v6_4-01093N/apps/make/*.so -lpthread >&5
    | #define HAVE_KAKADU 1
    configure:15888: gcc -o conftest -g -O2   conftest.c -lnsl  -lz -lm  /opt/kakadu/v6_4-01093N/apps/make/jpx.o /opt/kakadu/v6_4-01093N/apps/make/jp2.o /opt/kakadu/v6_4-01093N/apps/make/kdu_stripe_decompressor.o /opt/kakadu/v6_4-01093N/apps/make/*.so -lpthread >&5
    configure:15933: gcc -o conftest -g -O2   conftest.c -lsocket  -lnsl -lz -lm  /opt/kakadu/v6_4-01093N/apps/make/jpx.o /opt/kakadu/v6_4-01093N/apps/make/jp2.o /opt/kakadu/v6_4-01093N/apps/make/kdu_stripe_decompressor.o /opt/kakadu/v6_4-01093N/apps/make/*.so -lpthread >&5
    | #define HAVE_KAKADU 1
    configure:16106: gcc -o conftest -g -O2    conftest.c -lpthreads -lnsl -lz -lm  /opt/kakadu/v6_4-01093N/apps/make/jpx.o /opt/kakadu/v6_4-01093N/apps/make/jp2.o /opt/kakadu/v6_4-01093N/apps/make/kdu_stripe_decompressor.o /opt/kakadu/v6_4-01093N/apps/make/*.so -lpthread >&5
    | #define HAVE_KAKADU 1
    configure:16106: gcc -o conftest -g -O2    conftest.c  -lnsl -lz -lm  /opt/kakadu/v6_4-01093N/apps/make/jpx.o /opt/kakadu/v6_4-01093N/apps/make/jp2.o /opt/kakadu/v6_4-01093N/apps/make/kdu_stripe_decompressor.o /opt/kakadu/v6_4-01093N/apps/make/*.so -lpthread >&5
    configure:16148: gcc -o conftest -g -O2    conftest.c  -lnsl -lz -lm  /opt/kakadu/v6_4-01093N/apps/make/jpx.o /opt/kakadu/v6_4-01093N/apps/make/jp2.o /opt/kakadu/v6_4-01093N/apps/make/kdu_stripe_decompressor.o /opt/kakadu/v6_4-01093N/apps/make/*.so -lpthread >&5
    | #define HAVE_KAKADU 1
    configure:16523: gcc -o conftest -g -O2   conftest.c -lnsl -lz -lm  /opt/kakadu/v6_4-01093N/apps/make/jpx.o /opt/kakadu/v6_4-01093N/apps/make/jp2.o /opt/kakadu/v6_4-01093N/apps/make/kdu_stripe_decompressor.o /opt/kakadu/v6_4-01093N/apps/make/*.so -lpthread >&5
    ac_cv_file__opt_kakadu_v6_4_01093N_managed_all_includes_jpx_h=yes
    DEFS='-DPACKAGE_NAME=\"iipsrv\" -DPACKAGE_TARNAME=\"iipsrv\" -DPACKAGE_VERSION=\"0.9.9\" -DPACKAGE_STRING=\"iipsrv\ 0.9.9\" -DPACKAGE_BUGREPORT=\"ruven@users.sourceforge.net\" -DPACKAGE_URL=\"\" -DPACKAGE=\"iipsrv\" -DVERSION=\"0.9.9\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE__BOOL=1 -DHAVE_STDBOOL_H=1 -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_GLOB_H=1 -DHAVE_TIME_H=1 -DHAVE_SYS_TIME_H=1 -DHAVE_MATH_H=1 -DHAVE_LIBM=1 -DHAVE_STDLIB_H=1 -DHAVE_MALLOC=1 -DHAVE_FLOOR=1 -DHAVE_GETTIMEOFDAY=1 -DHAVE_POW=1 -DHAVE_SQRT=1 -DHAVE_JPEG=1 -DHAVE_LIBZ=1 -DHAVE_TIFF=1 -DHAVE_KAKADU=1 -DNPROCS=1 -DHAVE_STREAMBUF_CHAR_TYPE=1 -DHAVE_LIBNSL=1 -DHAVE_FPOS=1 -DHAVE_SYS_SOCKET_H=1 -DHAVE_NETDB_H=1 -DHAVE_NETINET_IN_H=1 -DHAVE_ARPA_INET_H=1 -DHAVE_SYS_TIME_H=1 -DHAVE_LIMITS_H=1 -DHAVE_SYS_PARAM_H=1 -DHAVE_UNISTD_H=1 -DHAVE_FILENO_PROTO=1 -DHAVE_STRERROR=1'
    ENABLE_KAKADU_FALSE='#'
    ENABLE_KAKADU_TRUE=''
    EXTRAS='KakaduImage.o'
    INCLUDES=' -I/opt/kakadu/v6_4-01093N/managed/all_includes/ -I.'
    LIBS='-lnsl -lz -lm  /opt/kakadu/v6_4-01093N/apps/make/jpx.o /opt/kakadu/v6_4-01093N/apps/make/jp2.o /opt/kakadu/v6_4-01093N/apps/make/kdu_stripe_decompressor.o /opt/kakadu/v6_4-01093N/apps/make/*.so -lpthread'
    #define HAVE_KAKADU 1
    

    Although I don't know how to enable dynamic module loading, Kakadu sources
    seems to be correctly found.

    After Apache2 restart and viewing of testing HTML with JavaScript code these
    lines appeared in iipsrv.log:

    <----------------------------------->
    Fri Nov 26 15:31:40 2010
    
    IIPImage Server. Version 0.9.9
    *** Ruven Pillay <ruven@users.sourceforge.net> ***
    
    Verbosity level set to 5
    Running in FCGI mode
    
    Setting maximum image cache size to 10MB
    Setting filesystem prefix to ''
    Setting default JPEG quality to 50
    Setting maximum CVT size to 3000
    Setting 3D file sequence name pattern to '_pyr_'
    Loading watermark
    
    Initialisation Complete.
    <----------------------------------->
    

    So Kakadu is not mentioned here.

     
  • Ruven
    Ruven
    2010-11-26

    Configure is finding Kakadu OK. Maybe you should do something like a make
    clean, then rebuild iipsrv again.

     
  • Leoš Junek
    Leoš Junek
    2010-11-26

    It has helped! Thanks a lot, ruven!

    I have already tested three pictures. When seeing the first one, only preview
    "window" in right hand upper corner appeared. Second JP2 image caused blank
    black screen and error message I mentioned in my first post in this thread.
    Third JP2 was shown correctly.

    Now my task would be tuning IIP server configuration and looking for a reason
    why some JP2´s are problematic and some of them not.

    Let me drop some parts of image server log. It is just for people, who want to
    see a log entry of correctly shown JP2.

    Image server starts, JP2 support is being reported

    IIPImage Server. Version 0.9.9
    *** Ruven Pillay <ruven@users.sourceforge.net> ***
    
    Verbosity level set to 5
    Running in FCGI mode
    
    Setting maximum image cache size to 10MB
    Setting filesystem prefix to ''
    Setting default JPEG quality to 50
    Setting maximum CVT size to 3000
    Setting 3D file sequence name pattern to '_pyr_'
    Setting up JPEG2000 support via Kakadu SDK
    Loading watermark
    
    Initialisation Complete.
    <----------------------------------->
    

    And log entry describing the view of the third of mentioned JP2´s (the one
    that was displayed correctly):

    CVT maximum viewport size set to 3000
    Full Request is FIF=/tmp/test6/test_bedrich_1_ku_20.jp2&cnt=1&sds=0,90&jtl=4,27
    [1/4]: Command / Argument is FIF : /tmp/test6/test_bedrich_1_ku_20.jp2
    FIF handler reached
    FIF :: URL decoding/filtering: /tmp/test6/test_bedrich_1_ku_20.jp2 => /tmp/test6/test_bedrich_1_ku_20.jp2
    FIF :: Image cache hit. Number of elements: 1
    FIF :: JPEG2000 image requested
    FIF :: Created image
    FIF :: Image dimensions are 3709 x 4826
    FIF :: Image timestamp: Wed, 10 Nov 2010 13:45:43 GMT
    FIF :: Total command time 530 microseconds
    [2/4]: Command / Argument is cnt : 1
    CNT handler reached
    CNT :: requested contrast adjustment is 1
    [3/4]: Command / Argument is sds : 0,90
    SDS handler reached
    SDS :: set to 0, 90
    [4/4]: Command / Argument is jtl : 4,27
    JTL handler reached
    TileManager :: Cache Miss for resolution: 4, tile: 27
    TileManager :: Cache Size: 8 tiles, 0.0771709 MB
    TileManager :: JPEG Compression Time: 2234 microseconds
    TileManager :: Tile cache insertion time: 18 microseconds
    TileManager :: Total Tile Access Time: 13925 microseconds
    JTL :: Tile size: 256 x 256
    JTL :: Channels per sample: 3
    JTL :: Bits per channel: 8
    JTL :: Compressed tile size is 6515
    JTL :: Total command time 14096 microseconds
    Total Request Time: 15085 microseconds
    image closed and deleted
    Server count is 9
    

    My colleague Vasek advised me to check binary file iipsrv.fcgi with ldd,
    whether Kakadu libraries are included.

    ldd /usr/local/httpd/fcgi-bin/iipsrv.fcgi
            linux-vdso.so.1 =>  (0x00007fff0ccd1000)
            /opt/kakadu/v6_4-01093N/apps/make/libkdu_v64R.so (0x00007f68bafe8000)
            libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f68badd0000)
            libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f68babb3000)
            libjpeg.so.8 => /usr/lib64/libjpeg.so.8 (0x00007f68ba97a000)
            libtiff.so.3 => /usr/lib64/libtiff.so.3 (0x00007f68ba716000)
            libz.so.1 => /lib64/libz.so.1 (0x00007f68ba500000)
            libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f68ba1f6000)
            libm.so.6 => /lib64/libm.so.6 (0x00007f68b9f9f000)
            libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f68b9d89000)
            libc.so.6 => /lib64/libc.so.6 (0x00007f68b9a29000)
            /lib64/ld-linux-x86-64.so.2 (0x00007f68bb2af000)
    
     
  • Ruven
    Ruven
    2010-11-29

    If you want really detailed logging for Kakadu, open the file KakaduImage.cc
    in the iipsrv/src/ subdirectory and near the top, you should uncomment the
    line
    //#define DEBUG 1

    Then recompile and restart the server. You will then see lots more detailed
    logging about the JPEG2000 decoding process.

     


Anonymous


Cancel   Add attachments