Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

memory problem related to squashfs

Help
Yong
2006-01-06
2013-05-29
  • Yong
    Yong
    2006-01-06

    Hi,

    I am using 2.6.8.1 with squashfs 2.0 in my device. Have found a problem related to squashfs? Any one knows how to solve this?

    If I open/read a file for the first time, I will see a used memory increase.

    After the first time, the used memory will become unchanged.

    This used memory seems not drop back.

    After enable the trace in squashfs. I found there are squashfs logmsg in the first time file read. The second/third time does not have these output.

    It seems the file will be stored in the memory in the first time read.

    This can be test easily with "cat filename". We also has a RAM fs. cat RAM fs files does not cause used memory increase.

    Is this caused by squashfs or by other linux config? Is there a way disable it?

    Thanks!

    Yong

    4.html is a 121 bytes html file.

    # sysinfo
    Number of processes: 32
    12:39am  up 39 min,
    load average: 1 min:0.00, 5 min:0.00, 15 min:0.10
                  total         used         free       shared      buffers
      Mem:        30428        14956        15472            0         1180
    Swap:            0            0            0
    Total:        30428        14956        15472
    #
    # handle_request filename 4.html
    handle_request output filename /webs/4.html

    SQUASHFS: Entered squashfs_readpage, page index 0, start block 3b47c
    SQUASHFS: Entered squashfs_get_cached_block [28ad65:136a]
    SQUASHFS: Block @ 0x3b47c, compressed size 105
    SQUASHFS: Calling decompress_lzma_7z: buffer_in size=105, buffer_out size=65536
    SQUASHFS: Done with decompress_lzma_7z: output size=65536
    inode.c read_data 65536
    SQUASHFS: bytes 65536, i 0, byte_offset 0, available_bytes 4096
    SQUASHFS: bytes 65536, i 1, byte_offset 4096, available_bytes 4096
    SQUASHFS: bytes 65536, i 2, byte_offset 8192, available_bytes 4096
    SQUASHFS: bytes 65536, i 3, byte_offset 12288, available_bytes 4096
    SQUASHFS: bytes 65536, i 4, byte_offset 16384, available_bytes 4096
    SQUASHFS: bytes 65536, i 5, byte_offset 20480, available_bytes 4096
    SQUASHFS: bytes 65536, i 6, byte_offset 24576, available_bytes 4096
    SQUASHFS: bytes 65536, i 7, byte_offset 28672, available_bytes 4096
    SQUASHFS: bytes 65536, i 8, byte_offset 32768, available_bytes 4096
    SQUASHFS: bytes 65536, i 9, byte_offset 36864, available_bytes 4096
    SQUASHFS: bytes 65536, i 10, byte_offset 40960, available_bytes 4096
    SQUASHFS: bytes 65536, i 11, byte_offset 45056, available_bytes 4096
    SQUASHFS: bytes 65536, i 12, byte_offset 49152, available_bytes 4096
    SQUASHFS: bytes 65536, i 13, byte_offset 53248, available_bytes 4096
    SQUASHFS: bytes 65536, i 14, byte_offset 57344, available_bytes 4096
    SQUASHFS: bytes 65536, i 15, byte_offset 61440, available_bytes 4096

    # sysinfo
    Number of processes: 32
    12:39am  up 40 min,
    load average: 1 min:0.00, 5 min:0.00, 15 min:0.09
                  total         used         free       shared      buffers
      Mem:        30428        15052        15376            0         1180
    Swap:            0            0            0
    Total:        30428        15052        15376
    # sysinfo
    Number of processes: 32
    12:40am  up 40 min,
    load average: 1 min:0.00, 5 min:0.00, 15 min:0.09
                  total         used         free       shared      buffers
      Mem:        30428        15052        15376            0         1180
    Swap:            0            0            0
    Total:        30428        15052        15376
    # handle_request filename 4.html
    handle_request output filename /webs/4.html

    # sysinfo
    Number of processes: 32
    12:40am  up 40 min,
    load average: 1 min:0.00, 5 min:0.00, 15 min:0.08
                  total         used         free       shared      buffers
      Mem:        30428        15060        15368            0         1180
    Swap:            0            0            0
    Total:        30428        15060        15368
    # sysinfo
    Number of processes: 32
    12:40am  up 40 min,
    load average: 1 min:0.00, 5 min:0.00, 15 min:0.08
                  total         used         free       shared      buffers
      Mem:        30428        15052        15376            0         1180
    Swap:            0            0            0
    Total:        30428        15052        15376
    #

     
    • > Have found a
      > problem related to squashfs? Any one knows how to solve this?
      > If I open/read a file for the first time, I will see a used memory
      > increase.

      The used memory increase is because of file caching.  This is normal behaviour for Linux.  Linux has three caches which are involved in filesystem reading: the page cache, the buffer cache and the dentry (directory entry) cache.  As files are read, Linux saves the information in these caches.

      These caches use unallocated free memory.  The memory used for the caches doesn't become unavailable for program use, in low-memory situations Linux will shrink these caches releasing the memory.

      > This used memory seems not drop back.

      The used memory will be returned when Linux needs it for other uses.

      > This can be test easily with "cat filename". We also has a
      > RAM fs. cat RAM fs files does not cause used memory increase.

      Linux RAM filesystems (i.e. tmpfs) store their data inside the caches (locked so it cannot be flushed).  Reading such filesystems do not pull data into the caches because it is already there.

      > After enable the trace in squashfs. I found there are squashfs logmsg
      > in the first time file read. The second/third time does not have these
      >output.

      The logging data is produced when Squashfs is entered to read filesystem (page) data.  In the second and third cases the file read is satisfied from the page cache, and so Squashfs isn't re-entered to read the data, and it therefore doesn't generate any additional logging data.

      This is entirely normal behaviour.

      > Is this caused by squashfs or by other linux config? Is there a way
      > disable it?

      As previously mentioned, this is normal Linux behaviour.  It is such a central part of the design of Linux that it is very hard to disable it. As the data used for the caches is returned when required, there is no reason why you would want to disable it either.

      Phillip Lougher