Menu ▾ ▴

#4588 Squid problem when cache is large

1.710
closed-fixed
nobody
squid (3)
5
2015-04-21
2015-04-20
Elias Andrade
No

Hi guys,

Sometime, when we apply a change on squid module (Access Control or Ports) happens a strange problem: squid process is restarted and store rebuilding happens.

The problem when the squid cache dir size increases (in my case i have 5 cache dir in /var/spool/squid3/ directory): a very high time do rebuild caches cause a Terminated abnormally of squid process.

O believe that problem is: squid restart process instead of giving a "-k reconfigure", forcing a store rebuilding:

cache.log:

==================
2015/04/17 09:39:48| Done reading /var/spool/squid3/cache5 swaplog (1864 entries)
2015/04/17 09:39:48| Store rebuilding is 14.83% complete
2015/04/17 09:40:37| Done reading /var/spool/squid3/cache1 swaplog (27647 entries)
2015/04/17 09:40:38| Done reading /var/spool/squid3/cache3 swaplog (27772 entries)
2015/04/17 09:40:38| Done reading /var/spool/squid3/cache2 swaplog (27982 entries)
2015/04/17 09:40:38| Done reading /var/spool/squid3/cache4 swaplog (27967 entries)
2015/04/17 09:40:38| Finished rebuilding storage from disk.
2015/04/17 09:40:38| 112998 Entries scanned
2015/04/17 09:40:38| 0 Invalid entries.
2015/04/17 09:40:38| 0 With invalid flags.
2015/04/17 09:40:38| 112782 Objects loaded.
2015/04/17 09:40:38| 0 Objects expired.
2015/04/17 09:40:38| 216 Objects cancelled.
2015/04/17 09:40:38| 18 Duplicate URLs purged.
2015/04/17 09:40:38| 0 Swapfile clashes avoided.
2015/04/17 09:40:38| Took 50.62 seconds (2227.89 objects/sec).
2015/04/17 09:40:38| Beginning Validation Procedure
2015/04/17 09:40:38| Completed Validation Procedure
2015/04/17 09:40:38| Validated 221055 Entries
2015/04/17 09:40:38| store_swap_size = 17731496256
FATAL: xcalloc: Unable to allocate 4073701254 blocks of 1 bytes!

Squid Cache (Version 3.1.20): Terminated abnormally.
CPU Usage: 50.167 seconds = 49.843 user + 0.324 sys
Maximum Resident Size: 232144 KB
Page faults with physical i/o: 0
Memory usage for squid via mallinfo():
total space in arena: 26868 KB
Ordinary blocks: 26227 KB 7 blks
Small blocks: 0 KB 0 blks
Holding blocks: 53556 KB 15 blks
Free Small blocks: 0 KB
Free Ordinary blocks: 640 KB
Total in use: 79783 KB 297%
Total free: 640 KB 2%
2015/04/17 09:40:42| Starting Squid Cache version 3.1.20 for x86_64-pc-linux-gnu...
2015/04/17 09:40:42| Process ID 42815
2015/04/17 09:40:42| With 65535 file descriptors available
2015/04/17 09:40:42| Initializing IP Cache...

===============

Have a workaround to this?

Thanks!

Related

Bugs: #4588

Discussion

  • Jamie Cameron

    Jamie Cameron - 2015-04-21

    In the Squid module of Webmin, click on the Module Config link and see what the "Command to apply changes" is set to.

     

    • Elias Andrade

      Elias Andrade - 2015-04-21

      Dear Jamie,

      I'm change from:
      service squid3 restart

      To:
      service squid3 reload

      And now the cache log don't show directory rebuild!

      Thanks

      On 21-04-2015 01:25, Jamie Cameron wrote:

      In the Squid module of Webmin, click on the Module Config link and see
      what the "Command to apply changes" is set to.

      [bugs:#4588] http://sourceforge.net/p/webadmin/bugs/4588 Squid
      problem when cache is large

      Status: open
      Group: 1.710
      Labels: squid
      Created: Mon Apr 20, 2015 04:51 PM UTC by Elias Andrade
      Last Updated: Mon Apr 20, 2015 04:51 PM UTC
      Owner: nobody

      Hi guys,

      Sometime, when we apply a change on squid module (Access Control or
      Ports) happens a strange problem: squid process is restarted and store
      rebuilding happens.

      The problem when the squid cache dir size increases (in my case i have 5
      cache dir in /var/spool/squid3/ directory): a very high time do rebuild
      caches cause a Terminated abnormally of squid process.

      O believe that problem is: squid restart process instead of giving a "-k
      reconfigure", forcing a store rebuilding:

      cache.log:

      ==================
      2015/04/17 09:39:48| Done reading /var/spool/squid3/cache5 swaplog (1864
      entries)
      2015/04/17 09:39:48| Store rebuilding is 14.83% complete
      2015/04/17 09:40:37| Done reading /var/spool/squid3/cache1 swaplog
      (27647 entries)
      2015/04/17 09:40:38| Done reading /var/spool/squid3/cache3 swaplog
      (27772 entries)
      2015/04/17 09:40:38| Done reading /var/spool/squid3/cache2 swaplog
      (27982 entries)
      2015/04/17 09:40:38| Done reading /var/spool/squid3/cache4 swaplog
      (27967 entries)
      2015/04/17 09:40:38| Finished rebuilding storage from disk.
      2015/04/17 09:40:38| 112998 Entries scanned
      2015/04/17 09:40:38| 0 Invalid entries.
      2015/04/17 09:40:38| 0 With invalid flags.
      2015/04/17 09:40:38| 112782 Objects loaded.
      2015/04/17 09:40:38| 0 Objects expired.
      2015/04/17 09:40:38| 216 Objects cancelled.
      2015/04/17 09:40:38| 18 Duplicate URLs purged.
      2015/04/17 09:40:38| 0 Swapfile clashes avoided.
      2015/04/17 09:40:38| Took 50.62 seconds (2227.89 objects/sec).
      2015/04/17 09:40:38| Beginning Validation Procedure
      2015/04/17 09:40:38| Completed Validation Procedure
      2015/04/17 09:40:38| Validated 221055 Entries
      2015/04/17 09:40:38| store_swap_size = 17731496256
      FATAL: xcalloc: Unable to allocate 4073701254 blocks of 1 bytes!

      Squid Cache (Version 3.1.20): Terminated abnormally.
      CPU Usage: 50.167 seconds = 49.843 user + 0.324 sys
      Maximum Resident Size: 232144 KB
      Page faults with physical i/o: 0
      Memory usage for squid via mallinfo():
      total space in arena: 26868 KB
      Ordinary blocks: 26227 KB 7 blks
      Small blocks: 0 KB 0 blks
      Holding blocks: 53556 KB 15 blks
      Free Small blocks: 0 KB
      Free Ordinary blocks: 640 KB
      Total in use: 79783 KB 297%
      Total free: 640 KB 2%
      2015/04/17 09:40:42| Starting Squid Cache version 3.1.20 for
      x86_64-pc-linux-gnu...
      2015/04/17 09:40:42| Process ID 42815
      2015/04/17 09:40:42| With 65535 file descriptors available
      2015/04/17 09:40:42| Initializing IP Cache...

      ===============

      Have a workaround to this?

      Thanks!

      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/webadmin/bugs/4588/
      https://sourceforge.net/p/webadmin/bugs/4588

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/
      https://sourceforge.net/auth/subscriptions

       

      Related

      Bugs: #4588

      • Jamie Cameron

        Jamie Cameron - 2015-04-21

        Thanks - I will have Webmin use that command in future by default.

        On 21/Apr/2015 00:08 Elias Andrade monesa@users.sf.net wrote ..

        Dear Jamie,

        I'm change from: service squid3 restart

        To: service squid3 reload

        And now the cache log don't show directory rebuild!

        Thanks

        On 21-04-2015 01:25, Jamie Cameron wrote:

        In the Squid module of Webmin, click on the Module Config link and see what the "Command to apply changes" is set to.

        [bugs:#4588] http://sourceforge.net/p/webadmin/bugs/4588 Squid problem when cache is large

        Status: open Group: 1.710 Labels: squid Created: Mon Apr 20, 2015 04:51 PM UTC by Elias Andrade Last Updated: Mon Apr 20, 2015 04:51 PM UTC Owner: nobody

        Hi guys,

        Sometime, when we apply a change on squid module (Access Control or Ports) happens a strange problem: squid process is restarted and store rebuilding happens.

        The problem when the squid cache dir size increases (in my case i have 5 cache dir in /var/spool/squid3/ directory): a very high time do rebuild caches cause a Terminated abnormally of squid process.

        O believe that problem is: squid restart process instead of giving a "-k reconfigure", forcing a store rebuilding:

        cache.log:

        ================== 2015/04/17 09:39:48| Done reading /var/spool/squid3/cache5 swaplog (1864 entries) 2015/04/17 09:39:48| Store rebuilding is 14.83% complete 2015/04/17 09:40:37| Done reading /var/spool/squid3/cache1 swaplog (27647 entries) 2015/04/17 09:40:38| Done reading /var/spool/squid3/cache3 swaplog (27772 entries) 2015/04/17 09:40:38| Done reading /var/spool/squid3/cache2 swaplog (27982 entries) 2015/04/17 09:40:38| Done reading /var/spool/squid3/cache4 swaplog (27967 entries) 2015/04/17 09:40:38| Finished rebuilding storage from disk. 2015/04/17 09:40:38| 112998 Entries scanned 2015/04/17 09:40:38| 0 Invalid entries. 2015/04/17 09:40:38| 0 With invalid flags. 2015/04/17 09:40:38| 112782 Objects loaded. 2015/04/17 09:40:38| 0 Objects expired. 2015/04/17 09:40:38| 216 Objects cancelled. 2015/04/17 09:40:38| 18 Duplicate URLs purged. 2015/04/17 09:40:38| 0 Swapfile clashes avoided. 2015/04/17 09:40:38| Took 50.62 seconds (2227.89 objects/sec). 2015/04/17 09:40:38| Begin
        ning Validation Procedure 2015/04/17 09:40:38| Completed Validation Procedure 2015/04/17 09:40:38| Validated 221055 Entries 2015/04/17 09:40:38| store_swap_size = 17731496256 FATAL: xcalloc: Unable to allocate 4073701254 blocks of 1 bytes!

        Squid Cache (Version 3.1.20): Terminated abnormally. CPU Usage: 50.167 seconds = 49.843 user + 0.324 sys Maximum Resident Size: 232144 KB Page faults with physical i/o: 0 Memory usage for squid via mallinfo(): total space in arena: 26868 KB Ordinary blocks: 26227 KB 7 blks Small blocks: 0 KB 0 blks Holding blocks: 53556 KB 15 blks Free Small blocks: 0 KB Free Ordinary blocks: 640 KB Total in use: 79783 KB 297% Total free: 640 KB 2% 2015/04/17 09:40:42| Starting Squid Cache version 3.1.20 for x86_64-pc-linux-gnu... 2015/04/17 09:40:42| Process ID 42815 2015/04/17 09:40:42| With 65535 file descriptors available 2015/04/17 09:40:42| Initializing IP Cache...

        ===============

        Have a workaround to this?

        Thanks!

        Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/webadmin/bugs/4588/ https://sourceforge.net/p/webadmin/bugs/4588

        To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/ https://sourceforge.net/auth/subscriptions

        [bugs:#4588] Squid problem when cache is large

        Status: open Group: 1.710 Labels: squid Created: Mon Apr 20, 2015 04:51 PM UTC by Elias Andrade Last Updated: Tue Apr 21, 2015 06:56 AM UTC Owner: nobody

        Hi guys,

        Sometime, when we apply a change on squid module (Access Control or Ports) happens a strange problem: squid process is restarted and store rebuilding happens.

        The problem when the squid cache dir size increases (in my case i have 5 cache dir in /var/spool/squid3/ directory): a very high time do rebuild caches cause a Terminated abnormally of squid process.

        O believe that problem is: squid restart process instead of giving a "-k reconfigure", forcing a store rebuilding:

        cache.log:

        ================== 2015/04/17 09:39:48| Done reading /var/spool/squid3/cache5 swaplog (1864 entries) 2015/04/17 09:39:48| Store rebuilding is 14.83% complete 2015/04/17 09:40:37| Done reading /var/spool/squid3/cache1 swaplog (27647 entries) 2015/04/17 09:40:38| Done reading /var/spool/squid3/cache3 swaplog (27772 entries) 2015/04/17 09:40:38| Done reading /var/spool/squid3/cache2 swaplog (27982 entries) 2015/04/17 09:40:38| Done reading /var/spool/squid3/cache4 swaplog (27967 entries) 2015/04/17 09:40:38| Finished rebuilding storage from disk. 2015/04/17 09:40:38| 112998 Entries scanned 2015/04/17 09:40:38| 0 Invalid entries. 2015/04/17 09:40:38| 0 With invalid flags. 2015/04/17 09:40:38| 112782 Objects loaded. 2015/04/17 09:40:38| 0 Objects expired. 2015/04/17 09:40:38| 216 Objects cancelled. 2015/04/17 09:40:38| 18 Duplicate URLs purged. 2015/04/17 09:40:38| 0 Swapfile clashes avoided. 2015/04/17 09:40:38| Took 50.62 seconds (2227.89 objects/sec). 2015/04/17 09:40:38| Begin
        ning Validation Procedure 2015/04/17 09:40:38| Completed Validation Procedure 2015/04/17 09:40:38| Validated 221055 Entries 2015/04/17 09:40:38| store_swap_size = 17731496256 FATAL: xcalloc: Unable to allocate 4073701254 blocks of 1 bytes!

        Squid Cache (Version 3.1.20): Terminated abnormally. CPU Usage: 50.167 seconds = 49.843 user + 0.324 sys Maximum Resident Size: 232144 KB Page faults with physical i/o: 0 Memory usage for squid via mallinfo(): total space in arena: 26868 KB Ordinary blocks: 26227 KB 7 blks Small blocks: 0 KB 0 blks Holding blocks: 53556 KB 15 blks Free Small blocks: 0 KB Free Ordinary blocks: 640 KB Total in use: 79783 KB 297% Total free: 640 KB 2% 2015/04/17 09:40:42| Starting Squid Cache version 3.1.20 for x86_64-pc-linux-gnu... 2015/04/17 09:40:42| Process ID 42815 2015/04/17 09:40:42| With 65535 file descriptors available 2015/04/17 09:40:42| Initializing IP Cache...

        ===============

        Have a workaround to this?

        Thanks!

        Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/webadmin/bugs/4588/

        To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

         

        Related

        Bugs: #4588

        alternate

  • Elias Andrade

    Elias Andrade - 2015-04-21

    Dear Jamie,

    I'm change from:
    service squid3 restart

    To:
    service squid3 reload

    And now the cache log don't show directory rebuild!

    Thanks

     

  • Jamie Cameron

    Jamie Cameron - 2015-04-21
    • status: open --> closed-fixed
     

  • Jamie Cameron

    Jamie Cameron - 2015-04-21

    I will have Webmin use "service squid3 reload" in future by default to apply settings, which should avoid this .

     

Log in to post a comment.