Menu

#501 Email autoclearing uses all available memory, crashes server

2.004
closed-fixed
nobody
None
5
2023-10-20
2023-10-18
No

Not sure if this is a new problem or a resurrection of #377. I recently enabled "Auto-clearing" on a customer's Trash folder in Usermin > Mail > Manage Folders > Trash. I think it contained about 4,000 messages before the clearout. An hour later the server ran out of memory and became unusable as the cron job fired and the disk started "thrashing" trying to use swap memory.

I rebooted the server but of course the same thing happened an hour later. It took me a while to work out what was eating all the memory.

I'm guessing the "push" statements in \usr\share\usermin\mailbox\auto.pl are responsible.

The server is running Ubuntu 22.04.3, Webmin 2.101, Virtualmin 7.8.2, Usermin 2.001, 4 GB of RAM, 8 GB Virtual memory, 3 cores.

Discussion

  • Jamie Cameron

    Jamie Cameron - 2023-10-18

    Was it the auto.pl process that was using all the RAM, or Dovecot?

    Also, how large in GB was this trash folder?

     
  • Phil McKerracher

    I'm not sure and I can't check it now, but I think I remember seeing auto.pl in "top" (but I was concentrating on CPU activity at the time). Similarly with trash folder size - it was large, several GB, maybe 10, I think.

     
  • Jamie Cameron

    Jamie Cameron - 2023-10-18

    10 GB is pretty large ... it's possible that scanning it could have used a lot of RAM.

    Do you know if the folder is in mbox (single file) or Maildir (one file per message) format?

     
  • Phil McKerracher

    It's in Maildir format. I've deleted it now but I do have backups. They contain potentially confidential messages but I might be able to get permission from the customer to release them if it would help. And anyway you're Australian and therefore trustworthy, right? :-)

     
    • Jamie Cameron

      Jamie Cameron - 2023-10-18

      Well, as everyone knows, and Australia is entirely peopled with criminals :-)

      I don't think I need access to the actual messages though, because the automatic cleanup script shouldn't actually be reading the entire files. However, it would be useful to know what "Action when messages to delete are found" is set to on the user's mailbox cleanup page.

       
      😄
      1
  • Phil McKerracher

    (Some of us descended from warders, which is worse!)

    Action set to "Delete matching messages" where criteria are "Messages older than 30 days", see attached.

     
  • Jamie Cameron

    Jamie Cameron - 2023-10-19

    Ok ... in that case, only the headers of the email messages should be read.

    Can you post a screenshot of the Manage Folders page in Usermin for this user?

     
  • Phil McKerracher

    Sure, here it is...

     
  • Jamie Cameron

    Jamie Cameron - 2023-10-19

    Ok I see the bug that causes this now! It will be fixed in the next Usermin release ...

     
  • Jamie Cameron

    Jamie Cameron - 2023-10-19
    • status: open --> closed-fixed
     
  • Phil McKerracher

    Brilliant, thanks.

     

Log in to post a comment.