I've come up with a way I've sped up the majority of metalog logs on my system, and I decided to share it with others. It involves using a ramdisk to store the unimportant logs. A ramdisk, for those who don't know, means a section of RAM that you store files to. This is useful because it avoids writing data to your hard drive, which is much slower than RAM, and the files are automatically gone when you turn off your computer (which IS sometimes an advantage).
The first thing you do is create a ramdisk where the logs will go; I already had one at /tmp so I'll use that as an example. You can create a ramdisk at /tmp, with new (2.6) kernel versions at least, by putting something like this is fstab:
none /tmp tmpfs defaults 0 0
Then you tell metalog to log absolutely everything to a folder in there, such as /tmp/logall. You should also restrict the size of it and the max # of files that get saved to to prevent your ramdisk from filling up. The section in my metalog.conf is:
Finally, I like to allow my regular user to have permission to read these logs. Since /tmp is deleted on shutdown, to do this you have to create the /tmp/logall directory at startup (in whatever startup script you use to run commands at startup) and give it the proper permissions in the script.
Now, with this setup, absolutely every message goes to /tmp/logall/current and only stays there until the computer is turned off. I also log important messages normally, such as all kernel messages except debugging messages and all critical messages, so I don't lose anything important. This way, I can always find out what the system logs are trying to tell me if I want to know, but I don't pay for that privilege with tons of little writes to my hard drive.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Using --async should have pretty much the same performance effect, but logs would still end up on disk eventually, unless the computer crashes before flushing its logs.
In recent metalog versions, there is a flush option for individual logs, so you can have some logs flushed after each message, and others not. sysklogd provides a similar feature.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've come up with a way I've sped up the majority of metalog logs on my system, and I decided to share it with others. It involves using a ramdisk to store the unimportant logs. A ramdisk, for those who don't know, means a section of RAM that you store files to. This is useful because it avoids writing data to your hard drive, which is much slower than RAM, and the files are automatically gone when you turn off your computer (which IS sometimes an advantage).
The first thing you do is create a ramdisk where the logs will go; I already had one at /tmp so I'll use that as an example. You can create a ramdisk at /tmp, with new (2.6) kernel versions at least, by putting something like this is fstab:
none /tmp tmpfs defaults 0 0
Then you tell metalog to log absolutely everything to a folder in there, such as /tmp/logall. You should also restrict the size of it and the max # of files that get saved to to prevent your ramdisk from filling up. The section in my metalog.conf is:
Everything :
facility = "*"
minimum = 7
maxsize = 1000000
maxfiles = 1
logdir = "/tmp/logall"
Finally, I like to allow my regular user to have permission to read these logs. Since /tmp is deleted on shutdown, to do this you have to create the /tmp/logall directory at startup (in whatever startup script you use to run commands at startup) and give it the proper permissions in the script.
Now, with this setup, absolutely every message goes to /tmp/logall/current and only stays there until the computer is turned off. I also log important messages normally, such as all kernel messages except debugging messages and all critical messages, so I don't lose anything important. This way, I can always find out what the system logs are trying to tell me if I want to know, but I don't pay for that privilege with tons of little writes to my hard drive.
nice trick indeed. Thanks.
Using --async should have pretty much the same performance effect, but logs would still end up on disk eventually, unless the computer crashes before flushing its logs.
In recent metalog versions, there is a flush option for individual logs, so you can have some logs flushed after each message, and others not. sysklogd provides a similar feature.