#1737 new tool: merging data files

open
nobody
None
5
2012-10-11
2008-09-24
Anonymous
No

For heavily used cloud of servers (i.e. 500mb of apache logs per day) the logresolvemerge script doesn't work... you have to download huge logs files and then merge them all together.

Also in such situation I use PurgeLog option to keep my logs in reasonable size.

Better solution would be to process logs on each machine, download processed data and merge into one data file. Currently, when Amazon EC2 service is so popular this would be grat additional tool.

Also - maybe separating functions responsible for handling data file IO into library would help to write contrib tools?

Discussion