Menu
▾
▴
Re: [Moosefs-users] Why is it increasing my DEL_LIMIT when I don't want it to!
|
From: WK <wk...@bn...> - 2011-09-23 01:07:10
|
Well I suppose it would be easy enough to grep the source code for 'DEL_LIMIT temporary increase' and start commenting some things out for a quick fix. However, I'd prefer if the maintainers addressed the issue with something more comprehensive and/or a flag for strict enforcement of the DEL_LIMIT setting or the current setup which is obviously some sort of 'oh no, we have a LOT of files we need to work through and need more resources' logic. There may also be some sort of reason they do this, such as some resource issue. We will see what they have to say. Maybe its already addressed in the next version. -bill On 9/22/2011 2:42 AM, Ólafur Ósvaldsson wrote: > We have the exact same problem, chunk deletions have caused problems in the past and we have DEL_LIMIT set at 5, but mfsmaster increases it to 40-50 right away and sometimes goes to 70/s > > This is also the case for chunk replications, it does not seem to honor the mfsmaster.cfg settings, although that does not get logged. > > /Oli > > On 22.9.2011, at 05:03, wk...@bn... wrote: > >> Ok, we deleted a couple hundred thousand files from a large Maildir >> folder set. >> >> We have had problems with deletions overwhelming the cluster in the >> past, so we have a DEL_LIMIT set to 20 (which we will probably lower) >> >> But when the expiretime hit, the server became lethargic. in checking >> the logs I see this >> >> Sep 21 21:10:31 mfs1master mfsmaster[2373]: DEL_LIMIT temporary >> increased to: 26/s >> Sep 21 21:15:30 mfs1master mfsmaster[2373]: DEL_LIMIT temporary >> increased to: 33/s >> Sep 21 21:55:24 mfs1master mfsmaster[2373]: DEL_LIMIT decreased back to: >> 26/s >> >> OK, WHY IS DOING THIS! I told it no more than 20 >> >> I do NOT want this, it kills my server and we've learned the hard way it >> can really screw up VM images (they go read-only) if the deletions >> overwhelm the cluster. >> >> >> -bill >> >> >> >> ------------------------------------------------------------------------------ >> All the data continuously generated in your IT infrastructure contains a >> definitive record of customers, application performance, security >> threats, fraudulent activity and more. Splunk takes this data and makes >> sense of it. Business sense. IT sense. Common sense. >> http://p.sf.net/sfu/splunk-d2dcopy1 >> _______________________________________________ >> moosefs-users mailing list >> moo...@li... >> https://lists.sourceforge.net/lists/listinfo/moosefs-users > -- > Ólafur Osvaldsson > System Administrator > Nethonnun ehf. > e-mail: osv...@ne... > phone: +354 517 3400 > > > ------------------------------------------------------------------------------ > All the data continuously generated in your IT infrastructure contains a > definitive record of customers, application performance, security > threats, fraudulent activity and more. Splunk takes this data and makes > sense of it. Business sense. IT sense. Common sense. > http://p.sf.net/sfu/splunk-d2dcopy1 > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users |
