Re: [Moosefs-users] Large scale deletions

[Michal B. <mic...@ge...>]

Hi Thomas!

 

Do you use ext3fs? Deleting files on this system is on its own very very
slow. And we assume there is also some mechanism blocking other operations
on disk while deleting built into kernel. These are things we cannot do
anything about it.

 

On the MooseFS side there are really some deletion limits which depend on
the number of chunks necessary to delete. We'll think of introducing some
option to configure this top limit.

 

 

Regards

-Michal

 

From: Thomas S Hatch [mailto:tha...@gm...] 
Sent: Tuesday, March 01, 2011 12:46 AM
To: moosefs-users
Subject: [Moosefs-users] Large scale deletions

 

I have run into a problem with Moosefs when I delete a LOT of files at once,
in my environment we add hundreds of thousands of files, process them into
smaller encoded files, and then delete the originals, when we delete the
originals the deletion process can lock up our writes, and dramatically
slows down our moosefs.

 

Is there a way to make large scale deletions behave a little more nicely?

 

-Thomas  S Hatch