Re: [ww-users] cleaning out old directories

[Tallulah A. <tal...@gm...>]

Perhaps having button to delete old/moved projects that would mark them for
clean out. This could potentially be combined into a delete wiki page
function particularly for when pages are moved or renamed that would erase
the page and clear out all background jobs and projects associated with the
page.

I think a Do-Not-Erase is sufficient for preserving slow projects.

Tallulah

On Mon, Mar 28, 2011 at 11:24 PM, Lee Worden <wor...@gm...> wrote:

> I have some open questions about WorkingWiki's features, where I would
> appreciate your perspective as people who use WW, or even run your own
> WW sites.  Please feel free to ignore this if you're busy or not
> interested.
>
> For quite a while, we've had a need to clean out old data that piles up
> and isn't in use - not in the wiki pages themselves, but in the working
> directories that we use behind the scenes to compute the output of
> latex, R, etc.
>
> Most obvious is the preview sessions - any time you preview a page that
> includes WW data while editing, it makes a copy of the data in the back
> end to keep it separate from the unedited page's data.  When you save,
> it gets rid of the copied files by merging with the saved ones, but if
> you abandon the changes without saving (a completely reasonable thing to
> do) the copy is left sitting there, and needs to be cleaned out sometime
> later.  These can be quite large - we've seen project directories that
> take up 4GB or even more.  That cleanout has been on my to do list, and
> now it's getting done.
>
> More controversially, there are also old projects that eventually need
> to disappear.  For instance, if someone creates a project and then
> change its name, the old working directory just sits there abandoned.
> Or if a page once had some WW files on it and now it doesn't, the
> project directory is abandoned.  But also if I use the inline latex
> features to add something like $$\alpha + \beta$$ to a page, then change
> it to something else, a project is created to process that latex code,
> and it needs to not be kept forever.  So at some point project
> directories need to be cleared away or the disk will eventually fill up
> with files that no one wants.  I'll probably do this by erasing things
> that haven't been touched in over 3 months or something.
>
> Generally, this should be harmless even if I erase files that someone is
> using, because they can be remade from the source files - it will just
> mean waiting a minute or two (maybe more...) for them to be made.
> Unfortunately,  in the worst case it could mean erasing a directory full
> of output files that can't be easily recreated.  I could implement a "Do
> Not Erase" feature to mark particular projects that are sensitive and
> should never be erased.
>
> My first question: Is there a better way to protect project files that
> should be permanent?  Does anyone have strong feelings about all this?
>
> Finally, I think background jobs should generally be left alone for as
> long as it takes for people to decide whether to erase them.  But there
> is a slight danger: suppose I create a project and run a background job,
> then erase or rename the project.  The background job becomes orphaned,
> and it won't show up in any listings.  So I should probably do something
> to erase things like that.
>
> One way to address both of these things is to check whether each project
> is actually connected to current pages in one of the wikis - that would
> clear up whether it's orphaned or not.  But the directory cleaning is
> done in a separate back-end component ("ProjectEngine") when requested
> by the front end ("WorkingWiki"), and I'm trying to avoid two-way
> communication where ProjectEngine has to ask questions of WorkingWiki
> while it's completing a request, so I'm looking for an alternative...
>
> Lee
>
>
> ------------------------------------------------------------------------------
> Enable your software for Intel(R) Active Management Technology to meet the
> growing manageability and security demands of your customers. Businesses
> are taking advantage of Intel(R) vPro (TM) technology - will your software
> be a part of the solution? Download the Intel(R) Manageability Checker
> today! http://p.sf.net/sfu/intel-dev2devmar
> _______________________________________________
> workingwiki-users mailing list
> wor...@li...
> https://lists.sourceforge.net/lists/listinfo/workingwiki-users
>