Re: [Perfparse-users] Tables cleanup

[Ciro I. <cy...@gm...>]

2007/1/28, Florian Gleixner <fl...@bi...>:
> Ciro Iriarte wrote:
> > 2007/1/22, Florian Gleixner <fl...@bi...>:
> >> Hi,
> >>
> >> the --no_transactions switch was made for people that have no innodb
> >> table type. You can try. If you disable inserts, transactions are not
> >> needed. If you did not cleanup your tables for a long time (or never) i
> >> would set the deletion policies only for one host, run a purge, add
> >> another host, run purge ...
> >> If you cannot clean a host at once try to make your delete policy longer
> >> (100 days or more) first. I had these problems first too, but since i
> >> clean every hour, problems are gone.
> >>
> >> Flo
> > Ok, got it running with --no_transactions, but it has been running for
> > 3 days!!, this can't be right.... The DB runs on Xeon servers (HP
> > Proliant), maybe there's a place to tweak the process?, maybe mysql?,
> > it's going, and going, and going, and going.....
> >
>
> The bigger the database the longer a cleanup takes. You could make it
> "incremental" if you first use large values for the deletion policies
> (example 100 days) and then reduce this value at every cleanup step.
> Database server tuning can help - but in this case large delete
> operations will take some time.
> Another idea could be to make it the other way round: If you want to
> delete 90% and keep 10% of the data, then it could be better to copy the
> data you want to keep in a new database. This requires some sql
> knowledge and some understanding of the perfparse table schema.
>
> Flo
>
Well, after 2 weeks running i had to kill the process, i will take the
incremental approach and if that fails i'll try to extract only the
needed data and probably skip the RAW data insertion completely in the
future (didn't find good use for it yet)

Thanks a lot

Ciro