Menu
▾
▴
fofredux-devel
|
From: Miles B. <mil...@gm...> - 2007-01-31 17:13:34
|
Hello, I've run into a problem using FoFRedux on a shared host. A few times I have had my account suspended briefly due to excessive cpu utilization. The error that I get is that the account had used more than 20% of the cpu and the account has briefly been suspended. Is there a way to still have FoFREdux work but limit the cpu utilization? I have 48 feeds that are checked once per hour. Thanks, -Miles |
|
From: Miles B. <mil...@gm...> - 2007-01-31 17:45:28
|
Here is the log of the cpu utlization. Is there a way to find out which feed is taking so long? -------------------------------------------------------------------------------- CPU Exceeded Log For Thu Oct 26 11:30:55 2006 Thu Oct 26 11:30:30 2006: used 0.19 seconds of cpu time for HTTP Request: : GET /?feed=rss2 HTTP/1.0 Thu Oct 26 11:30:50 2006: used 41.00 seconds of cpu time for [[mysql query]] -------------------------------------------------------------------------------- CPU Exceeded Log For Thu Oct 26 11:31:16 2006 Thu Oct 26 11:30:30 2006: used 0.19 seconds of cpu time for HTTP Request: : GET /?feed=rss2 HTTP/1.0 Thu Oct 26 11:30:50 2006: used 41.00 seconds of cpu time for [[mysql query]] -------------------------------------------------------------------------------- CPU Exceeded Log For Thu Oct 26 11:31:32 2006 Thu Oct 26 11:30:30 2006: used 0.19 seconds of cpu time for HTTP Request: : GET /?feed=rss2 HTTP/1.0 Thu Oct 26 11:30:50 2006: used 41.00 seconds of cpu time for [[mysql query]] On 1/31/07, Evan Roth <eva...@gm...> wrote: > i would start by checking the feeds more intermittently. > you can set a cron for each feed to: > http://yourhost/fofinstall/update.php?feed=XX > (see the link if you mouseover the u (for update) in the frameview) Thanks for the suggestion on spacing out the cron jobs. But having 48 seperate cron jobs kinda is overkill. I did notice that when you click on the Edit link for a feed you can set an update interval. I tried setting a feed to a one minute update and it did not seem to work. Using this method (if it worked) I could set up half my feeds to be every 30 minutes, and the other half to be every 60 minutes. Though even this would not solve the problem as it is an excessive mysql query which seems to be the culprit. |
|
From: Andrew T. <ajt...@hi...> - 2007-01-31 17:51:29
|
On 1/31/07, Miles Beck <mil...@gm...> wrote: > Here is the log of the cpu utlization. Is there a way to find out > I did notice that when you click on the Edit link for a feed you can > set an update interval. I tried setting a feed to a one minute update > and it did not seem to work. Using this method (if it worked) I could > set up half my feeds to be every 30 minutes, and the other half to be > every 60 minutes. Though even this would not solve the problem as it > is an excessive mysql query which seems to be the culprit. Yeah, I don't know when the 'update interval' showed up, but I don't believe it's being used currently. The 'slickest'/easiest solution would be to included a PHP script (or just bash script) that would handle spacing out updating your feeds. It would be called by cron every 30 or 60 mins, then do a query to the DB to get, say, the 5 oldest feeds that are due for updating (based on update interval) and update just those, and the next time cron called it, it would get 5 (or whatever) different ones. Sounds like something Kevin could probably do in 10 mins ;) |
|
From: Kevin <ke...@dr...> - 2007-02-01 00:54:09
|
> On 1/31/07, Miles Beck <mil...@gm...> wrote: >> Here is the log of the cpu utlization. Is there a way to find out >> I did notice that when you click on the Edit link for a feed you can >> set an update interval. I tried setting a feed to a one minute update >> and it did not seem to work. Using this method (if it worked) I could >> set up half my feeds to be every 30 minutes, and the other half to be >> every 60 minutes. Though even this would not solve the problem as it >> is an excessive mysql query which seems to be the culprit. > > Yeah, I don't know when the 'update interval' showed up, but I don't > believe it's being used currently. The 'slickest'/easiest solution > would be to included a PHP script (or just bash script) that would > handle spacing out updating your feeds. > > It would be called by cron every 30 or 60 mins, then do a query to the > DB to get, say, the 5 oldest feeds that are due for updating (based on > update interval) and update just those, and the next time cron called > it, it would get 5 (or whatever) different ones. Variable feed updates has been a feature for a while. 1) enable variable feed updates in settings page. 2) define an update interval for all your feeds, based on how fresh you want the feed to be. (default is 60min) 3) run update-quiet.php from cron periodically. (I do it every 30 mins) What it does is only update a feed if the last time it was updated is longer then $UPDATE_INTERVAL for that feed. So every time update-quiet.php is run, only a small number of feeds are updated. For example: A feed like a web comic that only changes once a day, can have an update interval set much higher (several hours) compared to a chatty blog or news website. The user can adjust the update intervals to their own liking. --=20 Kevin |
|
From: Miles B. <mil...@gm...> - 2007-01-31 18:41:03
|
This looks like the time I had set my feeds to be updated: Wed Jan 31 09:59:10 2007: used 0.34 seconds of cpu time for HTTP Request: : GET /?feed=rss2 HTTP/1.1 The others are from the blog that is hosted on the machine that my wife writes. Looks like the rss feeds to the blog articles took a long time for some reason. On 1/31/07, Evan Roth <eva...@gm...> wrote: > are you positive the problem is occurring with an update of feeds...and not > simply a get, when you are reading? > > you could also try to put your update at a weird time, like 19 minutes after > the hour. i could imagine that lots of people have crons running on the > hour...which would possibly add up to a cumulative delay for all the jobs. That's a good idea. I'll try that as well and monitor the cpu_exceeded logs to see when/if this occurs again. |
|
From: Miles B. <mil...@gm...> - 2007-01-31 21:30:17
|
I do have access to phpmysladmin. I did an optimization and here are the differences. The total table size went from 14.5 MiB to 13.2 MiB. The overhead went from 736.3 KiB to 0 KiB. -Miles On 1/31/07, Evan Roth <eva...@gm...> wrote: > if you have phpmysqladmin access to your database, try to optimize all of > the tables. with 48 feeds, i could imagine that the items table could be > quite fragmented, if you are not reading often and deleting stuff as it gets > added. perhaps your provider would let you see the SQL that is taking so long on > the server, to allow us to optimize. we could also take a deeper look at > the db itself...perhaps we should add some indexing somewhere or something. > or at least it tells us exactly what is causing the problem. |
|
From: Miles B. <mil...@gm...> - 2007-02-01 02:15:07
|
On 1/31/07, Kevin <ke...@dr...> wrote: > > I forget, what snapshot are you running? It looks like I have the snapshot right before the index change was made. A lot of the files have a time/date stamp of July of 2006 and a few of the directories have a date of August 3rd. So I could have the changes you made, or I just missed getting the changes. It sounds like Wordpress is having issues. I checked the biggest log and it showed 25 entries that took too long. One which used 42 seconds of cpu time and all the others from .20 to .71 seconds. It looks like the combined GETs that were slow was the culprit. Thanks for the help. -Miles |
Thanks for helping keep SourceForge clean.
X