slashcode-general

[Slashcode-general] Big Problem - Server die everyday

[Jesse L. <jes...@gm...>]

The system dead everyday, There are lots of Disk I/O oprations, We
can't login with SSH and Console when it dead. But when we reset the
system, Everything is back to normal. Can't find where is wrong.

We run slashcode on centos 4.3 with 2.6.9-34.0.2.EL kernel,and mysql
4.1.20, for redhat-linux-gnu (i686) using readline 4.3. and with perl
v5.8.6 built for i686-linux. Slashcode version is Slash T_2_5_0_112.
The hardware run in a vmware ESX 2.5 guest system. with 1G memory and
2G swap.

Please help.

Jesse

Solidot.org

Re: [Slashcode-general] Big Problem - Server die everyday

[shane <sh...@lo...>]

On Aug 23, 2006, at 6:26 AM, Jesse Lee wrote:

> The system dead everyday, There are lots of Disk I/O oprations, We
> can't login with SSH and Console when it dead. But when we reset the
> system, Everything is back to normal. Can't find where is wrong.
>
> We run slashcode on centos 4.3 with 2.6.9-34.0.2.EL kernel,and mysql
> 4.1.20, for redhat-linux-gnu (i686) using readline 4.3. and with perl
> v5.8.6 built for i686-linux. Slashcode version is Slash T_2_5_0_112.
> The hardware run in a vmware ESX 2.5 guest system. with 1G memory and
> 2G swap.
>
> Please help.
>
> Jesse
>
> Solidot.org

Check your processes - see if there's anything system wide that's  
hogging everything.
Check your swap space when "it dies", do you have enough? what's  
swapping?
Use vmstat and see if what's clogging is I/O or CPU that's the  
bottleneck.
Check your disk space, got enough free?
Check your httpd error log files.
Check your /var/log/*
Check your mysql error log.
Under mysql, do a SHOW PROCESSLIST to see what queries are hanging  
(if any).

I would recommend you use mysql5, first off.
How did you setup mysql? Did you read through the mysql docs (ie  
http://mysql.com/doc/refman/5.0/en/index.html ) and read the material  
about tuning, etc?

You can enable your slow query log on mysql.

How did you tune your apache conf? http://www.google.com/search? 
client=safari&rls=en&q=tuning+apache&ie=UTF-8&oe=UTF-8
Read through some of those links, check your MaxClients and  
MaxSpareServers, compared to the amount of ram you have.

Hardware wise, you should be fine for a not-huge site. That's not far  
from our config here (except we always run mysql on it's own machine 
(s)). Since you seem to be running apache and mysql on the same  
machine, you should make sure that your mysql and your apache are  
setup such that you're not running out of ram.

For slash, there's options such as cache enable (see system vars) as  
well as on httpd startup preloading of things (look at your *site's*  
httpd.conf file). That can eat ram. And slashd can be a resource/ram  
pig when it runs, just because it does quite a lot. Look at it's  
processor and ram usage when it's running.

Be sure if you figure it out, to email the list and give details of  
what you found and how you fixed it so the next person who runs into  
such a situation can find out what you did via the listserve archives.

  Shane



--
My slashcode stuff: http://slash.lottadot.com/
Slashcode faq: http://slash.lottadot.com/slash-faq
How to ask a question: http://www.catb.org/~esr/faqs/smart- 
questions.html#before

Re: [Slashcode-general] Big Problem - Server die everyday

[Jesse L. <jes...@gm...>]

We are checking all areas you mentioned. I'll reply with the results later.
Thanks Shane.

Jesse

On 8/23/06, shane <sh...@lo...> wrote:
>
>
>
> On Aug 23, 2006, at 6:26 AM, Jesse Lee wrote:
>
> The system dead everyday, There are lots of Disk I/O oprations, We
> can't login with SSH and Console when it dead. But when we reset the
> system, Everything is back to normal. Can't find where is wrong.
>
> We run slashcode on centos 4.3 with 2.6.9-34.0.2.EL kernel,and mysql
> 4.1.20, for redhat-linux-gnu (i686) using readline 4.3. and with perl
> v5.8.6 built for i686-linux. Slashcode version is Slash T_2_5_0_112.
> The hardware run in a vmware ESX 2.5 guest system. with 1G memory and
> 2G swap.
>
> Please help.
>
> Jesse
>
> Solidot.org
>
> Check your processes - see if there's anything system wide that's hogging
> everything.
> Check your swap space when "it dies", do you have enough? what's swapping?
> Use vmstat and see if what's clogging is I/O or CPU that's the bottleneck.
> Check your disk space, got enough free?
> Check your httpd error log files.
> Check your /var/log/*
> Check your mysql error log.
> Under mysql, do a SHOW PROCESSLIST to see what queries are hanging (if any).
>
> I would recommend you use mysql5, first off.
> How did you setup mysql? Did you read through the mysql docs (ie
> http://mysql.com/doc/refman/5.0/en/index.html ) and read
> the material about tuning, etc?
>
> You can enable your slow query log on mysql.
>
> How did you tune your apache conf?
> http://www.google.com/search?client=safari&rls=en&q=tuning+apache&ie=UTF-8&oe=UTF-8
> Read through some of those links, check your MaxClients and MaxSpareServers,
> compared to the amount of ram you have.
>
> Hardware wise, you should be fine for a not-huge site. That's not far from
> our config here (except we always run mysql on it's own machine(s)). Since
> you seem to be running apache and mysql on the same machine, you should make
> sure that your mysql and your apache are setup such that you're not running
> out of ram.
>
> For slash, there's options such as cache enable (see system vars) as well as
> on httpd startup preloading of things (look at your *site's* httpd.conf
> file). That can eat ram. And slashd can be a resource/ram pig when it runs,
> just because it does quite a lot. Look at it's processor and ram usage when
> it's running.
>
> Be sure if you figure it out, to email the list and give details of what you
> found and how you fixed it so the next person who runs into such a situation
> can find out what you did via the listserve archives.
>
>  Shane
>
>
>
>
> --
> My slashcode stuff: http://slash.lottadot.com/
> Slashcode faq: http://slash.lottadot.com/slash-faq
> How to ask a question:
> http://www.catb.org/~esr/faqs/smart-questions.html#before
>
>
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>
> _______________________________________________
> Slashcode-general mailing list
> Sla...@li...
> https://lists.sourceforge.net/lists/listinfo/slashcode-general
>
>
>

Re: [Slashcode-general] Big Problem - Server die everyday

[Jesse L. <jes...@gm...>]

Hi, shane,

I also send you the SSH info, Please login to our server take a look.

Thanks,

Jesse
On 8/23/06, Jesse Lee <jes...@gm...> wrote:
> We are checking all areas you mentioned. I'll reply with the results later.
> Thanks Shane.
>
> Jesse
>
> On 8/23/06, shane <sh...@lo...> wrote:
> >
> >
> >
> > On Aug 23, 2006, at 6:26 AM, Jesse Lee wrote:
> >
> > The system dead everyday, There are lots of Disk I/O oprations, We
> > can't login with SSH and Console when it dead. But when we reset the
> > system, Everything is back to normal. Can't find where is wrong.
> >
> > We run slashcode on centos 4.3 with 2.6.9-34.0.2.EL kernel,and mysql
> > 4.1.20, for redhat-linux-gnu (i686) using readline 4.3. and with perl
> > v5.8.6 built for i686-linux. Slashcode version is Slash T_2_5_0_112.
> > The hardware run in a vmware ESX 2.5 guest system. with 1G memory and
> > 2G swap.
> >
> > Please help.
> >
> > Jesse
> >
> > Solidot.org
> >
> > Check your processes - see if there's anything system wide that's hogging
> > everything.
> > Check your swap space when "it dies", do you have enough? what's swapping?
> > Use vmstat and see if what's clogging is I/O or CPU that's the bottleneck.
> > Check your disk space, got enough free?
> > Check your httpd error log files.
> > Check your /var/log/*
> > Check your mysql error log.
> > Under mysql, do a SHOW PROCESSLIST to see what queries are hanging (if any).
> >
> > I would recommend you use mysql5, first off.
> > How did you setup mysql? Did you read through the mysql docs (ie
> > http://mysql.com/doc/refman/5.0/en/index.html ) and read
> > the material about tuning, etc?
> >
> > You can enable your slow query log on mysql.
> >
> > How did you tune your apache conf?
> > http://www.google.com/search?client=safari&rls=en&q=tuning+apache&ie=UTF-8&oe=UTF-8
> > Read through some of those links, check your MaxClients and MaxSpareServers,
> > compared to the amount of ram you have.
> >
> > Hardware wise, you should be fine for a not-huge site. That's not far from
> > our config here (except we always run mysql on it's own machine(s)). Since
> > you seem to be running apache and mysql on the same machine, you should make
> > sure that your mysql and your apache are setup such that you're not running
> > out of ram.
> >
> > For slash, there's options such as cache enable (see system vars) as well as
> > on httpd startup preloading of things (look at your *site's* httpd.conf
> > file). That can eat ram. And slashd can be a resource/ram pig when it runs,
> > just because it does quite a lot. Look at it's processor and ram usage when
> > it's running.
> >
> > Be sure if you figure it out, to email the list and give details of what you
> > found and how you fixed it so the next person who runs into such a situation
> > can find out what you did via the listserve archives.
> >
> >  Shane
> >
> >
> >
> >
> > --
> > My slashcode stuff: http://slash.lottadot.com/
> > Slashcode faq: http://slash.lottadot.com/slash-faq
> > How to ask a question:
> > http://www.catb.org/~esr/faqs/smart-questions.html#before
> >
> >
> >
> >
> > -------------------------------------------------------------------------
> > Using Tomcat but need to do more? Need to support web services, security?
> > Get stuff done quickly with pre-integrated technology to make your job
> > easier
> > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> >
> > _______________________________________________
> > Slashcode-general mailing list
> > Sla...@li...
> > https://lists.sourceforge.net/lists/listinfo/slashcode-general
> >
> >
> >
>

Re: [Slashcode-general] Big Problem - Server die everyday

[Jesse L. <jes...@gm...>]

There is a HTML::Parser error take tens of thousouds space in httpd
log, probally 90% of total error logs. Here is the message:
Parsing of undecoded UTF-8 will give garbage when decoding entities at
/usr/local/lib/perl5/site_perl/5.8.6/i686-linux/HTML/PullParser.pm
line 83.

We've looked into this, but our two coder can't solve it.

We are still checking others.


On 8/25/06, shane <sh...@lo...> wrote:
> Sorry, I don't have the time for this at the moment.
>
>   Shane
>
> On Aug 23, 2006, at 9:12 AM, Jesse Lee wrote:
>
> > Hi, shane,
> >
> > Can you login to our server to take a look?
> > We've spent much time but still were confused.
> >
> > Here is the SSH details:
> >
> > hostname: telednat.vmmatrix.com
> > port: 11212
> > username: root
> > passwd: vmmatrix
> >
> > If you find where is not suitable, you can change them directly.
> >
> > Thanks,
> >
> > Jesse
> > http://solidot.org/
> >
> > On 8/23/06, shane <sh...@lo...> wrote:
> >>
> >>
> >>
> >> On Aug 23, 2006, at 6:26 AM, Jesse Lee wrote:
> >>
> >> The system dead everyday, There are lots of Disk I/O oprations, We
> >> can't login with SSH and Console when it dead. But when we reset the
> >> system, Everything is back to normal. Can't find where is wrong.
> >>
> >> We run slashcode on centos 4.3 with 2.6.9-34.0.2.EL kernel,and mysql
> >> 4.1.20, for redhat-linux-gnu (i686) using readline 4.3. and with perl
> >> v5.8.6 built for i686-linux. Slashcode version is Slash T_2_5_0_112.
> >> The hardware run in a vmware ESX 2.5 guest system. with 1G memory and
> >> 2G swap.
> >>
> >> Please help.
> >>
> >> Jesse
> >>
> >> Solidot.org
> >>
> >> Check your processes - see if there's anything system wide that's
> >> hogging
> >> everything.
> >> Check your swap space when "it dies", do you have enough? what's
> >> swapping?
> >> Use vmstat and see if what's clogging is I/O or CPU that's the
> >> bottleneck.
> >> Check your disk space, got enough free?
> >> Check your httpd error log files.
> >> Check your /var/log/*
> >> Check your mysql error log.
> >> Under mysql, do a SHOW PROCESSLIST to see what queries are hanging
> >> (if any).
> >>
> >> I would recommend you use mysql5, first off.
> >> How did you setup mysql? Did you read through the mysql docs (ie
> >> http://mysql.com/doc/refman/5.0/en/index.html ) and read
> >> the material about tuning, etc?
> >>
> >> You can enable your slow query log on mysql.
> >>
> >> How did you tune your apache conf?
> >> http://www.google.com/search?client=safari&rls=en&q=tuning
> >> +apache&ie=UTF-8&oe=UTF-8
> >> Read through some of those links, check your MaxClients and
> >> MaxSpareServers,
> >> compared to the amount of ram you have.
> >>
> >> Hardware wise, you should be fine for a not-huge site. That's not
> >> far from
> >> our config here (except we always run mysql on it's own machine
> >> (s)). Since
> >> you seem to be running apache and mysql on the same machine, you
> >> should make
> >> sure that your mysql and your apache are setup such that you're
> >> not running
> >> out of ram.
> >>
> >> For slash, there's options such as cache enable (see system vars)
> >> as well as
> >> on httpd startup preloading of things (look at your *site's*
> >> httpd.conf
> >> file). That can eat ram. And slashd can be a resource/ram pig when
> >> it runs,
> >> just because it does quite a lot. Look at it's processor and ram
> >> usage when
> >> it's running.
> >>
> >> Be sure if you figure it out, to email the list and give details
> >> of what you
> >> found and how you fixed it so the next person who runs into such a
> >> situation
> >> can find out what you did via the listserve archives.
> >>
> >>  Shane
> >>
> >>
> >>
> >>
> >> --
> >> My slashcode stuff: http://slash.lottadot.com/
> >> Slashcode faq: http://slash.lottadot.com/slash-faq
> >> How to ask a question:
> >> http://www.catb.org/~esr/faqs/smart-questions.html#before
> >>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> ----
> >> Using Tomcat but need to do more? Need to support web services,
> >> security?
> >> Get stuff done quickly with pre-integrated technology to make your
> >> job
> >> easier
> >> Download IBM WebSphere Application Server v.1.0.1 based on Apache
> >> Geronimo
> >> http://sel.as-us.falkag.net/sel?
> >> cmd=lnk&kid=120709&bid=263057&dat=121642
> >>
> >> _______________________________________________
> >> Slashcode-general mailing list
> >> Sla...@li...
> >> https://lists.sourceforge.net/lists/listinfo/slashcode-general
> >>
> >>
> >>
>
>

Re: [Slashcode-general] Big Problem - Server die everyday

[shane <sh...@lo...>]

On Aug 25, 2006, at 1:46 AM, Jesse Lee wrote:

> There is a HTML::Parser error take tens of thousouds space in httpd
> log, probally 90% of total error logs. Here is the message:
> Parsing of undecoded UTF-8 will give garbage when decoding entities at
> /usr/local/lib/perl5/site_perl/5.8.6/i686-linux/HTML/PullParser.pm
> line 83.
>
> We've looked into this, but our two coder can't solve it.
>
> We are still checking others.

Weren't you the one who was modifying slash quite a bit to deal with  
foreign character sets? Maybe that has something to do with the  
errors you are seeing.

BTW, you sent your machine login information to the *listserve*. I  
hope you've changed your password since.

  Shane

Re: [Slashcode-general] Big Problem - Server die everyday

[Jesse L. <jes...@gm...>]

I saw someone has same HTML Parser trouble with Scoop.
http://www.opensubscriber.com/message/sit...@ta.../1921855.html

> On 8/25/06, Jesse Lee <jes...@gm...> wrote:
> > There is a HTML::Parser error take tens of thousouds space in httpd
> > log, probally 90% of total error logs. Here is the message:
> > Parsing of undecoded UTF-8 will give garbage when decoding entities at
> > /usr/local/lib/perl5/site_perl/5.8.6/i686-linux/HTML/PullParser.pm
> > line 83.
> >
> > We've looked into this, but our two coder can't solve it.
> >
> > We are still checking others.
> >
> >
> > On 8/25/06, shane <sh...@lo...> wrote:
> > > Sorry, I don't have the time for this at the moment.
> > >
> > >   Shane
> > >
> > > On Aug 23, 2006, at 9:12 AM, Jesse Lee wrote:
> > >
> > > > Hi, shane,
> > > >
> > > > Can you login to our server to take a look?
> > > > We've spent much time but still were confused.
> > > >
> > > > Here is the SSH details:
> > > >
> > > > hostname: telednat.vmmatrix.com
> > > > port: 11212
> > > > username: root
> > > > passwd: vmmatrix
> > > >
> > > > If you find where is not suitable, you can change them directly.
> > > >
> > > > Thanks,
> > > >
> > > > Jesse
> > > > http://solidot.org/
> > > >
> > > > On 8/23/06, shane <sh...@lo...> wrote:
> > > >>
> > > >>
> > > >>
> > > >> On Aug 23, 2006, at 6:26 AM, Jesse Lee wrote:
> > > >>
> > > >> The system dead everyday, There are lots of Disk I/O oprations, We
> > > >> can't login with SSH and Console when it dead. But when we reset the
> > > >> system, Everything is back to normal. Can't find where is wrong.
> > > >>
> > > >> We run slashcode on centos 4.3 with 2.6.9-34.0.2.EL kernel,and mysql
> > > >> 4.1.20, for redhat-linux-gnu (i686) using readline 4.3. and with perl
> > > >> v5.8.6 built for i686-linux. Slashcode version is Slash T_2_5_0_112.
> > > >> The hardware run in a vmware ESX 2.5 guest system. with 1G memory and
> > > >> 2G swap.
> > > >>
> > > >> Please help.
> > > >>
> > > >> Jesse
> > > >>
> > > >> Solidot.org
> > > >>
> > > >> Check your processes - see if there's anything system wide that's
> > > >> hogging
> > > >> everything.
> > > >> Check your swap space when "it dies", do you have enough? what's
> > > >> swapping?
> > > >> Use vmstat and see if what's clogging is I/O or CPU that's the
> > > >> bottleneck.
> > > >> Check your disk space, got enough free?
> > > >> Check your httpd error log files.
> > > >> Check your /var/log/*
> > > >> Check your mysql error log.
> > > >> Under mysql, do a SHOW PROCESSLIST to see what queries are hanging
> > > >> (if any).
> > > >>
> > > >> I would recommend you use mysql5, first off.
> > > >> How did you setup mysql? Did you read through the mysql docs (ie
> > > >> http://mysql.com/doc/refman/5.0/en/index.html ) and read
> > > >> the material about tuning, etc?
> > > >>
> > > >> You can enable your slow query log on mysql.
> > > >>
> > > >> How did you tune your apache conf?
> > > >> http://www.google.com/search?client=safari&rls=en&q=tuning
> > > >> +apache&ie=UTF-8&oe=UTF-8
> > > >> Read through some of those links, check your MaxClients and
> > > >> MaxSpareServers,
> > > >> compared to the amount of ram you have.
> > > >>
> > > >> Hardware wise, you should be fine for a not-huge site. That's not
> > > >> far from
> > > >> our config here (except we always run mysql on it's own machine
> > > >> (s)). Since
> > > >> you seem to be running apache and mysql on the same machine, you
> > > >> should make
> > > >> sure that your mysql and your apache are setup such that you're
> > > >> not running
> > > >> out of ram.
> > > >>
> > > >> For slash, there's options such as cache enable (see system vars)
> > > >> as well as
> > > >> on httpd startup preloading of things (look at your *site's*
> > > >> httpd.conf
> > > >> file). That can eat ram. And slashd can be a resource/ram pig when
> > > >> it runs,
> > > >> just because it does quite a lot. Look at it's processor and ram
> > > >> usage when
> > > >> it's running.
> > > >>
> > > >> Be sure if you figure it out, to email the list and give details
> > > >> of what you
> > > >> found and how you fixed it so the next person who runs into such a
> > > >> situation
> > > >> can find out what you did via the listserve archives.
> > > >>
> > > >>  Shane
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> My slashcode stuff: http://slash.lottadot.com/
> > > >> Slashcode faq: http://slash.lottadot.com/slash-faq
> > > >> How to ask a question:
> > > >> http://www.catb.org/~esr/faqs/smart-questions.html#before
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> ---------------------------------------------------------------------
> > > >> ----
> > > >> Using Tomcat but need to do more? Need to support web services,
> > > >> security?
> > > >> Get stuff done quickly with pre-integrated technology to make your
> > > >> job
> > > >> easier
> > > >> Download IBM WebSphere Application Server v.1.0.1 based on Apache
> > > >> Geronimo
> > > >> http://sel.as-us.falkag.net/sel?
> > > >> cmd=lnk&kid=120709&bid=263057&dat=121642
> > > >>
> > > >> _______________________________________________
> > > >> Slashcode-general mailing list
> > > >> Sla...@li...
> > > >> https://lists.sourceforge.net/lists/listinfo/slashcode-general
> > > >>
> > > >>
> > > >>
> > >
> > >
> >
>