From: Victor B. <vb...@gm...> - 2013-10-17 05:56:46
|
Paul has suggested that we upgrade server to a new version of the OS / kernel due to a kernel bug that is causing the hangs (is this proven?). Last year (25-OCT-2012), I've asked whether we should upgrade to Ubuntu precise to John, Damien and Roland and the agreement was not to do it. Paul, did you want to do something else on the server? Are others aware of any other work that we need to get done on the server? The other thing to discuss is the timing. I would rather if we stay focused for the time being on getting 1.3.0a1 out of the door. I don't want us distracted with environmental issues that are not critical causing us downtime, creating work, etc. Here are the reasons: *** ROLAND *** I see no need to upgrade at the moment. The system is running quite stable . The version we are currently running (Ubuntu 10.04 LTS) is supported until April 2015. "Precise" is the new LTS version with support until April 2017 https://wiki.ubuntu.com/LTS We have 768MB memory available, during normal operation about 100-200MB free. I didn't check, but most of the time newer versions have a larger memory footprint. Maybe we would get worse performance caused by swapping. I won't say "Never change a running system" but "Change running systems only if needed" *** JOHN REESE *** As upgrading the server to 12.04 would likely require downtime, as well as possibly cause a problem that would entail even more downtime in cases where the dist-upgrade fails, I would agree with Roland in saying no. If there is a significant benefit to the newer release, and we do decide to run a dist-upgrade, I would just like to remind us that putting the site into a read-only mode beforehand would be a good precaution, as would running a manual Linode backup. If something does fail during the upgrade, reverting to the latest Linode snapshot would be the best bet. *** DAMIEN REGAD *** Silence :) but he agreed based on follow up discussion. Thoughts? Thanks, -Victor |
From: John R. <jo...@no...> - 2013-10-17 06:31:13
|
I still stand behind my advice, but if doing this might solve some of the hang issues, it could be worth a shot. One option for testing the upgrade without affecting the site could be to use a backup snap shot to create clone, test the dist-upgrade, and then either swap out the old server with the new one, or then do the upgrade on the old one instead. On Oct 16, 2013 10:57 PM, "Victor Boctor" <vb...@gm...> wrote: > Paul has suggested that we upgrade server to a new version of the OS / > kernel due to a kernel bug that is causing the hangs (is this proven?). > Last year (25-OCT-2012), I've asked whether we should upgrade to Ubuntu > precise to John, Damien and Roland and the agreement was not to do it. > > Paul, did you want to do something else on the server? > > Are others aware of any other work that we need to get done on the server? > > The other thing to discuss is the timing. I would rather if we stay > focused for the time being on getting 1.3.0a1 out of the door. I don't > want us distracted with environmental issues that are not critical causing > us downtime, creating work, etc. > > Here are the reasons: > > *** ROLAND *** > > I see no need to upgrade at the moment. > The system is running quite stable . > > The version we are currently running (Ubuntu 10.04 LTS) is supported until > April > 2015. > "Precise" is the new LTS version with support until April 2017 > https://wiki.ubuntu.com/LTS > > We have 768MB memory available, during normal operation about 100-200MB > free. > I didn't check, but most of the time newer versions have a larger memory > footprint. > Maybe we would get worse performance caused by swapping. > > I won't say "Never change a running system" but "Change running systems > only if > needed" > > *** JOHN REESE *** > > As upgrading the server to 12.04 would likely require downtime, as > well as possibly cause a problem that would entail even more downtime > in cases where the dist-upgrade fails, I would agree with Roland in > saying no. If there is a significant benefit to the newer release, > and we do decide to run a dist-upgrade, I would just like to remind us > that putting the site into a read-only mode beforehand would be a good > precaution, as would running a manual Linode backup. If something > does fail during the upgrade, reverting to the latest Linode snapshot > would be the best bet. > > *** DAMIEN REGAD *** > > Silence :) but he agreed based on follow up discussion. > > Thoughts? > > Thanks, > -Victor > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk > _______________________________________________ > mantisbt-dev mailing list > man...@li... > https://lists.sourceforge.net/lists/listinfo/mantisbt-dev > > |
From: Robert M. <rob...@gm...> - 2013-10-17 07:18:58
|
+1 on not upgrading yet, let's keep our energy focused on 1.3.x. One suggestion, based on my recent experience, is to move server configuration to something like puppet or chef ; I've done it for a couple of my servers and it makes building , rebuilding and replicating servers extremely easy. Robert On Thu, Oct 17, 2013 at 9:01 AM, John Reese <jo...@no...> wrote: > I still stand behind my advice, but if doing this might solve some of the > hang issues, it could be worth a shot. One option for testing the upgrade > without affecting the site could be to use a backup snap shot to create > clone, test the dist-upgrade, and then either swap out the old server with > the new one, or then do the upgrade on the old one instead. > > On Oct 16, 2013 10:57 PM, "Victor Boctor" <vb...@gm...> wrote: >> >> Paul has suggested that we upgrade server to a new version of the OS / >> kernel due to a kernel bug that is causing the hangs (is this proven?). >> Last year (25-OCT-2012), I've asked whether we should upgrade to Ubuntu >> precise to John, Damien and Roland and the agreement was not to do it. >> >> Paul, did you want to do something else on the server? >> >> Are others aware of any other work that we need to get done on the server? >> >> The other thing to discuss is the timing. I would rather if we stay >> focused for the time being on getting 1.3.0a1 out of the door. I don't want >> us distracted with environmental issues that are not critical causing us >> downtime, creating work, etc. >> >> Here are the reasons: >> >> ** ROLAND ** >> >> I see no need to upgrade at the moment. >> The system is running quite stable . >> >> The version we are currently running (Ubuntu 10.04 LTS) is supported until >> April >> 2015. >> "Precise" is the new LTS version with support until April 2017 >> https://wiki.ubuntu.com/LTS >> >> We have 768MB memory available, during normal operation about 100-200MB >> free. >> I didn't check, but most of the time newer versions have a larger memory >> footprint. >> Maybe we would get worse performance caused by swapping. >> >> I won't say "Never change a running system" but "Change running systems >> only if >> needed" >> >> ** JOHN REESE ** >> >> As upgrading the server to 12.04 would likely require downtime, as >> well as possibly cause a problem that would entail even more downtime >> in cases where the dist-upgrade fails, I would agree with Roland in >> saying no. If there is a significant benefit to the newer release, >> and we do decide to run a dist-upgrade, I would just like to remind us >> that putting the site into a read-only mode beforehand would be a good >> precaution, as would running a manual Linode backup. If something >> does fail during the upgrade, reverting to the latest Linode snapshot >> would be the best bet. >> >> ** DAMIEN REGAD ** >> >> Silence :) but he agreed based on follow up discussion. >> >> Thoughts? >> >> Thanks, >> -Victor >> >> >> ------------------------------------------------------------------------------ >> October Webinars: Code for Performance >> Free Intel webinars can help you accelerate application performance. >> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most >> from >> the latest Intel processors and coprocessors. See abstracts and register > >> >> http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk >> _______________________________________________ >> mantisbt-dev mailing list >> man...@li... >> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev >> > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk > _______________________________________________ > mantisbt-dev mailing list > man...@li... > https://lists.sourceforge.net/lists/listinfo/mantisbt-dev > -- http://robert.muntea.nu/ |
From: Damien R. <dr...@ma...> - 2013-10-17 07:26:52
|
On 2013-10-17 07:56, Victor Boctor wrote: > Paul has suggested that we upgrade server to a new version of the OS / > kernel due to a kernel bug that is causing the hangs (is this proven?). Not proven, just a suspicion. > Are others aware of any other work that we need to get done on the server? The only thing which I think would be nice to have atm, is a more recent version of Publican (docbook building tool). The one we have in 10.04 (1.0~20090927-2) is really outdated and sometimes causes issues (build warnings or errors) when documentation build scripts run. Precise comes with 2.8-2 which I use on my dev box and works fine. > The other thing to discuss is the timing. I would rather if we stay > focused for the time being on getting 1.3.0a1 out of the door. I don't > want us distracted with environmental issues that are not critical > causing us downtime, creating work, etc.. +1 > *** DAMIEN REGAD *** > > Silence :) but he agreed based on follow up discussion. Let me come out of my silence then ;-) In light of the recent arguments put forward by Paul wrt some xen bug in the kernel being a potential reason for the frequent crashes we're suffering (it is not proven that it's indeed the root cause btw), I think it would be worth a shot to try an upgrade now. I think 12.04LTS/precise has been out long enough that major issues would have been sorted by now. That said, except for the publican issue I mentioned above, there are no major immediate benefits that I can see, other than running generally more recent software - which can also be a shortcoming as we could be introducing new issues ;-). With regards to the upgrade process, I do not recommend upgrading in-place, due to the long potential downtime and risk of failure requiring restore. As I was saying in some other e-mail, I think we should instead work on a copy of mantisbt.org as a test platform. That means running 2 servers concurrently (assuming Linode allows us to do that), the live site and a test upgrade one. If the upgrade proves successful, then we can bring mantisbt.org offline, copy the databases over to the upgraded box then switch the images. Damien |
From: Paul R. <pa...@ma...> - 2013-10-17 08:05:48
|
at work atm but: http://lists.xen.org/archives/html/xen-devel/2011-09/msg01461.html On Thu, Oct 17, 2013 at 8:26 AM, Damien Regad <dr...@ma...> wrote: > On 2013-10-17 07:56, Victor Boctor wrote: > > Paul has suggested that we upgrade server to a new version of the OS / > > kernel due to a kernel bug that is causing the hangs (is this proven?). > > Not proven, just a suspicion. > > > Are others aware of any other work that we need to get done on the > server? > > The only thing which I think would be nice to have atm, is a more recent > version of Publican (docbook building tool). The one we have in 10.04 > (1.0~20090927-2) is really outdated and sometimes causes issues (build > warnings or errors) when documentation build scripts run. Precise comes > with 2.8-2 which I use on my dev box and works fine. > > > The other thing to discuss is the timing. I would rather if we stay > > focused for the time being on getting 1.3.0a1 out of the door. I don't > > want us distracted with environmental issues that are not critical > > causing us downtime, creating work, etc.. > > +1 > > > *** DAMIEN REGAD *** > > > > Silence :) but he agreed based on follow up discussion. > > Let me come out of my silence then ;-) > > In light of the recent arguments put forward by Paul wrt some xen bug in > the kernel being a potential reason for the frequent crashes we're > suffering (it is not proven that it's indeed the root cause btw), I > think it would be worth a shot to try an upgrade now. I think > 12.04LTS/precise has been out long enough that major issues would have > been sorted by now. > > That said, except for the publican issue I mentioned above, there are no > major immediate benefits that I can see, other than running generally > more recent software - which can also be a shortcoming as we could be > introducing new issues ;-). > > With regards to the upgrade process, I do not recommend upgrading > in-place, due to the long potential downtime and risk of failure > requiring restore. > > As I was saying in some other e-mail, I think we should instead work on > a copy of mantisbt.org as a test platform. That means running 2 servers > concurrently (assuming Linode allows us to do that), the live site and a > test upgrade one. If the upgrade proves successful, then we can bring > mantisbt.org offline, copy the databases over to the upgraded box then > switch the images. > > Damien > > > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk > _______________________________________________ > mantisbt-dev mailing list > man...@li... > https://lists.sourceforge.net/lists/listinfo/mantisbt-dev > |
From: Paul R. <pa...@ma...> - 2013-10-24 21:14:02
|
www.mantisbt.org/bugs seems to be responding slowly - do we have plans to try and fix this? It does't look good to end users when it takes >20 seconds to render our own bug tracker. Paul On Thu, Oct 17, 2013 at 9:05 AM, Paul Richards <pa...@ma...> wrote: > at work atm but: > http://lists.xen.org/archives/html/xen-devel/2011-09/msg01461.html > > > On Thu, Oct 17, 2013 at 8:26 AM, Damien Regad <dr...@ma...> wrote: > >> On 2013-10-17 07:56, Victor Boctor wrote: >> > Paul has suggested that we upgrade server to a new version of the OS / >> > kernel due to a kernel bug that is causing the hangs (is this proven?). >> >> Not proven, just a suspicion. >> >> > Are others aware of any other work that we need to get done on the >> server? >> >> The only thing which I think would be nice to have atm, is a more recent >> version of Publican (docbook building tool). The one we have in 10.04 >> (1.0~20090927-2) is really outdated and sometimes causes issues (build >> warnings or errors) when documentation build scripts run. Precise comes >> with 2.8-2 which I use on my dev box and works fine. >> >> > The other thing to discuss is the timing. I would rather if we stay >> > focused for the time being on getting 1.3.0a1 out of the door. I don't >> > want us distracted with environmental issues that are not critical >> > causing us downtime, creating work, etc.. >> >> +1 >> >> > *** DAMIEN REGAD *** >> > >> > Silence :) but he agreed based on follow up discussion. >> >> Let me come out of my silence then ;-) >> >> In light of the recent arguments put forward by Paul wrt some xen bug in >> the kernel being a potential reason for the frequent crashes we're >> suffering (it is not proven that it's indeed the root cause btw), I >> think it would be worth a shot to try an upgrade now. I think >> 12.04LTS/precise has been out long enough that major issues would have >> been sorted by now. >> >> That said, except for the publican issue I mentioned above, there are no >> major immediate benefits that I can see, other than running generally >> more recent software - which can also be a shortcoming as we could be >> introducing new issues ;-). >> >> With regards to the upgrade process, I do not recommend upgrading >> in-place, due to the long potential downtime and risk of failure >> requiring restore. >> >> As I was saying in some other e-mail, I think we should instead work on >> a copy of mantisbt.org as a test platform. That means running 2 servers >> concurrently (assuming Linode allows us to do that), the live site and a >> test upgrade one. If the upgrade proves successful, then we can bring >> mantisbt.org offline, copy the databases over to the upgraded box then >> switch the images. >> >> Damien >> >> >> >> >> ------------------------------------------------------------------------------ >> October Webinars: Code for Performance >> Free Intel webinars can help you accelerate application performance. >> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most >> from >> the latest Intel processors and coprocessors. See abstracts and register > >> >> http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk >> _______________________________________________ >> mantisbt-dev mailing list >> man...@li... >> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev >> > > |
From: Roland B. <ro...@at...> - 2013-10-24 21:22:53
|
> it takes >20 seconds to render our own bug tracker. I am not able to reproduce it e.g. Main page < 0.1 Sec, My View < 0.2 Sec, Roadmap < 0.6 Sec > Paul Richards <pa...@ma...> hat am 24. Oktober 2013 um 23:13 > geschrieben: > > > www.mantisbt.org/bugs seems to be responding slowly - do we have plans to > try and fix this? > > It does't look good to end users when it takes >20 seconds to render our > own bug tracker. > > Paul > > > On Thu, Oct 17, 2013 at 9:05 AM, Paul Richards <pa...@ma...> wrote: > > > at work atm but: > > http://lists.xen.org/archives/html/xen-devel/2011-09/msg01461.html > > > > > > On Thu, Oct 17, 2013 at 8:26 AM, Damien Regad <dr...@ma...> wrote: > > > >> On 2013-10-17 07:56, Victor Boctor wrote: > >> > Paul has suggested that we upgrade server to a new version of the OS / > >> > kernel due to a kernel bug that is causing the hangs (is this proven?). > >> > >> Not proven, just a suspicion. > >> > >> > Are others aware of any other work that we need to get done on the > >> server? > >> > >> The only thing which I think would be nice to have atm, is a more recent > >> version of Publican (docbook building tool). The one we have in 10.04 > >> (1.0~20090927-2) is really outdated and sometimes causes issues (build > >> warnings or errors) when documentation build scripts run. Precise comes > >> with 2.8-2 which I use on my dev box and works fine. > >> > >> > The other thing to discuss is the timing. I would rather if we stay > >> > focused for the time being on getting 1.3.0a1 out of the door. I don't > >> > want us distracted with environmental issues that are not critical > >> > causing us downtime, creating work, etc.. > >> > >> +1 > >> > >> > *** DAMIEN REGAD *** > >> > > >> > Silence :) but he agreed based on follow up discussion. > >> > >> Let me come out of my silence then ;-) > >> > >> In light of the recent arguments put forward by Paul wrt some xen bug in > >> the kernel being a potential reason for the frequent crashes we're > >> suffering (it is not proven that it's indeed the root cause btw), I > >> think it would be worth a shot to try an upgrade now. I think > >> 12.04LTS/precise has been out long enough that major issues would have > >> been sorted by now. > >> > >> That said, except for the publican issue I mentioned above, there are no > >> major immediate benefits that I can see, other than running generally > >> more recent software - which can also be a shortcoming as we could be > >> introducing new issues ;-). > >> > >> With regards to the upgrade process, I do not recommend upgrading > >> in-place, due to the long potential downtime and risk of failure > >> requiring restore. > >> > >> As I was saying in some other e-mail, I think we should instead work on > >> a copy of mantisbt.org as a test platform. That means running 2 servers > >> concurrently (assuming Linode allows us to do that), the live site and a > >> test upgrade one. If the upgrade proves successful, then we can bring > >> mantisbt.org offline, copy the databases over to the upgraded box then > >> switch the images. > >> > >> Damien > >> > >> > >> > >> > >> ------------------------------------------------------------------------------ > >> October Webinars: Code for Performance > >> Free Intel webinars can help you accelerate application performance. > >> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > >> from > >> the latest Intel processors and coprocessors. See abstracts and register > > >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk > >> _______________________________________________ > >> mantisbt-dev mailing list > >> man...@li... > >> https://lists.sourceforge.net/lists/listinfo/mantisbt-dev > >> > > > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk_______________________________________________ > mantisbt-dev mailing list > man...@li... > https://lists.sourceforge.net/lists/listinfo/mantisbt-dev |
From: Damien R. <dr...@ma...> - 2014-01-23 17:12:22
|
On 2013-10-17 10:05, Paul Richards wrote: > On Thu, Oct 17, 2013 at 8:26 AM, Damien Regad wrote: > > On 2013-10-17 07:56, Victor Boctor wrote: > > Paul has suggested that we upgrade server to a new version of the > OS / > > kernel due to a kernel bug that is causing the hangs (is this > proven?).. > > at work atm but: > http://lists.xen.org/archives/html/xen-devel/2011-09/msg01461.html Going on 7 days without a server crash following last week's Kernel upgrade (compare that to 1-2 reboots per week on average before), looking good :-) Keep your fingers crossed ;-) D |
From: Victor B. <vb...@gm...> - 2014-01-23 17:33:16
|
Yep, so far so good! Great :) On Thu, Jan 23, 2014 at 9:12 AM, Damien Regad <dr...@ma...> wrote: > On 2013-10-17 10:05, Paul Richards wrote: > > On Thu, Oct 17, 2013 at 8:26 AM, Damien Regad wrote: > > > > On 2013-10-17 07:56, Victor Boctor wrote: > > > Paul has suggested that we upgrade server to a new version of the > > OS / > > > kernel due to a kernel bug that is causing the hangs (is this > > proven?).. > > > > at work atm but: > > http://lists.xen.org/archives/html/xen-devel/2011-09/msg01461.html > > Going on 7 days without a server crash following last week's Kernel > upgrade (compare that to 1-2 reboots per week on average before), > looking good :-) > > Keep your fingers crossed ;-) > D > > > > > ------------------------------------------------------------------------------ > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > Learn Why More Businesses Are Choosing CenturyLink Cloud For > Critical Workloads, Development Environments & Everything In Between. > Get a Quote or Start a Free Trial Today. > > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk > _______________________________________________ > mantisbt-dev mailing list > man...@li... > https://lists.sourceforge.net/lists/listinfo/mantisbt-dev > |