treebase-devel — TreeBASE developer mailing list

Re: [Treebase-devel] Treebase Dev problems

[Rutger V. <R....@re...>]

I think we're doing that already.

On Thu, Nov 17, 2011 at 5:14 PM, Shyket, Harry <har...@ya...> wrote:
> What about caching at the Hibernate level?  Possibly using Ehcache?
>
> Harry Shyket
> Digital Media Specialist
> Yale University Peabody Museum
> ph. 203-436-9428
> har...@ya...
>
>
> -----Original Message-----
> From: Rutger Vos [mailto:R....@re...]
> Sent: Thursday, November 17, 2011 11:48 AM
> To: William Piel
> Cc: TreeBASE devel
> Subject: Re: [Treebase-devel] Treebase Dev problems
>
> An alternative to this is that we might consider our own caching: the
> method generateAFileDynamically could be made to check whether there
> is already a pre-computed copy of the file in some directory and
> forward to that (perhaps with some additional flag that we can set if
> we want to re-generate all the files if our nexus or nexml changes).
> This assumes that the biggest performance hit is incurred by the
> generating, not the searching (I'm pretty sure that that's the case,
> though).
>
> On Thu, Nov 17, 2011 at 4:14 PM, William Piel <wil...@ya...> wrote:
>>
>> On Nov 17, 2011, at 10:02 AM, William Piel wrote:
>>
>>> Therefore, if you also know of an Apache plugin that will cache results for "/phylows/study/TB2:", that would greatly help.
>>
>> Looking at mod_cache, I wonder if this would work:
>>
>> cacheRoot c:/cacheroot
>> cacheEnable disk /treebase-web/phylows/study/TB2:
>> cacheDirLevels 1
>> cacheDirLength 20
>> cacheMinFileSize 1
>> cacheMaxFileSize 50000000
>> cacheIgnorecacheControl Off
>> cacheIgnoreNoLastMod On
>> cacheMaxExpire 2592000
>>
>> ... resulting in a one-month cache on all TB2 objects. What's unclear to me is whether the cacheEnable string allows substrings or whether it needs to end in "/".  If that's a limitation, are there third-party plugins that can cache using wildcards?
>>
>> bp
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> All the data continuously generated in your IT infrastructure
>> contains a definitive record of customers, application performance,
>> security threats, fraudulent activity, and more. Splunk takes this
>> data and makes sense of it. IT sense. And common sense.
>> http://p.sf.net/sfu/splunk-novd2d
>> _______________________________________________
>> Treebase-devel mailing list
>> Tre...@li...
>> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>>
>
>
>
> --
> Dr. Rutger A. Vos
> School of Biological Sciences
> Philip Lyle Building, Level 4
> University of Reading
> Reading, RG6 6BX, United Kingdom
> Tel: +44 (0) 118 378 7535
> http://rutgervos.blogspot.com
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure
> contains a definitive record of customers, application performance,
> security threats, fraudulent activity, and more. Splunk takes this
> data and makes sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-novd2d
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure
> contains a definitive record of customers, application performance,
> security threats, fraudulent activity, and more. Splunk takes this
> data and makes sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-novd2d
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>



-- 
Dr. Rutger A. Vos
School of Biological Sciences
Philip Lyle Building, Level 4
University of Reading
Reading, RG6 6BX, United Kingdom
Tel: +44 (0) 118 378 7535
http://rutgervos.blogspot.com

Re: [Treebase-devel] Treebase Dev problems

[Shyket, H. <har...@ya...>]

What about caching at the Hibernate level?  Possibly using Ehcache?

Harry Shyket
Digital Media Specialist
Yale University Peabody Museum
ph. 203-436-9428
har...@ya...


-----Original Message-----
From: Rutger Vos [mailto:R....@re...] 
Sent: Thursday, November 17, 2011 11:48 AM
To: William Piel
Cc: TreeBASE devel
Subject: Re: [Treebase-devel] Treebase Dev problems

An alternative to this is that we might consider our own caching: the
method generateAFileDynamically could be made to check whether there
is already a pre-computed copy of the file in some directory and
forward to that (perhaps with some additional flag that we can set if
we want to re-generate all the files if our nexus or nexml changes).
This assumes that the biggest performance hit is incurred by the
generating, not the searching (I'm pretty sure that that's the case,
though).

On Thu, Nov 17, 2011 at 4:14 PM, William Piel <wil...@ya...> wrote:
>
> On Nov 17, 2011, at 10:02 AM, William Piel wrote:
>
>> Therefore, if you also know of an Apache plugin that will cache results for "/phylows/study/TB2:", that would greatly help.
>
> Looking at mod_cache, I wonder if this would work:
>
> cacheRoot c:/cacheroot
> cacheEnable disk /treebase-web/phylows/study/TB2:
> cacheDirLevels 1
> cacheDirLength 20
> cacheMinFileSize 1
> cacheMaxFileSize 50000000
> cacheIgnorecacheControl Off
> cacheIgnoreNoLastMod On
> cacheMaxExpire 2592000
>
> ... resulting in a one-month cache on all TB2 objects. What's unclear to me is whether the cacheEnable string allows substrings or whether it needs to end in "/".  If that's a limitation, are there third-party plugins that can cache using wildcards?
>
> bp
>
>
>
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure
> contains a definitive record of customers, application performance,
> security threats, fraudulent activity, and more. Splunk takes this
> data and makes sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-novd2d
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>



-- 
Dr. Rutger A. Vos
School of Biological Sciences
Philip Lyle Building, Level 4
University of Reading
Reading, RG6 6BX, United Kingdom
Tel: +44 (0) 118 378 7535
http://rutgervos.blogspot.com

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure 
contains a definitive record of customers, application performance, 
security threats, fraudulent activity, and more. Splunk takes this 
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
Treebase-devel mailing list
Tre...@li...
https://lists.sourceforge.net/lists/listinfo/treebase-devel

Re: [Treebase-devel] Treebase Dev problems

[Rutger V. <R....@re...>]

An alternative to this is that we might consider our own caching: the
method generateAFileDynamically could be made to check whether there
is already a pre-computed copy of the file in some directory and
forward to that (perhaps with some additional flag that we can set if
we want to re-generate all the files if our nexus or nexml changes).
This assumes that the biggest performance hit is incurred by the
generating, not the searching (I'm pretty sure that that's the case,
though).

On Thu, Nov 17, 2011 at 4:14 PM, William Piel <wil...@ya...> wrote:
>
> On Nov 17, 2011, at 10:02 AM, William Piel wrote:
>
>> Therefore, if you also know of an Apache plugin that will cache results for "/phylows/study/TB2:", that would greatly help.
>
> Looking at mod_cache, I wonder if this would work:
>
> cacheRoot c:/cacheroot
> cacheEnable disk /treebase-web/phylows/study/TB2:
> cacheDirLevels 1
> cacheDirLength 20
> cacheMinFileSize 1
> cacheMaxFileSize 50000000
> cacheIgnorecacheControl Off
> cacheIgnoreNoLastMod On
> cacheMaxExpire 2592000
>
> ... resulting in a one-month cache on all TB2 objects. What's unclear to me is whether the cacheEnable string allows substrings or whether it needs to end in "/".  If that's a limitation, are there third-party plugins that can cache using wildcards?
>
> bp
>
>
>
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure
> contains a definitive record of customers, application performance,
> security threats, fraudulent activity, and more. Splunk takes this
> data and makes sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-novd2d
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>



-- 
Dr. Rutger A. Vos
School of Biological Sciences
Philip Lyle Building, Level 4
University of Reading
Reading, RG6 6BX, United Kingdom
Tel: +44 (0) 118 378 7535
http://rutgervos.blogspot.com

Re: [Treebase-devel] Treebase Dev problems

[William P. <wil...@ya...>]

On Nov 17, 2011, at 10:02 AM, William Piel wrote:

> Therefore, if you also know of an Apache plugin that will cache results for "/phylows/study/TB2:", that would greatly help. 

Looking at mod_cache, I wonder if this would work:

cacheRoot c:/cacheroot 
cacheEnable disk /treebase-web/phylows/study/TB2:
cacheDirLevels 1 
cacheDirLength 20 
cacheMinFileSize 1 
cacheMaxFileSize 50000000 
cacheIgnorecacheControl Off 
cacheIgnoreNoLastMod On 
cacheMaxExpire 2592000

... resulting in a one-month cache on all TB2 objects. What's unclear to me is whether the cacheEnable string allows substrings or whether it needs to end in "/".  If that's a limitation, are there third-party plugins that can cache using wildcards? 

bp

Re: [Treebase-devel] Treebase Dev problems

[William P. <wil...@ya...>]

On Nov 17, 2011, at 9:11 AM, Mattison Ward wrote:

> Apache has a module that can limit the number of requests per second to specific URLs. For example, requests to /treebase-web/phylows/ could be limited to 10 requests per second while requests to any other URL could be unlimited.  That would be another option to control overloading by API requests if we can isolate the URLs.

That might do the trick. Unfortunately it's not as easy as simply throttling "purl.org" because purls to objects are used by the web interface as well (to encourage people to cite the canonical URI to objects). 

My sense is that the following can be throttled:

http://purl.org/phylo/treebase/phylows/study/find?query=[...]
http://purl.org/phylo/treebase/phylows/matrix/find?query=[...]
http://purl.org/phylo/treebase/phylows/tree/find?query=[...]
http://purl.org/phylo/treebase/phylows/taxon/find?query=[...]

But the following should not be throttled: 

http://purl.org/phylo/treebase/phylows/study/TB2:[...]
http://purl.org/phylo/treebase/phylows/matrix/TB2:[...]
http://purl.org/phylo/treebase/phylows/tree/TB2:[...]
http://purl.org/phylo/treebase/phylows/taxon/TB2:[...]

Also, the information returned by calls to "/phylows/study/TB2:" do not change very much, so they can be cached. Every once in a while the data returned might get updated, but typically it's always the same. Therefore, if you also know of an Apache plugin that will cache results for "/phylows/study/TB2:", that would greatly help. 

bp

Re: [Treebase-devel] Treebase Dev problems

[Shyket, H. <har...@ya...>]

The exceptions from the logs seem to be thrown already (just making them look a little cleaner for the user).  Do you have a monitor to see what traffic comes in/how much memory is in use?

Harry Shyket
Digital Media Specialist
Yale University Peabody Museum
ph. 203-436-9428
har...@ya...

From: Mattison Ward [mailto:mat...@ne...]
Sent: Thursday, November 17, 2011 9:12 AM
To: William Piel
Cc: TreeBASE devel
Subject: Re: [Treebase-devel] Treebase Dev problems

If/When we have problems today, I will check the logs to see if there are a large number of phylows requests at the time of the problem.

I would be curious to see if fixing the uncaught exceptions takes care of most of the overloading issues.

Apache has a module that can limit the number of requests per second to specific URLs. For example, requests to /treebase-web/phylows/ could be limited to 10 requests per second while requests to any other URL could be unlimited.  That would be another option to control overloading by API requests if we can isolate the URLs.

-Mattison

On Thu, Nov 17, 2011 at 12:01 AM, William Piel <wil...@ya...<mailto:wil...@ya...>> wrote:

Hi all,

So we continue to get hammered by, I think, someone who is making heavy use of the API. Maybe Mattison can better identify the source of this -- but I don't think it's just higher UI traffic or robot activities because Google Analytics is not showing a jump in activity. While logs show some uncaught exceptions (and Harry is fixing some of these) these are probably not actually causing the slowdown. Rather, objects in memory are probably not being disposed of fast enough and the database is clogging up (direct queries against postgresql are also very slow).

We need a solution because when TreeBASE bogs down it becomes too slow to submit data, etc.  In fact, right now (11:50PM) TreeBASE doesn't seem to be responding at all.

Can anyone suggest a solution?

Would this be a possible solution: deploy another VM with Tomcat and another database instance (e.g. take over treebase-stage.nescent.org<http://treebase-stage.nescent.org> for this purpose). The database is refreshed nightly from production; the war is rebuilt whenever production is rebuilt; but the two are sufficiently isolated so that heavy API activity doesn't cripple production. Traffic headed to http:://www.treebase.org<http://www.treebase.org> goes to production. Traffic headed to http://purl.org/phylo/treebase/phylows/ goes to this other instance.

I guess one possible snag is that when a /phylows/ query includes "format=html", it is jumping to the web page environment -- yet we don't want this API machine to serve web pages to people in any way. So we'd want a /phylows/...?format=html to redirect to production instead.

bp








--
Mattison Ward
NESCent at Duke University
2024 W. Main Street, Suite A200
Durham, NC 27705-4667
919-668-4585 (desk)
919-668-4551 (alternate)
919-668-9198 (fax)

Re: [Treebase-devel] Treebase Dev problems

[Mattison W. <mat...@ne...>]

If/When we have problems today, I will check the logs to see if there are a
large number of phylows requests at the time of the problem.

I would be curious to see if fixing the uncaught exceptions takes care of
most of the overloading issues.

Apache has a module that can limit the number of requests per second to
specific URLs. For example, requests to /treebase-web/phylows/ could be
limited to 10 requests per second while requests to any other URL could be
unlimited.  That would be another option to control overloading by API
requests if we can isolate the URLs.

-Mattison

On Thu, Nov 17, 2011 at 12:01 AM, William Piel <wil...@ya...>wrote:

>
> Hi all,
>
> So we continue to get hammered by, I think, someone who is making heavy
> use of the API. Maybe Mattison can better identify the source of this --
> but I don't think it's just higher UI traffic or robot activities because
> Google Analytics is not showing a jump in activity. While logs show some
> uncaught exceptions (and Harry is fixing some of these) these are probably
> not actually causing the slowdown. Rather, objects in memory are probably
> not being disposed of fast enough and the database is clogging up (direct
> queries against postgresql are also very slow).
>
> We need a solution because when TreeBASE bogs down it becomes too slow to
> submit data, etc.  In fact, right now (11:50PM) TreeBASE doesn't seem to be
> responding at all.
>
> Can anyone suggest a solution?
>
> Would this be a possible solution: deploy another VM with Tomcat and
> another database instance (e.g. take over treebase-stage.nescent.org for
> this purpose). The database is refreshed nightly from production; the war
> is rebuilt whenever production is rebuilt; but the two are sufficiently
> isolated so that heavy API activity doesn't cripple production. Traffic
> headed to http:://www.treebase.org goes to production. Traffic headed to
> http://purl.org/phylo/treebase/phylows/ goes to this other instance.
>
> I guess one possible snag is that when a /phylows/ query includes
> "format=html", it is jumping to the web page environment -- yet we don't
> want this API machine to serve web pages to people in any way. So we'd want
> a /phylows/...?format=html to redirect to production instead.
>
> bp
>
>
>
>
>
>
>


-- 
Mattison Ward
NESCent at Duke University
2024 W. Main Street, Suite A200
Durham, NC 27705-4667
919-668-4585 (desk)
919-668-4551 (alternate)
919-668-9198 (fax)

Re: [Treebase-devel] Treebase Dev problems

[Rutger V. <R....@re...>]

The idea of deploying another instance just for API use is a good one
but can't we, for now, just block the IP address that's doing the
hammering? This is obviously not fair usage.

On Thu, Nov 17, 2011 at 5:01 AM, William Piel <wil...@ya...> wrote:
>
> Hi all,
>
> So we continue to get hammered by, I think, someone who is making heavy use of the API. Maybe Mattison can better identify the source of this -- but I don't think it's just higher UI traffic or robot activities because Google Analytics is not showing a jump in activity. While logs show some uncaught exceptions (and Harry is fixing some of these) these are probably not actually causing the slowdown. Rather, objects in memory are probably not being disposed of fast enough and the database is clogging up (direct queries against postgresql are also very slow).
>
> We need a solution because when TreeBASE bogs down it becomes too slow to submit data, etc.  In fact, right now (11:50PM) TreeBASE doesn't seem to be responding at all.
>
> Can anyone suggest a solution?
>
> Would this be a possible solution: deploy another VM with Tomcat and another database instance (e.g. take over treebase-stage.nescent.org for this purpose). The database is refreshed nightly from production; the war is rebuilt whenever production is rebuilt; but the two are sufficiently isolated so that heavy API activity doesn't cripple production. Traffic headed to http:://www.treebase.org goes to production. Traffic headed to http://purl.org/phylo/treebase/phylows/ goes to this other instance.
>
> I guess one possible snag is that when a /phylows/ query includes "format=html", it is jumping to the web page environment -- yet we don't want this API machine to serve web pages to people in any way. So we'd want a /phylows/...?format=html to redirect to production instead.
>
> bp
>
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure
> contains a definitive record of customers, application performance,
> security threats, fraudulent activity, and more. Splunk takes this
> data and makes sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-novd2d
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>



-- 
Dr. Rutger A. Vos
School of Biological Sciences
Philip Lyle Building, Level 4
University of Reading
Reading, RG6 6BX, United Kingdom
Tel: +44 (0) 118 378 7535
http://rutgervos.blogspot.com

Re: [Treebase-devel] Treebase Dev problems

[William P. <wil...@ya...>]

Hi all,

So we continue to get hammered by, I think, someone who is making heavy use of the API. Maybe Mattison can better identify the source of this -- but I don't think it's just higher UI traffic or robot activities because Google Analytics is not showing a jump in activity. While logs show some uncaught exceptions (and Harry is fixing some of these) these are probably not actually causing the slowdown. Rather, objects in memory are probably not being disposed of fast enough and the database is clogging up (direct queries against postgresql are also very slow). 

We need a solution because when TreeBASE bogs down it becomes too slow to submit data, etc.  In fact, right now (11:50PM) TreeBASE doesn't seem to be responding at all.

Can anyone suggest a solution? 

Would this be a possible solution: deploy another VM with Tomcat and another database instance (e.g. take over treebase-stage.nescent.org for this purpose). The database is refreshed nightly from production; the war is rebuilt whenever production is rebuilt; but the two are sufficiently isolated so that heavy API activity doesn't cripple production. Traffic headed to http:://www.treebase.org goes to production. Traffic headed to http://purl.org/phylo/treebase/phylows/ goes to this other instance. 

I guess one possible snag is that when a /phylows/ query includes "format=html", it is jumping to the web page environment -- yet we don't want this API machine to serve web pages to people in any way. So we'd want a /phylows/...?format=html to redirect to production instead. 

bp

Re: [Treebase-devel] Treebase Dev problems

[William P. <wil...@ya...>]

On Nov 15, 2011, at 10:51 AM, Mattison Ward wrote:

> Production seems to have started with the same problem around 10:15 AM.
> 
> catalina.out does not provide timestamps which makes it difficult to
> correlate tomcat errors and the apache logs which show IP addresses.
> 
> My guess is that one of these two IP addresses are causing the current
> "overload"/slow down on production:
> 
> 141.2.253.17 - Johann Wolfgang Goethe-Universitaet Frankfurt
> 31.16.108.211 - Kabel Deutschland Breitband Customer
> 
> -Mattison

hmm... there are a number of candidates... e.g. someone from Annette Klussmann-Kolb's lab or Markus Pfenninger's lab. But I'm guessing it might be Lin Himmelmann, seeing has he has done a meta-analysis on TreeBASE before (e.g. here). 

I'll send them an email to inquire. 

bp

Re: [Treebase-devel] Treebase Dev problems

[Mattison W. <mat...@ne...>]

Production seems to have started with the same problem around 10:15 AM.

catalina.out does not provide timestamps which makes it difficult to
correlate tomcat errors and the apache logs which show IP addresses.

My guess is that one of these two IP addresses are causing the current
"overload"/slow down on production:

141.2.253.17 - Johann Wolfgang Goethe-Universitaet Frankfurt
31.16.108.211 - Kabel Deutschland Breitband Customer

-Mattison

On Tue, Nov 15, 2011 at 9:26 AM, Hilmar Lapp <hl...@ne...> wrote:
> I can't think of any good reason why we would want any search engines crawling any of the dev or staging sites' contents. For production we do, though, at least so long as it doesn't harm the stability of the site.
>
> -hilmar
>
> Sent with a tap.
>
> On Nov 15, 2011, at 7:50 AM, Mattison Ward <mat...@ne...> wrote:
>
>> The Nagios monitoring system queries treebase-dev every few minutes to
>> make sure it is up using this query:
>>
>> http://treebase-dev.nescent.org/treebase-web/search/studySearch.html?query=prism.publicationName=Nature&format=null&recordSchema=null
>>
>>
>> It might be unrelated, but I saw a fair amount of activity from search
>> engines in the web server logs.
>>
>> I can set up a robots.txt file to keep search engines from crawling
>> the dev and staging sites.
>>
>> Would it make sense to keep search engines from crawling any sections
>> of the production site?
>>
>> -Mattison
>>
>> On Mon, Nov 14, 2011 at 5:10 PM, William Piel <wil...@ya...> wrote:
>>>
>>> Is anyone hitting dev with a lot of queries (e.g. asking for
>>> publication=="Nature") ?  Or is there anything else in these logs that
>>> indicates what might be taking down treebasedev ? (please see the attached
>>> logs)
>>> Admittedly, it doesn't take much to cause a denial-of-service... But I was
>>> thinking that it might be in our group, seeing as dev is being hit.
>>> bp
>>>
>>> Begin forwarded message:
>>>
>>> From: Mattison Ward <mat...@ne...>
>>> Date: November 14, 2011 4:12:35 PM EST
>>> To: William Piel <wil...@ya...>, Harry Shyket
>>> <har...@ya...>
>>> Cc: David Palmer <dav...@ne...>
>>> Subject: Treebase Dev problems
>>>
>>> Hi Bill and Harry.
>>>
>>> Starting late last night, the treebasedev tomcat process has been
>>> overloading the server.  I restarted tomcat several times today but
>>> after a few hours the problem occurred again.  No changes have been
>>> made to the system recently and I don't see any deployments from
>>> Hudson recently either.
>>>
>>> I have the treebasedev tomcat service shut down now.
>>>
>>> I have attached the logs for you to review to see if it looks like an
>>> application problem.  I can send them in a different format if you
>>> don't like tgz files.
>>>
>>> Regards,
>>>
>>> Mattison
>>>
>>> ---------- Forwarded message ----------
>>> From: root <ro...@tr...>
>>> Date: Mon, Nov 14, 2011 at 4:07 PM
>>> Subject: treebaselogs
>>> To: mat...@gm...
>>>
>>>
>>> logs
>>>
>>>
>>>
>>> --
>>> Mattison Ward
>>> NESCent at Duke University
>>> 2024 W. Main Street, Suite A200
>>> Durham, NC 27705-4667
>>> 919-668-4585 (desk)
>>> 919-668-4551 (alternate)
>>> 919-668-9198 (fax)
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Mattison Ward
>> NESCent at Duke University
>> 2024 W. Main Street, Suite A200
>> Durham, NC 27705-4667
>> 919-668-4585 (desk)
>> 919-668-4551 (alternate)
>> 919-668-9198 (fax)
>>
>> ------------------------------------------------------------------------------
>> RSA(R) Conference 2012
>> Save $700 by Nov 18
>> Register now
>> http://p.sf.net/sfu/rsa-sfdev2dev1
>> _______________________________________________
>> Treebase-devel mailing list
>> Tre...@li...
>> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>



-- 
Mattison Ward
NESCent at Duke University
2024 W. Main Street, Suite A200
Durham, NC 27705-4667
919-668-4585 (desk)
919-668-4551 (alternate)
919-668-9198 (fax)

Re: [Treebase-devel] Fwd: Treebase Dev problems

[Shyket, H. <har...@ya...>]

Maybe it would be possible to block the search engines from saving session data.  We could possibly check the hostname and if it matches GoogleBot, Yahoo or Bing, stop it from saving to session.

Harry Shyket
Digital Media Specialist
Yale University Peabody Museum
ph. 203-436-9428
har...@ya...

From: William Piel [mailto:wil...@ya...]
Sent: Tuesday, November 15, 2011 9:54 AM
To: TreeBASE devel
Subject: Re: [Treebase-devel] Fwd: Treebase Dev problems


On Nov 15, 2011, at 9:35 AM, Rutger Vos wrote:


We want search engines to crawl as much of the production server as we can manage (but remember when the google bot brought it to its knees) but none of dev and stage - that would only lead to inconsistent search results. Ideally we would let the bots crawl a site map with the purls so that it is those that are in the search results (though I wonder whether a bot would use the purl as the address or whatever that purl forwards to).

yes, currently we publish a site map for Google -- which is why this query results in over 60,000 hits:

http://www.google.com/search?client=safari&rls=en&q=site:treebase.org

Google finds 4,000 hits using this one:

http://www.google.com/search?client=safari&rls=en&q=site:treebase-dev.nescent.org

which is probably done by indexing without the guidance of our site map.

So, bad Google.

Definitely, let's make dev and stage off-limits to search robots.

bp

Re: [Treebase-devel] Fwd: Treebase Dev problems

[William P. <wil...@ya...>]

On Nov 15, 2011, at 9:35 AM, Rutger Vos wrote:

> We want search engines to crawl as much of the production server as we can manage (but remember when the google bot brought it to its knees) but none of dev and stage - that would only lead to inconsistent search results. Ideally we would let the bots crawl a site map with the purls so that it is those that are in the search results (though I wonder whether a bot would use the purl as the address or whatever that purl forwards to).

yes, currently we publish a site map for Google -- which is why this query results in over 60,000 hits:

http://www.google.com/search?client=safari&rls=en&q=site:treebase.org

Google finds 4,000 hits using this one:

http://www.google.com/search?client=safari&rls=en&q=site:treebase-dev.nescent.org

which is probably done by indexing without the guidance of our site map. 

So, bad Google. 

Definitely, let's make dev and stage off-limits to search robots. 

bp

Re: [Treebase-devel] Treebase Dev problems

[Mattison W. <mat...@ne...>]

Ok.  The robots.txt is in place now for dev and staging.  I'll keep an
eye on it to see if that resolves the problem.  If not, I'll correlate
the logs to see what IP address the problem is coming from.

-Mattison

On Tue, Nov 15, 2011 at 9:31 AM, William Piel <wil...@ya...> wrote:
>
> On Nov 15, 2011, at 8:50 AM, Mattison Ward wrote:
>
> The Nagios monitoring system queries treebase-dev every few minutes to
> make sure it is up using this query:
>
> http://treebase-dev.nescent.org/treebase-web/search/studySearch.html?query=prism.publicationName=Nature&format=null&recordSchema=null
>
>
> It might be unrelated, but I saw a fair amount of activity from search
> engines in the web server logs.
>
> I can set up a robots.txt file to keep search engines from crawling
> the dev and staging sites.
>
> Would it make sense to keep search engines from crawling any sections
> of the production site?
>
> Hi Mattison:
> Search engines are already blocked from crawling production:
> http://www.treebase.org/robots.txt
> ... though I don't find this on stage or dev:
> http://treebase-stage.nescent.org/robots.txt
> http://treebase-dev.nescent.org/robots.txt
> So definitely, please have a robots block on stage and dev -- in fact, it
> should block *everything* on those two sites because we don't want them to
> compete with production.
> So perhaps this has nothing to do with Carl's R bindings. That would be
> great news if true.
> Do you have logs that provide the IP identity of users responsible for
> taking down dev?
> bp
>
>
>
>
>



-- 
Mattison Ward
NESCent at Duke University
2024 W. Main Street, Suite A200
Durham, NC 27705-4667
919-668-4585 (desk)
919-668-4551 (alternate)
919-668-9198 (fax)

Re: [Treebase-devel] Fwd: Treebase Dev problems

[Rutger V. <R....@re...>]

We want search engines to crawl as much of the production server as we can
manage (but remember when the google bot brought it to its knees) but none
of dev and stage - that would only lead to inconsistent search results.
Ideally we would let the bots crawl a site map with the purls so that it is
those that are in the search results (though I wonder whether a bot would
use the purl as the address or whatever that purl forwards to).

On Tuesday, November 15, 2011, Hilmar Lapp <hl...@ne...> wrote:
> I can't think of any good reason why we would want any search engines
crawling any of the dev or staging sites' contents. For production we do,
though, at least so long as it doesn't harm the stability of the site.
>
> -hilmar
>
> Sent with a tap.
>
> On Nov 15, 2011, at 7:50 AM, Mattison Ward <mat...@ne...>
wrote:
>
>> The Nagios monitoring system queries treebase-dev every few minutes to
>> make sure it is up using this query:
>>
>>
http://treebase-dev.nescent.org/treebase-web/search/studySearch.html?query=prism.publicationName=Nature&format=null&recordSchema=null
>>
>>
>> It might be unrelated, but I saw a fair amount of activity from search
>> engines in the web server logs.
>>
>> I can set up a robots.txt file to keep search engines from crawling
>> the dev and staging sites.
>>
>> Would it make sense to keep search engines from crawling any sections
>> of the production site?
>>
>> -Mattison
>>
>> On Mon, Nov 14, 2011 at 5:10 PM, William Piel <wil...@ya...>
wrote:
>>>
>>> Is anyone hitting dev with a lot of queries (e.g. asking for
>>> publication=="Nature") ?  Or is there anything else in these logs that
>>> indicates what might be taking down treebasedev ? (please see the
attached
>>> logs)
>>> Admittedly, it doesn't take much to cause a denial-of-service... But I
was
>>> thinking that it might be in our group, seeing as dev is being hit.
>>> bp
>>>
>>> Begin forwarded message:
>>>
>>> From: Mattison Ward <mat...@ne...>
>>> Date: November 14, 2011 4:12:35 PM EST
>>> To: William Piel <wil...@ya...>, Harry Shyket
>>> <har...@ya...>
>>> Cc: David Palmer <dav...@ne...>
>>> Subject: Treebase Dev problems
>>>
>>> Hi Bill and Harry.
>>>
>>> Starting late last night, the treebasedev tomcat process has been
>>> overloading the server.  I restarted tomcat several times today but
>>> after a few hours the problem occurred again.  No changes have been
>>> made to the system recently and I don't see any deployments from
>>> Hudson recently either.
>>>
>>> I have the treebasedev tomcat service shut down now.
>>>
>>> I have attached the logs for you to review to see if it looks like an
>>> application problem.  I can send them in a different format if you
>>> don't like tgz files.
>>>
>>> Regards,
>>>
>>> Mattison
>>>
>>> ---------- Forwarded message ----------
>>> From: root <ro...@tr...>
>>> Date: Mon, Nov 14, 2011 at 4:07 PM
>>> Subject: treebaselogs
>>> To: mat...@gm...
>>>
>>>
>>> logs
>>>
>>>
>>>
>>> --
>>> Mattison Ward
>>> NESCent at Duke University
>>> 2024 W. Main Street, Suite A200
>>> Durham, NC 27705-4667
>>> 919-668-4585 (desk)
>>> 919-668-4551 (alternate)
>>> 919-668-9198 (fax)
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Mattison Ward
>> NESCent at Duke University
>> 2024 W. Main Street, Suite A200
>> Durham, NC 27705-4667
>> 919-668-4585 (desk)
>> 919-668-4551 (alternate)
>> 919-668-9198 (fax)
>>
>>
------------------------------------------------------------------------------
>> RSA(R) Conference 2012
>> Save $700 by Nov 18
>> Register now
>> http://p.sf.net/sfu/rsa-sfdev2dev1
>> _______________________________________________
>> Treebase-devel mailing list
>> Tre...@li...
>> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>
>
------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>

-- 
Dr. Rutger A. Vos
School of Biological Sciences
Philip Lyle Building, Level 4
University of Reading
Reading, RG6 6BX, United Kingdom
Tel: +44 (0) 118 378 7535
http://rutgervos.blogspot.com

Re: [Treebase-devel] Treebase Dev problems

[William P. <wil...@ya...>]

On Nov 15, 2011, at 8:50 AM, Mattison Ward wrote:

> The Nagios monitoring system queries treebase-dev every few minutes to
> make sure it is up using this query:
> 
> http://treebase-dev.nescent.org/treebase-web/search/studySearch.html?query=prism.publicationName=Nature&format=null&recordSchema=null
> 
> 
> It might be unrelated, but I saw a fair amount of activity from search
> engines in the web server logs.
> 
> I can set up a robots.txt file to keep search engines from crawling
> the dev and staging sites.
> 
> Would it make sense to keep search engines from crawling any sections
> of the production site?

Hi Mattison:

Search engines are already blocked from crawling production:

http://www.treebase.org/robots.txt

... though I don't find this on stage or dev:

http://treebase-stage.nescent.org/robots.txt
http://treebase-dev.nescent.org/robots.txt

So definitely, please have a robots block on stage and dev -- in fact, it should block *everything* on those two sites because we don't want them to compete with production. 

So perhaps this has nothing to do with Carl's R bindings. That would be great news if true. 

Do you have logs that provide the IP identity of users responsible for taking down dev? 

bp

Re: [Treebase-devel] Treebase Dev problems

[Hilmar L. <hl...@ne...>]

I can't think of any good reason why we would want any search engines crawling any of the dev or staging sites' contents. For production we do, though, at least so long as it doesn't harm the stability of the site.

-hilmar

Sent with a tap.

On Nov 15, 2011, at 7:50 AM, Mattison Ward <mat...@ne...> wrote:

> The Nagios monitoring system queries treebase-dev every few minutes to
> make sure it is up using this query:
> 
> http://treebase-dev.nescent.org/treebase-web/search/studySearch.html?query=prism.publicationName=Nature&format=null&recordSchema=null
> 
> 
> It might be unrelated, but I saw a fair amount of activity from search
> engines in the web server logs.
> 
> I can set up a robots.txt file to keep search engines from crawling
> the dev and staging sites.
> 
> Would it make sense to keep search engines from crawling any sections
> of the production site?
> 
> -Mattison
> 
> On Mon, Nov 14, 2011 at 5:10 PM, William Piel <wil...@ya...> wrote:
>> 
>> Is anyone hitting dev with a lot of queries (e.g. asking for
>> publication=="Nature") ?  Or is there anything else in these logs that
>> indicates what might be taking down treebasedev ? (please see the attached
>> logs)
>> Admittedly, it doesn't take much to cause a denial-of-service... But I was
>> thinking that it might be in our group, seeing as dev is being hit.
>> bp
>> 
>> Begin forwarded message:
>> 
>> From: Mattison Ward <mat...@ne...>
>> Date: November 14, 2011 4:12:35 PM EST
>> To: William Piel <wil...@ya...>, Harry Shyket
>> <har...@ya...>
>> Cc: David Palmer <dav...@ne...>
>> Subject: Treebase Dev problems
>> 
>> Hi Bill and Harry.
>> 
>> Starting late last night, the treebasedev tomcat process has been
>> overloading the server.  I restarted tomcat several times today but
>> after a few hours the problem occurred again.  No changes have been
>> made to the system recently and I don't see any deployments from
>> Hudson recently either.
>> 
>> I have the treebasedev tomcat service shut down now.
>> 
>> I have attached the logs for you to review to see if it looks like an
>> application problem.  I can send them in a different format if you
>> don't like tgz files.
>> 
>> Regards,
>> 
>> Mattison
>> 
>> ---------- Forwarded message ----------
>> From: root <ro...@tr...>
>> Date: Mon, Nov 14, 2011 at 4:07 PM
>> Subject: treebaselogs
>> To: mat...@gm...
>> 
>> 
>> logs
>> 
>> 
>> 
>> --
>> Mattison Ward
>> NESCent at Duke University
>> 2024 W. Main Street, Suite A200
>> Durham, NC 27705-4667
>> 919-668-4585 (desk)
>> 919-668-4551 (alternate)
>> 919-668-9198 (fax)
>> 
>> 
>> 
>> 
> 
> 
> 
> -- 
> Mattison Ward
> NESCent at Duke University
> 2024 W. Main Street, Suite A200
> Durham, NC 27705-4667
> 919-668-4585 (desk)
> 919-668-4551 (alternate)
> 919-668-9198 (fax)
> 
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel

Re: [Treebase-devel] Treebase Dev problems

[Mattison W. <mat...@ne...>]

The Nagios monitoring system queries treebase-dev every few minutes to
make sure it is up using this query:

http://treebase-dev.nescent.org/treebase-web/search/studySearch.html?query=prism.publicationName=Nature&format=null&recordSchema=null


It might be unrelated, but I saw a fair amount of activity from search
engines in the web server logs.

I can set up a robots.txt file to keep search engines from crawling
the dev and staging sites.

Would it make sense to keep search engines from crawling any sections
of the production site?

-Mattison

On Mon, Nov 14, 2011 at 5:10 PM, William Piel <wil...@ya...> wrote:
>
> Is anyone hitting dev with a lot of queries (e.g. asking for
> publication=="Nature") ?  Or is there anything else in these logs that
> indicates what might be taking down treebasedev ? (please see the attached
> logs)
> Admittedly, it doesn't take much to cause a denial-of-service... But I was
> thinking that it might be in our group, seeing as dev is being hit.
> bp
>
> Begin forwarded message:
>
> From: Mattison Ward <mat...@ne...>
> Date: November 14, 2011 4:12:35 PM EST
> To: William Piel <wil...@ya...>, Harry Shyket
> <har...@ya...>
> Cc: David Palmer <dav...@ne...>
> Subject: Treebase Dev problems
>
> Hi Bill and Harry.
>
> Starting late last night, the treebasedev tomcat process has been
> overloading the server.  I restarted tomcat several times today but
> after a few hours the problem occurred again.  No changes have been
> made to the system recently and I don't see any deployments from
> Hudson recently either.
>
> I have the treebasedev tomcat service shut down now.
>
> I have attached the logs for you to review to see if it looks like an
> application problem.  I can send them in a different format if you
> don't like tgz files.
>
> Regards,
>
> Mattison
>
> ---------- Forwarded message ----------
> From: root <ro...@tr...>
> Date: Mon, Nov 14, 2011 at 4:07 PM
> Subject: treebaselogs
> To: mat...@gm...
>
>
> logs
>
>
>
> --
> Mattison Ward
> NESCent at Duke University
> 2024 W. Main Street, Suite A200
> Durham, NC 27705-4667
> 919-668-4585 (desk)
> 919-668-4551 (alternate)
> 919-668-9198 (fax)
>
>
>
>



-- 
Mattison Ward
NESCent at Duke University
2024 W. Main Street, Suite A200
Durham, NC 27705-4667
919-668-4585 (desk)
919-668-4551 (alternate)
919-668-9198 (fax)

Re: [Treebase-devel] Fwd: Treebase Dev problems

[Carl B. <cbo...@gm...>]

Thanks, this sounds like a good place to start.

It isn't common that it does this, only if a user writes a loop that makes
repeated calls to the server.  Most of my use examples show constructing
single queries.  I've just added a function for users to make repeated
calls that should have a bit less handshaking each time, and can pause over
the loop.

-Carl

On Mon, Nov 14, 2011 at 6:16 PM, William Piel <wil...@ya...> wrote:

>
> Yeah, I totally agree -- if it takes down dev, it will also take down
> production. It would be better to point it to a third deployment (e.g.
> perhaps http://treebase-stage.nescent.org/treebase-web/), or better yet,
> one that runs on a separate spare machine so that it won't take down any
> other services -- and meanwhile we try to figure out how to make the API
> more robust. As a stateless API, it ought not to be filling up memory
> faster than it can dispose of it (if that's what's happening), but perhaps
> the problem is something else (corrupt data?).
>
> bp
>
> On Nov 14, 2011, at 7:56 PM, Rutger Vos wrote:
>
> Hey Carl,
>
> production isn't that much more ironclad to be honest. If a user is
> doing some serious harvesting of dozens/hundreds of requests for
> studies there's a good chance some of those are going to be ones that
> will construct long running queries to re-constitute all the data in a
> study.
>
> Until we have some caching strategy there's not really a good
> recommendation to make for rate of calls: even relatively low rates
> (several seconds apart, say) may be able to bring the server to its
> knees.
>
> Rutger
>
>
>


-- 
Carl Boettiger
UC Davis
http://www.carlboettiger.info/

Re: [Treebase-devel] Fwd: Treebase Dev problems

[William P. <wil...@ya...>]

Yeah, I totally agree -- if it takes down dev, it will also take down production. It would be better to point it to a third deployment (e.g. perhaps http://treebase-stage.nescent.org/treebase-web/), or better yet, one that runs on a separate spare machine so that it won't take down any other services -- and meanwhile we try to figure out how to make the API more robust. As a stateless API, it ought not to be filling up memory faster than it can dispose of it (if that's what's happening), but perhaps the problem is something else (corrupt data?). 

bp

On Nov 14, 2011, at 7:56 PM, Rutger Vos wrote:

> Hey Carl,
> 
> production isn't that much more ironclad to be honest. If a user is
> doing some serious harvesting of dozens/hundreds of requests for
> studies there's a good chance some of those are going to be ones that
> will construct long running queries to re-constitute all the data in a
> study.
> 
> Until we have some caching strategy there's not really a good
> recommendation to make for rate of calls: even relatively low rates
> (several seconds apart, say) may be able to bring the server to its
> knees.
> 
> Rutger

Re: [Treebase-devel] Fwd: Treebase Dev problems

[Rutger V. <R....@re...>]

Hey Carl,

production isn't that much more ironclad to be honest. If a user is
doing some serious harvesting of dozens/hundreds of requests for
studies there's a good chance some of those are going to be ones that
will construct long running queries to re-constitute all the data in a
study.

Until we have some caching strategy there's not really a good
recommendation to make for rate of calls: even relatively low rates
(several seconds apart, say) may be able to bring the server to its
knees.

Rutger

On Tue, Nov 15, 2011 at 12:48 AM, Carl Boettiger <cbo...@gm...> wrote:
> Based on the logs it looks like that's happening due to calls from some user
> of my treebase package.  I'm switching it over to call only the production
> server, not the development server.  I'll also see what I can do to prevent
> the user from making too high a rate of queries.  Sorry about this.  Any
> suggestions/guidelines for rate of calls?
>
> Carl
>
> On Mon, Nov 14, 2011 at 3:04 PM, Hilmar Lapp <hl...@ne...> wrote:
>>
>> Perhaps some URLs to the dev site that are running stray?
>>
>> -hilmar
>> Sent with a tap.
>> On Nov 14, 2011, at 4:10 PM, William Piel <wil...@ya...> wrote:
>>
>>
>> Is anyone hitting dev with a lot of queries (e.g. asking for
>> publication=="Nature") ?  Or is there anything else in these logs that
>> indicates what might be taking down treebasedev ? (please see the attached
>> logs)
>> Admittedly, it doesn't take much to cause a denial-of-service... But I was
>> thinking that it might be in our group, seeing as dev is being hit.
>> bp
>>
>> Begin forwarded message:
>>
>> From: Mattison Ward <mat...@ne...>
>> Date: November 14, 2011 4:12:35 PM EST
>> To: William Piel <wil...@ya...>, Harry Shyket
>> <har...@ya...>
>> Cc: David Palmer <dav...@ne...>
>> Subject: Treebase Dev problems
>>
>> Hi Bill and Harry.
>>
>> Starting late last night, the treebasedev tomcat process has been
>> overloading the server.  I restarted tomcat several times today but
>> after a few hours the problem occurred again.  No changes have been
>> made to the system recently and I don't see any deployments from
>> Hudson recently either.
>>
>> I have the treebasedev tomcat service shut down now.
>>
>> I have attached the logs for you to review to see if it looks like an
>> application problem.  I can send them in a different format if you
>> don't like tgz files.
>>
>> Regards,
>>
>> Mattison
>>
>> ---------- Forwarded message ----------
>> From: root <ro...@tr...>
>> Date: Mon, Nov 14, 2011 at 4:07 PM
>> Subject: treebaselogs
>> To: mat...@gm...
>>
>>
>> logs
>>
>>
>>
>> --
>> Mattison Ward
>> NESCent at Duke University
>> 2024 W. Main Street, Suite A200
>> Durham, NC 27705-4667
>> 919-668-4585 (desk)
>> 919-668-4551 (alternate)
>> 919-668-9198 (fax)
>>
>> <logs.tgz>
>>
>>
>> ------------------------------------------------------------------------------
>> RSA(R) Conference 2012
>> Save $700 by Nov 18
>> Register now
>> http://p.sf.net/sfu/rsa-sfdev2dev1
>>
>> _______________________________________________
>> Treebase-devel mailing list
>> Tre...@li...
>> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>>
>>
>> ------------------------------------------------------------------------------
>> RSA(R) Conference 2012
>> Save $700 by Nov 18
>> Register now
>> http://p.sf.net/sfu/rsa-sfdev2dev1
>> _______________________________________________
>> Treebase-devel mailing list
>> Tre...@li...
>> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>>
>
>
>
> --
> Carl Boettiger
> UC Davis
> http://www.carlboettiger.info/
>
>
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>
>



-- 
Dr. Rutger A. Vos
School of Biological Sciences
Philip Lyle Building, Level 4
University of Reading
Reading, RG6 6BX, United Kingdom
Tel: +44 (0) 118 378 7535
http://rutgervos.blogspot.com

Re: [Treebase-devel] Fwd: Treebase Dev problems

[Carl B. <cbo...@gm...>]

Based on the logs it looks like that's happening due to calls from some
user of my treebase package.  I'm switching it over to call only the
production server, not the development server.  I'll also see what I can do
to prevent the user from making too high a rate of queries.  Sorry about
this.  Any suggestions/guidelines for rate of calls?

Carl

On Mon, Nov 14, 2011 at 3:04 PM, Hilmar Lapp <hl...@ne...> wrote:

> Perhaps some URLs to the dev site that are running stray?
>
> -hilmar
>
> Sent with a tap.
>
> On Nov 14, 2011, at 4:10 PM, William Piel <wil...@ya...> wrote:
>
>
> Is anyone hitting dev with a lot of queries (e.g. asking for
> publication=="Nature") ?  Or is there anything else in these logs that
> indicates what might be taking down treebasedev ? (please see the attached
> logs)
>
> Admittedly, it doesn't take much to cause a denial-of-service... But I was
> thinking that it might be in our group, seeing as dev is being hit.
>
> bp
>
>
> Begin forwarded message:
>
> *From: *Mattison Ward <mat...@ne...>
> *Date: *November 14, 2011 4:12:35 PM EST
> *To: *William Piel <wil...@ya...>, Harry Shyket <
> har...@ya...>
> *Cc: *David Palmer <dav...@ne...>
> *Subject: **Treebase Dev problems*
>
> Hi Bill and Harry.
>
> Starting late last night, the treebasedev tomcat process has been
> overloading the server.  I restarted tomcat several times today but
> after a few hours the problem occurred again.  No changes have been
> made to the system recently and I don't see any deployments from
> Hudson recently either.
>
> I have the treebasedev tomcat service shut down now.
>
> I have attached the logs for you to review to see if it looks like an
> application problem.  I can send them in a different format if you
> don't like tgz files.
>
> Regards,
>
> Mattison
>
> ---------- Forwarded message ----------
> From: root <ro...@tr...>
> Date: Mon, Nov 14, 2011 at 4:07 PM
> Subject: treebaselogs
> To: mat...@gm...
>
>
> logs
>
>
>
> --
> Mattison Ward
> NESCent at Duke University
> 2024 W. Main Street, Suite A200
> Durham, NC 27705-4667
> 919-668-4585 (desk)
> 919-668-4551 (alternate)
> 919-668-9198 (fax)
>
> <logs.tgz>
>
>
>
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
>
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>
>
>
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>
>


-- 
Carl Boettiger
UC Davis
http://www.carlboettiger.info/

Re: [Treebase-devel] Fwd: Treebase Dev problems

[Hilmar L. <hl...@ne...>]

Perhaps some URLs to the dev site that are running stray?

-hilmar

Sent with a tap.

On Nov 14, 2011, at 4:10 PM, William Piel <wil...@ya...> wrote:

> 
> Is anyone hitting dev with a lot of queries (e.g. asking for publication=="Nature") ?  Or is there anything else in these logs that indicates what might be taking down treebasedev ? (please see the attached logs)
> 
> Admittedly, it doesn't take much to cause a denial-of-service... But I was thinking that it might be in our group, seeing as dev is being hit. 
> 
> bp
> 
> 
> Begin forwarded message:
> 
>> From: Mattison Ward <mat...@ne...>
>> Date: November 14, 2011 4:12:35 PM EST
>> To: William Piel <wil...@ya...>, Harry Shyket <har...@ya...>
>> Cc: David Palmer <dav...@ne...>
>> Subject: Treebase Dev problems
>> 
>> Hi Bill and Harry.
>> 
>> Starting late last night, the treebasedev tomcat process has been
>> overloading the server.  I restarted tomcat several times today but
>> after a few hours the problem occurred again.  No changes have been
>> made to the system recently and I don't see any deployments from
>> Hudson recently either.
>> 
>> I have the treebasedev tomcat service shut down now.
>> 
>> I have attached the logs for you to review to see if it looks like an
>> application problem.  I can send them in a different format if you
>> don't like tgz files.
>> 
>> Regards,
>> 
>> Mattison
>> 
>> ---------- Forwarded message ----------
>> From: root <ro...@tr...>
>> Date: Mon, Nov 14, 2011 at 4:07 PM
>> Subject: treebaselogs
>> To: mat...@gm...
>> 
>> 
>> logs
>> 
>> 
>> 
>> -- 
>> Mattison Ward
>> NESCent at Duke University
>> 2024 W. Main Street, Suite A200
>> Durham, NC 27705-4667
>> 919-668-4585 (desk)
>> 919-668-4551 (alternate)
>> 919-668-9198 (fax)
> <logs.tgz>
> 
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel

Re: [Treebase-devel] Fwd: Treebase Dev problems

[Rutger V. <R....@re...>]

I'm not doing anything.

On Mon, Nov 14, 2011 at 10:10 PM, William Piel <wil...@ya...> wrote:
>
> Is anyone hitting dev with a lot of queries (e.g. asking for
> publication=="Nature") ?  Or is there anything else in these logs that
> indicates what might be taking down treebasedev ? (please see the attached
> logs)
> Admittedly, it doesn't take much to cause a denial-of-service... But I was
> thinking that it might be in our group, seeing as dev is being hit.
> bp
>
> Begin forwarded message:
>
> From: Mattison Ward <mat...@ne...>
> Date: November 14, 2011 4:12:35 PM EST
> To: William Piel <wil...@ya...>, Harry Shyket
> <har...@ya...>
> Cc: David Palmer <dav...@ne...>
> Subject: Treebase Dev problems
>
> Hi Bill and Harry.
>
> Starting late last night, the treebasedev tomcat process has been
> overloading the server.  I restarted tomcat several times today but
> after a few hours the problem occurred again.  No changes have been
> made to the system recently and I don't see any deployments from
> Hudson recently either.
>
> I have the treebasedev tomcat service shut down now.
>
> I have attached the logs for you to review to see if it looks like an
> application problem.  I can send them in a different format if you
> don't like tgz files.
>
> Regards,
>
> Mattison
>
> ---------- Forwarded message ----------
> From: root <ro...@tr...>
> Date: Mon, Nov 14, 2011 at 4:07 PM
> Subject: treebaselogs
> To: mat...@gm...
>
>
> logs
>
>
>
> --
> Mattison Ward
> NESCent at Duke University
> 2024 W. Main Street, Suite A200
> Durham, NC 27705-4667
> 919-668-4585 (desk)
> 919-668-4551 (alternate)
> 919-668-9198 (fax)
>
>
>
> ------------------------------------------------------------------------------
> RSA(R) Conference 2012
> Save $700 by Nov 18
> Register now
> http://p.sf.net/sfu/rsa-sfdev2dev1
> _______________________________________________
> Treebase-devel mailing list
> Tre...@li...
> https://lists.sourceforge.net/lists/listinfo/treebase-devel
>
>



-- 
Dr. Rutger A. Vos
School of Biological Sciences
Philip Lyle Building, Level 4
University of Reading
Reading, RG6 6BX, United Kingdom
Tel: +44 (0) 118 378 7535
http://rutgervos.blogspot.com

[Treebase-devel] Fwd: Treebase Dev problems

[William P. <wil...@ya...>]

Is anyone hitting dev with a lot of queries (e.g. asking for publication=="Nature") ?  Or is there anything else in these logs that indicates what might be taking down treebasedev ? (please see the attached logs)

Admittedly, it doesn't take much to cause a denial-of-service... But I was thinking that it might be in our group, seeing as dev is being hit. 

bp


Begin forwarded message:

> From: Mattison Ward <mat...@ne...>
> Date: November 14, 2011 4:12:35 PM EST
> To: William Piel <wil...@ya...>, Harry Shyket <har...@ya...>
> Cc: David Palmer <dav...@ne...>
> Subject: Treebase Dev problems
> 
> Hi Bill and Harry.
> 
> Starting late last night, the treebasedev tomcat process has been
> overloading the server.  I restarted tomcat several times today but
> after a few hours the problem occurred again.  No changes have been
> made to the system recently and I don't see any deployments from
> Hudson recently either.
> 
> I have the treebasedev tomcat service shut down now.
> 
> I have attached the logs for you to review to see if it looks like an
> application problem.  I can send them in a different format if you
> don't like tgz files.
> 
> Regards,
> 
> Mattison
> 
> ---------- Forwarded message ----------
> From: root <ro...@tr...>
> Date: Mon, Nov 14, 2011 at 4:07 PM
> Subject: treebaselogs
> To: mat...@gm...
> 
> 
> logs
> 
> 
> 
> -- 
> Mattison Ward
> NESCent at Duke University
> 2024 W. Main Street, Suite A200
> Durham, NC 27705-4667
> 919-668-4585 (desk)
> 919-668-4551 (alternate)
> 919-668-9198 (fax)