simpleweb-support — General user and support issues

RE: [Simpleweb-Support] FIX: Simple stops responding

[Jorge C. <Jor...@ge...>]

Great news!

Of course you can contact me directly.

I've tested the change a couple of times (actually changing the static
initializer to pass 100 as parameter to the constructor) and Simple doesn't
"hang". I saw that probably the ready ObjectQueue mixes the things up by
reusing positions in its Object[] to store the Daemons, but I can't see very
clearly why this is producing deadlocks. Before the queue was sized at 50
and we were using 35 threads... are we only pushing the problem? Can you see
the problem clearly?

Regardless if this is the problem, the implementation of the resize method
should be changed in order to resize the ObjectQueue (a resize method should
be added).

Thanks a lot for your support.

Jorge
-----Original Message-----
From: sim...@li...
[mailto:sim...@li...]On Behalf Of Niall
Gallagher
Sent: Friday, August 05, 2005 2:11 PM
To: sim...@li...
Subject: RE: [Simpleweb-Support] FIX: Simple stops responding


Hi Jorge,

Apologies for contacting you directly. However, I
think I have found a solution, but I don't want to be
too hasty. What triggered this is when you said that
at least 35 threads had to be involved.

So, I downloaded the simple-test.tar.gz and resized
the ProcessQueue, this caused a deadlock with the very
same test case. So if you could be so kind as to check
to see if this works for you! In the file
ProcessQueue.java change the following line.

this.ready = new ObjectQueue(size);

to

this.ready = new ObjectQueue(100);

Making sure that you dont resize beyond 100. I think
the problem is that the ProcessQueue.resize does not
resize the ObjectQueue. This seems to work for my
tests?

Niall



--- Jorge CAMPOS <Jor...@ge...> wrote:

> Hi everybody!
>
> From our tests with our application, it seems that
> Simple hangs only when
> using at least 35 threads in parallel (we're testing
> in a 4Way Xeon with HT
> running SuSE SLES 9.0 and JDK 1.5.0_04). From the
> thread dump and some
> printlns in the code it seems that one thread
> (dequeuer) blocks because
> there are no more requests to be processed. After
> that everything starts
> piling up (all the dequeuers wait for the first one
> and the enqueuers too).
> I know it sound a bit weird, but it could be related
> to the locking queue
> implementation.
>
> I've tried to reproduce the problem using the demo
> server and I didn't
> succeed. I guess the context is different (to
> simulate some load I've put
> one sleep and some printlns). This made me look
> again at our thread dump...
> and I can't see anything related to our application
> (it seems that Simple
> blocks within its code).
>
> One question Andrew: what are the specs of your
> machine?
>
> I'm asking this, because I've seen some locking
> problems in applications
> running with HyperThreading ON.
>
> We will test the application running with
> HyperThreading off and in another
> machine to see if we can reproduce the problem.
>
> All your support is greatly appreciated.
>
> Ah, one more thing. Since in our thread dump it
> seems all the threads are
> still alive, I don't think this particular case
> relates to a out of memory
> exception (also there are no exceptions in the logs)
>
>
> Jorge
>
> -----Original Message-----
> From: sim...@li...
>
[mailto:sim...@li...]On
> Behalf Of Niall
> Gallagher
> Sent: Wednesday, August 03, 2005 10:21 AM
> To: sim...@li...
> Subject: Re: [Simpleweb-Support] FIX: Simple stops
> responding
>
>
> Hi,
>
> I have taken note of the last two comments with
> regard
> to the server freezing! In response to the queue
> implementation, I think that the implementation of
> the
> Scheduler, ObjectQueue, and ProcessQueue involved in
> dispatching requests is fine. There are no problems
> here.
>
> However, the simple.http.connect implementation may
> be
> at fault, and I have no doubt whatsoever that the
> java.lang.OutOfMemory error will cause the server to
> stop responding under very heavy load.
>
> I have uploaded my test framework for anyone wishing
> to  use it to see if it will freeze their server. If
> it does then great! let me know what happened.
> However, I have put through well over a 100 million
> requests using my test framework today alone and,
> with
> the catch(Throwable) amendments there is no
> deadlock,
> not even a lost thread.
>
> The test framework can be downloaded at
> http://simpleweb.sf.net/simple-test.tar.gz.
>
> So, in conclusion, the application is thread safe
> and
> working properly with catch(Throwable) amendments.
> However, the TCP related issues have not been
> thought
> out carefully! And I believe the simple.http.connect
> package needs to be reworked! I will release all the
> bug fixes as soon as I am convinced everything is
> working.
>
> Thanks for the bug reports, if anyone thinks the
> freeze is related to somthing else please let me
> know
> so that I can explore it!
>
> Niall
>
>
> Niall Gallagher
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam
> protection around
> http://mail.yahoo.com
>
>
>
-------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux
> Migration Strategies
> from IBM. Find simple to follow Roadmaps,
> straightforward articles,
> informative Webcasts and more! Get everything you
> need to get up to
> speed, fast.
>
http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> Simpleweb-Support mailing list
> Sim...@li...
>
https://lists.sourceforge.net/lists/listinfo/simpleweb-support
>
>
>
>
-------------------------------------------------------
> SF.Net email is Sponsored by the Better Software
> Conference & EXPO
> September 19-22, 2005 * San Francisco, CA *
> Development Lifecycle Practices
> Agile & Plan-Driven Development * Managing Projects
> & Teams * Testing & QA
> Security * Process Improvement & Measurement *
> http://www.sqe.com/bsce5sf
> _______________________________________________
> Simpleweb-Support mailing list
> Sim...@li...
>
https://lists.sourceforge.net/lists/listinfo/simpleweb-support
>


Niall Gallagher



____________________________________________________
Start your day with Yahoo! - make it your home page
http://www.yahoo.com/r/hs



-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
Simpleweb-Support mailing list
Sim...@li...
https://lists.sourceforge.net/lists/listinfo/simpleweb-support

RE: [Simpleweb-Support] FIX: Simple stops responding

[Niall G. <gal...@ya...>]

Hi Jorge,

Apologies for contacting you directly. However, I
think I have found a solution, but I don't want to be
too hasty. What triggered this is when you said that
at least 35 threads had to be involved. 

So, I downloaded the simple-test.tar.gz and resized
the ProcessQueue, this caused a deadlock with the very
same test case. So if you could be so kind as to check
to see if this works for you! In the file
ProcessQueue.java change the following line.

this.ready = new ObjectQueue(size);

to 

this.ready = new ObjectQueue(100);

Making sure that you dont resize beyond 100. I think
the problem is that the ProcessQueue.resize does not
resize the ObjectQueue. This seems to work for my
tests?

Niall



--- Jorge CAMPOS <Jor...@ge...> wrote:

> Hi everybody!
> 
> From our tests with our application, it seems that
> Simple hangs only when
> using at least 35 threads in parallel (we're testing
> in a 4Way Xeon with HT
> running SuSE SLES 9.0 and JDK 1.5.0_04). From the
> thread dump and some
> printlns in the code it seems that one thread
> (dequeuer) blocks because
> there are no more requests to be processed. After
> that everything starts
> piling up (all the dequeuers wait for the first one
> and the enqueuers too).
> I know it sound a bit weird, but it could be related
> to the locking queue
> implementation.
> 
> I've tried to reproduce the problem using the demo
> server and I didn't
> succeed. I guess the context is different (to
> simulate some load I've put
> one sleep and some printlns). This made me look
> again at our thread dump...
> and I can't see anything related to our application
> (it seems that Simple
> blocks within its code).
> 
> One question Andrew: what are the specs of your
> machine?
> 
> I'm asking this, because I've seen some locking
> problems in applications
> running with HyperThreading ON.
> 
> We will test the application running with
> HyperThreading off and in another
> machine to see if we can reproduce the problem.
> 
> All your support is greatly appreciated.
> 
> Ah, one more thing. Since in our thread dump it
> seems all the threads are
> still alive, I don't think this particular case
> relates to a out of memory
> exception (also there are no exceptions in the logs)
> 
> 
> Jorge
> 
> -----Original Message-----
> From: sim...@li...
>
[mailto:sim...@li...]On
> Behalf Of Niall
> Gallagher
> Sent: Wednesday, August 03, 2005 10:21 AM
> To: sim...@li...
> Subject: Re: [Simpleweb-Support] FIX: Simple stops
> responding
> 
> 
> Hi,
> 
> I have taken note of the last two comments with
> regard
> to the server freezing! In response to the queue
> implementation, I think that the implementation of
> the
> Scheduler, ObjectQueue, and ProcessQueue involved in
> dispatching requests is fine. There are no problems
> here.
> 
> However, the simple.http.connect implementation may
> be
> at fault, and I have no doubt whatsoever that the
> java.lang.OutOfMemory error will cause the server to
> stop responding under very heavy load.
> 
> I have uploaded my test framework for anyone wishing
> to  use it to see if it will freeze their server. If
> it does then great! let me know what happened.
> However, I have put through well over a 100 million
> requests using my test framework today alone and,
> with
> the catch(Throwable) amendments there is no
> deadlock,
> not even a lost thread.
> 
> The test framework can be downloaded at
> http://simpleweb.sf.net/simple-test.tar.gz.
> 
> So, in conclusion, the application is thread safe
> and
> working properly with catch(Throwable) amendments.
> However, the TCP related issues have not been
> thought
> out carefully! And I believe the simple.http.connect
> package needs to be reworked! I will release all the
> bug fixes as soon as I am convinced everything is
> working.
> 
> Thanks for the bug reports, if anyone thinks the
> freeze is related to somthing else please let me
> know
> so that I can explore it!
> 
> Niall
> 
> 
> Niall Gallagher
> 
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam
> protection around
> http://mail.yahoo.com
> 
> 
>
-------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux
> Migration Strategies
> from IBM. Find simple to follow Roadmaps,
> straightforward articles,
> informative Webcasts and more! Get everything you
> need to get up to
> speed, fast.
>
http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> Simpleweb-Support mailing list
> Sim...@li...
>
https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> 
> 
> 
>
-------------------------------------------------------
> SF.Net email is Sponsored by the Better Software
> Conference & EXPO
> September 19-22, 2005 * San Francisco, CA *
> Development Lifecycle Practices
> Agile & Plan-Driven Development * Managing Projects
> & Teams * Testing & QA
> Security * Process Improvement & Measurement *
> http://www.sqe.com/bsce5sf
> _______________________________________________
> Simpleweb-Support mailing list
> Sim...@li...
>
https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> 


Niall Gallagher


		
____________________________________________________
Start your day with Yahoo! - make it your home page 
http://www.yahoo.com/r/hs 
 

Re: [Simpleweb-Support] FIX: Simple stops responding

[Niall G. <gal...@ya...>]

Hi Everyone,

Just an update on the problem, have to admit that it
is the Scheduler. I can reproduce a deadlock, very
easily, with the following test on a HyperThreading P4
machine.

http://simpleweb.sourceforge.net/SimpleSchedulerTest.java

Ill hopefully be able to resolve this soon! If anyone
can find what the problem is in the meantime I'd
appreciate it!

Niall

--- Andrew Madigan <and...@ya...> wrote:

> Ok, I'm sure it's an absolutely horrible
> implementation but this is what 
> I did (keep in mind I'm not sure this is the same
> problem, but it seems 
> to have the same symptoms).
> 
> public class BlockingQueue    {
>     private Link first;
>     private Link last;
>     private int maxWait = 100;
>        
>     public BlockingQueue() {}
>     public BlockingQueue(int wait)    {
>         maxWait = wait;
>     }
>    
>     public void add(Object o)    {
>         if (o == null) throw new
> NullPointerException();
>         synchronized(this)    {
>             if (first == null)    {
>                 first = new Link(o);
>                 first.timeout = -1;
>                 last = first;
>             }
>             else    {
>                 last.next = new Link(o);
>                 last.next.timeout = -1;
>             }
>             notify();
>         }
>     }
>    
>     public void add(Object o, long wait)    {
>         if (o == null) throw new
> NullPointerException();
>         synchronized(this)    {
>             if (first == null)    {
>                 first = new Link(o);
>                 first.timeout =
> System.currentTimeMillis() + 
> Math.min(wait,maxWait);
>                 last = first;
>             }
>             else    {
>                 last.next = new Link(o);
>                 last.next.timeout =
> System.currentTimeMillis() + 
> Math.min(wait,maxWait);
>             }
>             notify();
>         }
>     }
>    
>     public Object getNext() throws
> InterruptedException    {
>         synchronized (this)    {
>             while (first == null) wait();
>             sleepUntil(first.timeout);
>             Object o = first.value;
>             first = first.next;
>             return o;
>         }
>     }
>    
>     public void clear()    {
>         first = null;
>         last = null;
>     }
>    
>     private class Link {
>         public Link next;
>         public Object value;
>         public long timeout;
>        
>         public Link(Object value) {this.value =
> value;}
>     }
>    
>     protected void sleepUntil(long awaken) throws
> InterruptedException {
>         long time = System.currentTimeMillis();
>         long sleep = awaken - time;
>         if(sleep <= 0) return;
>         Thread.sleep(sleep);
>    }
> }
> 
> Niall Gallagher wrote:
> 
> >Hi Andrew,
> >
> >I would like to see your implementation! I have
> >searched through all classes in the
> >simple.util.schedule package and also tested them,
> and
> >I had not found errors or deadlock situations?
> Perhaps
> >I can learn somthing from your implementation that
> I
> >have overlooked. I would certainly appreciate it!
> >
> >Niall
> >
> >--- Andrew Madigan <and...@ya...> wrote:
> >
> >  
> >
> >>Alright I've taken another look at the the code I
> >>used, this is what 
> >>I've figured out:
> >>- I assumed that simple used
> >>util.schedule.SchedulerQueue and that this 
> >>was causing problems
> >>- I replaced the use of this class (still trying
> to
> >>find where) with a 
> >>different implementation that used a simpler
> method
> >>of blocking
> >>and did not use arrays
> >>- As I said I am no threading expert so there are
> >>probably flaws in my 
> >>queue implementation, but I have never had
> problems
> >>with it and I use it 
> >>in the application that I run on top of simple as
> >>well.
> >>
> >>If you would like me to post the code for the
> queue
> >>I will just let me know.
> >>
> >>Jorge CAMPOS wrote:
> >>
> >>    
> >>
> >>>Hi everybody!
> >>>
> >>>>From our tests with our application, it seems
> that
> >>>      
> >>>
> >>Simple hangs only when
> >>    
> >>
> >>>using at least 35 threads in parallel (we're
> >>>      
> >>>
> >>testing in a 4Way Xeon with HT
> >>    
> >>
> >>>running SuSE SLES 9.0 and JDK 1.5.0_04). From the
> >>>      
> >>>
> >>thread dump and some
> >>    
> >>
> >>>printlns in the code it seems that one thread
> >>>      
> >>>
> >>(dequeuer) blocks because
> >>    
> >>
> >>>there are no more requests to be processed. After
> >>>      
> >>>
> >>that everything starts
> >>    
> >>
> >>>piling up (all the dequeuers wait for the first
> one
> >>>      
> >>>
> >>and the enqueuers too).
> >>    
> >>
> >>>I know it sound a bit weird, but it could be
> >>>      
> >>>
> >>related to the locking queue
> >>    
> >>
> >>>implementation.
> >>>
> >>>I've tried to reproduce the problem using the
> demo
> >>>      
> >>>
> >>server and I didn't
> >>    
> >>
> >>>succeed. I guess the context is different (to
> >>>      
> >>>
> >>simulate some load I've put
> >>    
> >>
> >>>one sleep and some printlns). This made me look
> >>>      
> >>>
> 
=== message truncated ===


Niall Gallagher

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [Simpleweb-Support] FIX: Simple stops responding

[Alex C. <al...@cl...>]

Niall Gallagher wrote:

>Exactly, there are no tests of TCP whatsoever. My goal
>is to test the engine. If the engine works and there
>is no failures then the engine cannot be at fault.
>  
>
>Yes, I am not testing I/O here, I am testing for a
>deadlock situation.
>  
>
"How does your airliner react when an engine fails?"

"Hey, I don't make the engines."

;)

-0xe1a

Re: [Simpleweb-Support] FIX: Simple stops responding

[Andrew M. <and...@ya...>]

Ok, I'm sure it's an absolutely horrible implementation but this is what 
I did (keep in mind I'm not sure this is the same problem, but it seems 
to have the same symptoms).

public class BlockingQueue    {
    private Link first;
    private Link last;
    private int maxWait = 100;
       
    public BlockingQueue() {}
    public BlockingQueue(int wait)    {
        maxWait = wait;
    }
   
    public void add(Object o)    {
        if (o == null) throw new NullPointerException();
        synchronized(this)    {
            if (first == null)    {
                first = new Link(o);
                first.timeout = -1;
                last = first;
            }
            else    {
                last.next = new Link(o);
                last.next.timeout = -1;
            }
            notify();
        }
    }
   
    public void add(Object o, long wait)    {
        if (o == null) throw new NullPointerException();
        synchronized(this)    {
            if (first == null)    {
                first = new Link(o);
                first.timeout = System.currentTimeMillis() + 
Math.min(wait,maxWait);
                last = first;
            }
            else    {
                last.next = new Link(o);
                last.next.timeout = System.currentTimeMillis() + 
Math.min(wait,maxWait);
            }
            notify();
        }
    }
   
    public Object getNext() throws InterruptedException    {
        synchronized (this)    {
            while (first == null) wait();
            sleepUntil(first.timeout);
            Object o = first.value;
            first = first.next;
            return o;
        }
    }
   
    public void clear()    {
        first = null;
        last = null;
    }
   
    private class Link {
        public Link next;
        public Object value;
        public long timeout;
       
        public Link(Object value) {this.value = value;}
    }
   
    protected void sleepUntil(long awaken) throws InterruptedException {
        long time = System.currentTimeMillis();
        long sleep = awaken - time;
        if(sleep <= 0) return;
        Thread.sleep(sleep);
   }
}

Niall Gallagher wrote:

>Hi Andrew,
>
>I would like to see your implementation! I have
>searched through all classes in the
>simple.util.schedule package and also tested them, and
>I had not found errors or deadlock situations? Perhaps
>I can learn somthing from your implementation that I
>have overlooked. I would certainly appreciate it!
>
>Niall
>
>--- Andrew Madigan <and...@ya...> wrote:
>
>  
>
>>Alright I've taken another look at the the code I
>>used, this is what 
>>I've figured out:
>>- I assumed that simple used
>>util.schedule.SchedulerQueue and that this 
>>was causing problems
>>- I replaced the use of this class (still trying to
>>find where) with a 
>>different implementation that used a simpler method
>>of blocking
>>and did not use arrays
>>- As I said I am no threading expert so there are
>>probably flaws in my 
>>queue implementation, but I have never had problems
>>with it and I use it 
>>in the application that I run on top of simple as
>>well.
>>
>>If you would like me to post the code for the queue
>>I will just let me know.
>>
>>Jorge CAMPOS wrote:
>>
>>    
>>
>>>Hi everybody!
>>>
>>>>From our tests with our application, it seems that
>>>      
>>>
>>Simple hangs only when
>>    
>>
>>>using at least 35 threads in parallel (we're
>>>      
>>>
>>testing in a 4Way Xeon with HT
>>    
>>
>>>running SuSE SLES 9.0 and JDK 1.5.0_04). From the
>>>      
>>>
>>thread dump and some
>>    
>>
>>>printlns in the code it seems that one thread
>>>      
>>>
>>(dequeuer) blocks because
>>    
>>
>>>there are no more requests to be processed. After
>>>      
>>>
>>that everything starts
>>    
>>
>>>piling up (all the dequeuers wait for the first one
>>>      
>>>
>>and the enqueuers too).
>>    
>>
>>>I know it sound a bit weird, but it could be
>>>      
>>>
>>related to the locking queue
>>    
>>
>>>implementation.
>>>
>>>I've tried to reproduce the problem using the demo
>>>      
>>>
>>server and I didn't
>>    
>>
>>>succeed. I guess the context is different (to
>>>      
>>>
>>simulate some load I've put
>>    
>>
>>>one sleep and some printlns). This made me look
>>>      
>>>
>>again at our thread dump...
>>    
>>
>>>and I can't see anything related to our application
>>>      
>>>
>>(it seems that Simple
>>    
>>
>>>blocks within its code).
>>>
>>>One question Andrew: what are the specs of your
>>>      
>>>
>>machine?
>>    
>>
>>>I'm asking this, because I've seen some locking
>>>      
>>>
>>problems in applications
>>    
>>
>>>running with HyperThreading ON.
>>>
>>>We will test the application running with
>>>      
>>>
>>HyperThreading off and in another
>>    
>>
>>>machine to see if we can reproduce the problem.
>>>
>>>All your support is greatly appreciated.
>>>
>>>Ah, one more thing. Since in our thread dump it
>>>      
>>>
>>seems all the threads are
>>    
>>
>>>still alive, I don't think this particular case
>>>      
>>>
>>relates to a out of memory
>>    
>>
>>>exception (also there are no exceptions in the
>>>      
>>>
>>logs)
>>    
>>
>>>Jorge
>>>
>>>-----Original Message-----
>>>From: sim...@li...
>>>      
>>>
>>[mailto:sim...@li...]On
>>Behalf Of Niall
>>    
>>
>>>Gallagher
>>>Sent: Wednesday, August 03, 2005 10:21 AM
>>>To: sim...@li...
>>>Subject: Re: [Simpleweb-Support] FIX: Simple stops
>>>      
>>>
>>responding
>>    
>>
>>>Hi,
>>>
>>>I have taken note of the last two comments with
>>>      
>>>
>>regard
>>    
>>
>>>to the server freezing! In response to the queue
>>>implementation, I think that the implementation of
>>>      
>>>
>>the
>>    
>>
>>>Scheduler, ObjectQueue, and ProcessQueue involved
>>>      
>>>
>>in
>>    
>>
>>>dispatching requests is fine. There are no problems
>>>here.
>>>
>>>However, the simple.http.connect implementation may
>>>      
>>>
>>be
>>    
>>
>>>at fault, and I have no doubt whatsoever that the
>>>java.lang.OutOfMemory error will cause the server
>>>      
>>>
>>to
>>    
>>
>>>stop responding under very heavy load.
>>>
>>>I have uploaded my test framework for anyone
>>>      
>>>
>>wishing
>>    
>>
>>>to  use it to see if it will freeze their server.
>>>      
>>>
>>If
>>    
>>
>>>it does then great! let me know what happened.
>>>However, I have put through well over a 100 million
>>>requests using my test framework today alone and,
>>>      
>>>
>>with
>>    
>>
>>>the catch(Throwable) amendments there is no
>>>      
>>>
>>deadlock,
>>    
>>
>>>not even a lost thread.
>>>
>>>The test framework can be downloaded at
>>>http://simpleweb.sf.net/simple-test.tar.gz.
>>>
>>>So, in conclusion, the application is thread safe
>>>      
>>>
>>and
>>    
>>
>>>working properly with catch(Throwable) amendments.
>>>However, the TCP related issues have not been
>>>      
>>>
>>thought
>>    
>>
>>>out carefully! And I believe the
>>>      
>>>
>>simple.http.connect
>>    
>>
>>>package needs to be reworked! I will release all
>>>      
>>>
>>the
>>    
>>
>>>bug fixes as soon as I am convinced everything is
>>>working.
>>>
>>>Thanks for the bug reports, if anyone thinks the
>>>freeze is related to somthing else please let me
>>>      
>>>
>>know
>>    
>>
>>>so that I can explore it!
>>>
>>>Niall
>>>
>>>
>>>Niall Gallagher
>>>
>>>__________________________________________________
>>>Do You Yahoo!?
>>>Tired of spam?  Yahoo! Mail has the best spam
>>>      
>>>
>>protection around
>>    
>>
>>>http://mail.yahoo.com
>>>
>>>
>>>      
>>>
>>-------------------------------------------------------
>>    
>>
>>>SF.Net email is sponsored by: Discover Easy Linux
>>>      
>>>
>>Migration Strategies
>>>from IBM. Find simple to follow Roadmaps,
>>straightforward articles,
>>    
>>
>>>informative Webcasts and more! Get everything you
>>>      
>>>
>>need to get up to
>>    
>>
>>>speed, fast.
>>>      
>>>
>http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
>  
>
>>>_______________________________________________
>>>Simpleweb-Support mailing list
>>>Sim...@li...
>>>      
>>>
>>https://lists.sourceforge.net/lists/listinfo/simpleweb-support
>>    
>>
>>>
>>>      
>>>
>>-------------------------------------------------------
>>    
>>
>>>SF.Net email is Sponsored by the Better Software
>>>      
>>>
>>Conference & EXPO
>>    
>>
>>>September 19-22, 2005 * San Francisco, CA *
>>>      
>>>
>>Development Lifecycle Practices
>>    
>>
>>>Agile & Plan-Driven Development * Managing Projects
>>>      
>>>
>>& Teams * Testing & QA
>>    
>>
>>>Security * Process Improvement & Measurement *
>>>      
>>>
>>http://www.sqe.com/bsce5sf
>>    
>>
>>>_______________________________________________
>>>Simpleweb-Support mailing list
>>>Sim...@li...
>>>      
>>>
>>https://lists.sourceforge.net/lists/listinfo/simpleweb-support
>>    
>>
>>> 
>>>
>>>      
>>>
>>
>>
>>    
>>
>-------------------------------------------------------
>  
>
>>SF.Net email is Sponsored by the Better Software
>>Conference & EXPO
>>September 19-22, 2005 * San Francisco, CA *
>>Development Lifecycle Practices
>>Agile & Plan-Driven Development * Managing Projects
>>& Teams * Testing & QA
>>Security * Process Improvement & Measurement *
>>http://www.sqe.com/bsce5sf
>>_______________________________________________
>>Simpleweb-Support mailing list
>>Sim...@li...
>>
>>    
>>
>https://lists.sourceforge.net/lists/listinfo/simpleweb-support
>  
>
>
>
>Niall Gallagher
>
>
>		
>____________________________________________________
>Start your day with Yahoo! - make it your home page 
>http://www.yahoo.com/r/hs 
> 
>
>
>-------------------------------------------------------
>SF.Net email is Sponsored by the Better Software Conference & EXPO
>September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
>Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
>Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
>_______________________________________________
>Simpleweb-Support mailing list
>Sim...@li...
>https://lists.sourceforge.net/lists/listinfo/simpleweb-support
>
>  
>

Re: [Simpleweb-Support] FIX: Simple stops responding

[Niall G. <gal...@ya...>]

Hi Andrew,

I would like to see your implementation! I have
searched through all classes in the
simple.util.schedule package and also tested them, and
I had not found errors or deadlock situations? Perhaps
I can learn somthing from your implementation that I
have overlooked. I would certainly appreciate it!

Niall

--- Andrew Madigan <and...@ya...> wrote:

> Alright I've taken another look at the the code I
> used, this is what 
> I've figured out:
> - I assumed that simple used
> util.schedule.SchedulerQueue and that this 
> was causing problems
> - I replaced the use of this class (still trying to
> find where) with a 
> different implementation that used a simpler method
> of blocking
> and did not use arrays
> - As I said I am no threading expert so there are
> probably flaws in my 
> queue implementation, but I have never had problems
> with it and I use it 
> in the application that I run on top of simple as
> well.
> 
> If you would like me to post the code for the queue
> I will just let me know.
> 
> Jorge CAMPOS wrote:
> 
> >Hi everybody!
> >
> >>From our tests with our application, it seems that
> Simple hangs only when
> >using at least 35 threads in parallel (we're
> testing in a 4Way Xeon with HT
> >running SuSE SLES 9.0 and JDK 1.5.0_04). From the
> thread dump and some
> >printlns in the code it seems that one thread
> (dequeuer) blocks because
> >there are no more requests to be processed. After
> that everything starts
> >piling up (all the dequeuers wait for the first one
> and the enqueuers too).
> >I know it sound a bit weird, but it could be
> related to the locking queue
> >implementation.
> >
> >I've tried to reproduce the problem using the demo
> server and I didn't
> >succeed. I guess the context is different (to
> simulate some load I've put
> >one sleep and some printlns). This made me look
> again at our thread dump...
> >and I can't see anything related to our application
> (it seems that Simple
> >blocks within its code).
> >
> >One question Andrew: what are the specs of your
> machine?
> >
> >I'm asking this, because I've seen some locking
> problems in applications
> >running with HyperThreading ON.
> >
> >We will test the application running with
> HyperThreading off and in another
> >machine to see if we can reproduce the problem.
> >
> >All your support is greatly appreciated.
> >
> >Ah, one more thing. Since in our thread dump it
> seems all the threads are
> >still alive, I don't think this particular case
> relates to a out of memory
> >exception (also there are no exceptions in the
> logs)
> >
> >
> >Jorge
> >
> >-----Original Message-----
> >From: sim...@li...
>
>[mailto:sim...@li...]On
> Behalf Of Niall
> >Gallagher
> >Sent: Wednesday, August 03, 2005 10:21 AM
> >To: sim...@li...
> >Subject: Re: [Simpleweb-Support] FIX: Simple stops
> responding
> >
> >
> >Hi,
> >
> >I have taken note of the last two comments with
> regard
> >to the server freezing! In response to the queue
> >implementation, I think that the implementation of
> the
> >Scheduler, ObjectQueue, and ProcessQueue involved
> in
> >dispatching requests is fine. There are no problems
> >here.
> >
> >However, the simple.http.connect implementation may
> be
> >at fault, and I have no doubt whatsoever that the
> >java.lang.OutOfMemory error will cause the server
> to
> >stop responding under very heavy load.
> >
> >I have uploaded my test framework for anyone
> wishing
> >to  use it to see if it will freeze their server.
> If
> >it does then great! let me know what happened.
> >However, I have put through well over a 100 million
> >requests using my test framework today alone and,
> with
> >the catch(Throwable) amendments there is no
> deadlock,
> >not even a lost thread.
> >
> >The test framework can be downloaded at
> >http://simpleweb.sf.net/simple-test.tar.gz.
> >
> >So, in conclusion, the application is thread safe
> and
> >working properly with catch(Throwable) amendments.
> >However, the TCP related issues have not been
> thought
> >out carefully! And I believe the
> simple.http.connect
> >package needs to be reworked! I will release all
> the
> >bug fixes as soon as I am convinced everything is
> >working.
> >
> >Thanks for the bug reports, if anyone thinks the
> >freeze is related to somthing else please let me
> know
> >so that I can explore it!
> >
> >Niall
> >
> >
> >Niall Gallagher
> >
> >__________________________________________________
> >Do You Yahoo!?
> >Tired of spam?  Yahoo! Mail has the best spam
> protection around
> >http://mail.yahoo.com
> >
> >
>
>-------------------------------------------------------
> >SF.Net email is sponsored by: Discover Easy Linux
> Migration Strategies
> >from IBM. Find simple to follow Roadmaps,
> straightforward articles,
> >informative Webcasts and more! Get everything you
> need to get up to
> >speed, fast.
>
http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> >_______________________________________________
> >Simpleweb-Support mailing list
> >Sim...@li...
>
>https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> >
> >
> >
>
>-------------------------------------------------------
> >SF.Net email is Sponsored by the Better Software
> Conference & EXPO
> >September 19-22, 2005 * San Francisco, CA *
> Development Lifecycle Practices
> >Agile & Plan-Driven Development * Managing Projects
> & Teams * Testing & QA
> >Security * Process Improvement & Measurement *
> http://www.sqe.com/bsce5sf
> >_______________________________________________
> >Simpleweb-Support mailing list
> >Sim...@li...
>
>https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> >
> >  
> >
> 
> 
> 
>
-------------------------------------------------------
> SF.Net email is Sponsored by the Better Software
> Conference & EXPO
> September 19-22, 2005 * San Francisco, CA *
> Development Lifecycle Practices
> Agile & Plan-Driven Development * Managing Projects
> & Teams * Testing & QA
> Security * Process Improvement & Measurement *
> http://www.sqe.com/bsce5sf
> _______________________________________________
> Simpleweb-Support mailing list
> Sim...@li...
>
https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> 


Niall Gallagher


		
____________________________________________________
Start your day with Yahoo! - make it your home page 
http://www.yahoo.com/r/hs 
 

Re: [Simpleweb-Support] FIX: Simple stops responding

[Andrew M. <and...@ya...>]

Alright I've taken another look at the the code I used, this is what 
I've figured out:
- I assumed that simple used util.schedule.SchedulerQueue and that this 
was causing problems
- I replaced the use of this class (still trying to find where) with a 
different implementation that used a simpler method of blocking
and did not use arrays
- As I said I am no threading expert so there are probably flaws in my 
queue implementation, but I have never had problems with it and I use it 
in the application that I run on top of simple as well.

If you would like me to post the code for the queue I will just let me know.

Jorge CAMPOS wrote:

>Hi everybody!
>
>>From our tests with our application, it seems that Simple hangs only when
>using at least 35 threads in parallel (we're testing in a 4Way Xeon with HT
>running SuSE SLES 9.0 and JDK 1.5.0_04). From the thread dump and some
>printlns in the code it seems that one thread (dequeuer) blocks because
>there are no more requests to be processed. After that everything starts
>piling up (all the dequeuers wait for the first one and the enqueuers too).
>I know it sound a bit weird, but it could be related to the locking queue
>implementation.
>
>I've tried to reproduce the problem using the demo server and I didn't
>succeed. I guess the context is different (to simulate some load I've put
>one sleep and some printlns). This made me look again at our thread dump...
>and I can't see anything related to our application (it seems that Simple
>blocks within its code).
>
>One question Andrew: what are the specs of your machine?
>
>I'm asking this, because I've seen some locking problems in applications
>running with HyperThreading ON.
>
>We will test the application running with HyperThreading off and in another
>machine to see if we can reproduce the problem.
>
>All your support is greatly appreciated.
>
>Ah, one more thing. Since in our thread dump it seems all the threads are
>still alive, I don't think this particular case relates to a out of memory
>exception (also there are no exceptions in the logs)
>
>
>Jorge
>
>-----Original Message-----
>From: sim...@li...
>[mailto:sim...@li...]On Behalf Of Niall
>Gallagher
>Sent: Wednesday, August 03, 2005 10:21 AM
>To: sim...@li...
>Subject: Re: [Simpleweb-Support] FIX: Simple stops responding
>
>
>Hi,
>
>I have taken note of the last two comments with regard
>to the server freezing! In response to the queue
>implementation, I think that the implementation of the
>Scheduler, ObjectQueue, and ProcessQueue involved in
>dispatching requests is fine. There are no problems
>here.
>
>However, the simple.http.connect implementation may be
>at fault, and I have no doubt whatsoever that the
>java.lang.OutOfMemory error will cause the server to
>stop responding under very heavy load.
>
>I have uploaded my test framework for anyone wishing
>to  use it to see if it will freeze their server. If
>it does then great! let me know what happened.
>However, I have put through well over a 100 million
>requests using my test framework today alone and, with
>the catch(Throwable) amendments there is no deadlock,
>not even a lost thread.
>
>The test framework can be downloaded at
>http://simpleweb.sf.net/simple-test.tar.gz.
>
>So, in conclusion, the application is thread safe and
>working properly with catch(Throwable) amendments.
>However, the TCP related issues have not been thought
>out carefully! And I believe the simple.http.connect
>package needs to be reworked! I will release all the
>bug fixes as soon as I am convinced everything is
>working.
>
>Thanks for the bug reports, if anyone thinks the
>freeze is related to somthing else please let me know
>so that I can explore it!
>
>Niall
>
>
>Niall Gallagher
>
>__________________________________________________
>Do You Yahoo!?
>Tired of spam?  Yahoo! Mail has the best spam protection around
>http://mail.yahoo.com
>
>
>-------------------------------------------------------
>SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
>from IBM. Find simple to follow Roadmaps, straightforward articles,
>informative Webcasts and more! Get everything you need to get up to
>speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
>_______________________________________________
>Simpleweb-Support mailing list
>Sim...@li...
>https://lists.sourceforge.net/lists/listinfo/simpleweb-support
>
>
>
>-------------------------------------------------------
>SF.Net email is Sponsored by the Better Software Conference & EXPO
>September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
>Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
>Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
>_______________________________________________
>Simpleweb-Support mailing list
>Sim...@li...
>https://lists.sourceforge.net/lists/listinfo/simpleweb-support
>
>  
>

Re: [Simpleweb-Support] FIX: Simple stops responding

[Andrew M. <and...@ya...>]

The system I was testing on was a 2.0 GHz Celeron, so no 
hyper-threading. I'm going to take another look at the code and see if I 
can remember where I thought the problem was. I remember that one change 
I made by changing queues was that I did not fully implement the 
interface of the class I was replacing. The blocking queue I used was an 
exteremely simple one, but it never caused any problems. I was running 
on Debian Sid (and later Ubuntu Hoary), I experienced problems on 
SableVM, GIJ, JDK 1.4, and JDK 1.5. I am hardly an expert at threading, 
but I looked at the blocking queue implementation (when the server 
started to hang), said "this is way too complicated" and replaced it.

Jorge CAMPOS wrote:

>Hi everybody!
>
>>From our tests with our application, it seems that Simple hangs only when
>using at least 35 threads in parallel (we're testing in a 4Way Xeon with HT
>running SuSE SLES 9.0 and JDK 1.5.0_04). From the thread dump and some
>printlns in the code it seems that one thread (dequeuer) blocks because
>there are no more requests to be processed. After that everything starts
>piling up (all the dequeuers wait for the first one and the enqueuers too).
>I know it sound a bit weird, but it could be related to the locking queue
>implementation.
>
>I've tried to reproduce the problem using the demo server and I didn't
>succeed. I guess the context is different (to simulate some load I've put
>one sleep and some printlns). This made me look again at our thread dump...
>and I can't see anything related to our application (it seems that Simple
>blocks within its code).
>
>One question Andrew: what are the specs of your machine?
>
>I'm asking this, because I've seen some locking problems in applications
>running with HyperThreading ON.
>
>We will test the application running with HyperThreading off and in another
>machine to see if we can reproduce the problem.
>
>All your support is greatly appreciated.
>
>Ah, one more thing. Since in our thread dump it seems all the threads are
>still alive, I don't think this particular case relates to a out of memory
>exception (also there are no exceptions in the logs)
>
>
>Jorge
>
>-----Original Message-----
>From: sim...@li...
>[mailto:sim...@li...]On Behalf Of Niall
>Gallagher
>Sent: Wednesday, August 03, 2005 10:21 AM
>To: sim...@li...
>Subject: Re: [Simpleweb-Support] FIX: Simple stops responding
>
>
>Hi,
>
>I have taken note of the last two comments with regard
>to the server freezing! In response to the queue
>implementation, I think that the implementation of the
>Scheduler, ObjectQueue, and ProcessQueue involved in
>dispatching requests is fine. There are no problems
>here.
>
>However, the simple.http.connect implementation may be
>at fault, and I have no doubt whatsoever that the
>java.lang.OutOfMemory error will cause the server to
>stop responding under very heavy load.
>
>I have uploaded my test framework for anyone wishing
>to  use it to see if it will freeze their server. If
>it does then great! let me know what happened.
>However, I have put through well over a 100 million
>requests using my test framework today alone and, with
>the catch(Throwable) amendments there is no deadlock,
>not even a lost thread.
>
>The test framework can be downloaded at
>http://simpleweb.sf.net/simple-test.tar.gz.
>
>So, in conclusion, the application is thread safe and
>working properly with catch(Throwable) amendments.
>However, the TCP related issues have not been thought
>out carefully! And I believe the simple.http.connect
>package needs to be reworked! I will release all the
>bug fixes as soon as I am convinced everything is
>working.
>
>Thanks for the bug reports, if anyone thinks the
>freeze is related to somthing else please let me know
>so that I can explore it!
>
>Niall
>
>
>Niall Gallagher
>
>__________________________________________________
>Do You Yahoo!?
>Tired of spam?  Yahoo! Mail has the best spam protection around
>http://mail.yahoo.com
>
>
>-------------------------------------------------------
>SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
>from IBM. Find simple to follow Roadmaps, straightforward articles,
>informative Webcasts and more! Get everything you need to get up to
>speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
>_______________________________________________
>Simpleweb-Support mailing list
>Sim...@li...
>https://lists.sourceforge.net/lists/listinfo/simpleweb-support
>
>
>
>-------------------------------------------------------
>SF.Net email is Sponsored by the Better Software Conference & EXPO
>September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
>Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
>Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
>_______________________________________________
>Simpleweb-Support mailing list
>Sim...@li...
>https://lists.sourceforge.net/lists/listinfo/simpleweb-support
>
>  
>

RE: [Simpleweb-Support] FIX: Simple stops responding

[Jorge C. <Jor...@ge...>]

Hi everybody!

From our tests with our application, it seems that Simple hangs only when
using at least 35 threads in parallel (we're testing in a 4Way Xeon with HT
running SuSE SLES 9.0 and JDK 1.5.0_04). From the thread dump and some
printlns in the code it seems that one thread (dequeuer) blocks because
there are no more requests to be processed. After that everything starts
piling up (all the dequeuers wait for the first one and the enqueuers too).
I know it sound a bit weird, but it could be related to the locking queue
implementation.

I've tried to reproduce the problem using the demo server and I didn't
succeed. I guess the context is different (to simulate some load I've put
one sleep and some printlns). This made me look again at our thread dump...
and I can't see anything related to our application (it seems that Simple
blocks within its code).

One question Andrew: what are the specs of your machine?

I'm asking this, because I've seen some locking problems in applications
running with HyperThreading ON.

We will test the application running with HyperThreading off and in another
machine to see if we can reproduce the problem.

All your support is greatly appreciated.

Ah, one more thing. Since in our thread dump it seems all the threads are
still alive, I don't think this particular case relates to a out of memory
exception (also there are no exceptions in the logs)


Jorge

-----Original Message-----
From: sim...@li...
[mailto:sim...@li...]On Behalf Of Niall
Gallagher
Sent: Wednesday, August 03, 2005 10:21 AM
To: sim...@li...
Subject: Re: [Simpleweb-Support] FIX: Simple stops responding


Hi,

I have taken note of the last two comments with regard
to the server freezing! In response to the queue
implementation, I think that the implementation of the
Scheduler, ObjectQueue, and ProcessQueue involved in
dispatching requests is fine. There are no problems
here.

However, the simple.http.connect implementation may be
at fault, and I have no doubt whatsoever that the
java.lang.OutOfMemory error will cause the server to
stop responding under very heavy load.

I have uploaded my test framework for anyone wishing
to  use it to see if it will freeze their server. If
it does then great! let me know what happened.
However, I have put through well over a 100 million
requests using my test framework today alone and, with
the catch(Throwable) amendments there is no deadlock,
not even a lost thread.

The test framework can be downloaded at
http://simpleweb.sf.net/simple-test.tar.gz.

So, in conclusion, the application is thread safe and
working properly with catch(Throwable) amendments.
However, the TCP related issues have not been thought
out carefully! And I believe the simple.http.connect
package needs to be reworked! I will release all the
bug fixes as soon as I am convinced everything is
working.

Thanks for the bug reports, if anyone thinks the
freeze is related to somthing else please let me know
so that I can explore it!

Niall


Niall Gallagher

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
Simpleweb-Support mailing list
Sim...@li...
https://lists.sourceforge.net/lists/listinfo/simpleweb-support

Re: [Simpleweb-Support] util.schedule [was Re: ... hang ...]

[Niall G. <gal...@ya...>]

Hi Michael,

With respect to the Scheduler, there is a maximum wait
timeout, if the time you request to wait for is longer
than the maximum wait setting then the wait time is
capped.

I used the pulse scheduler because it reduces the
number of thread interrupts required. I found that
when there were a lot of objects being scheduled the
thread interrupts are quite an overhead. With the
pulse scheduler an interrupt can be shared over a
number of objects (potentially a very large number).

I will have a look at the simple.util.schedule
package, it has been a while since I last tested it.
Might be good to write a few tests for the scheduler!

Thanks,
Niall

--- "Michael \"Luni\" Libes"
<lun...@ya...> wrote:

> Niall,
> 
> Thanks for your quick support.
> 
> I spent a great deal of time reading through the
> Simple source
> code in the last week.  I didn't find anything wrong
> with the
> simple.http.connect classes, albeit the sheer number
> of classes
> involved in connections seems a bit too much.
> 
> What I did find was that
> simple.util.schedule.Scheduler did not
> seem to work as advertised.  When looking for the
> "hang" bug, I
> first attempted to ensure that partial queries were
> not
> deadlocking the request queue.  What I found was
> that with a
> single request, the Scheduler was ignoring the
> exponentially
> increasing wait times.  The sleep() method seemed to
> always drop
> these times to near zero and thus instead of waiting
> for up to
> 120 seconds to receive the request, the actual time
> was closer
> to 3 seconds.
> 
> Maybe this is just a small bug in sleep()?  Or is
> there a
> motivation behind this code?
> 
> And similarly, the default Scheduler is the
> PulseScheduler. 
> That design I did not even try and understand. 
> Instead, I
> changed simple.http.PipelineProcessor to instantiate
> a
> Scheduler.
> 
>   Luni
> 
> --- Niall Gallagher <gal...@ya...>
> wrote:
> 
> > Hi,
> > 
> > I have taken note of the last two comments with
> regard
> > to the server freezing! In response to the queue
> > implementation, I think that the implementation of
> the
> > Scheduler, ObjectQueue, and ProcessQueue involved
> in
> > dispatching requests is fine. There are no
> problems
> > here.
> > 
> > However, the simple.http.connect implementation
> may be
> > at fault, and I have no doubt whatsoever that the
> > java.lang.OutOfMemory error will cause the server
> to
> > stop responding under very heavy load. 
> > 
> > I have uploaded my test framework for anyone
> wishing
> > to  use it to see if it will freeze their server.
> If
> > it does then great! let me know what happened.
> > However, I have put through well over a 100
> million
> > requests using my test framework today alone and,
> with
> > the catch(Throwable) amendments there is no
> deadlock,
> > not even a lost thread.
> > 
> > The test framework can be downloaded at
> > http://simpleweb.sf.net/simple-test.tar.gz.
> > 
> > So, in conclusion, the application is thread safe
> and
> > working properly with catch(Throwable) amendments.
> > However, the TCP related issues have not been
> thought
> > out carefully! And I believe the
> simple.http.connect
> > package needs to be reworked! I will release all
> the
> > bug fixes as soon as I am convinced everything is
> > working.
> > 
> > Thanks for the bug reports, if anyone thinks the
> > freeze is related to somthing else please let me
> know
> > so that I can explore it!
> > 
> > Niall
> > 
> > 
> > Niall Gallagher
> > 
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam?  Yahoo! Mail has the best spam
> protection
> > around 
> > http://mail.yahoo.com 
> > 
> > 
> >
>
-------------------------------------------------------
> > SF.Net email is sponsored by: Discover Easy Linux
> Migration
> > Strategies
> > from IBM. Find simple to follow Roadmaps,
> straightforward
> > articles,
> > informative Webcasts and more! Get everything you
> need to get
> > up to
> > speed, fast.
> >
>
http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> > _______________________________________________
> > Simpleweb-Support mailing list
> > Sim...@li...
> >
>
https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> > 
> 
> 
> 
>
-------------------------------------------------------
> SF.Net email is Sponsored by the Better Software
> Conference & EXPO
> September 19-22, 2005 * San Francisco, CA *
> Development Lifecycle Practices
> Agile & Plan-Driven Development * Managing Projects
> & Teams * Testing & QA
> Security * Process Improvement & Measurement *
> http://www.sqe.com/bsce5sf
> _______________________________________________
> Simpleweb-Support mailing list
> Sim...@li...
>
https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> 


Niall Gallagher

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

[Simpleweb-Support] util.schedule [was Re: ... hang ...]

[Michael \Luni\ L. <lun...@ya...>]

Niall,

Thanks for your quick support.

I spent a great deal of time reading through the Simple source
code in the last week.  I didn't find anything wrong with the
simple.http.connect classes, albeit the sheer number of classes
involved in connections seems a bit too much.

What I did find was that simple.util.schedule.Scheduler did not
seem to work as advertised.  When looking for the "hang" bug, I
first attempted to ensure that partial queries were not
deadlocking the request queue.  What I found was that with a
single request, the Scheduler was ignoring the exponentially
increasing wait times.  The sleep() method seemed to always drop
these times to near zero and thus instead of waiting for up to
120 seconds to receive the request, the actual time was closer
to 3 seconds.

Maybe this is just a small bug in sleep()?  Or is there a
motivation behind this code?

And similarly, the default Scheduler is the PulseScheduler. 
That design I did not even try and understand.  Instead, I
changed simple.http.PipelineProcessor to instantiate a
Scheduler.

  Luni

--- Niall Gallagher <gal...@ya...> wrote:

> Hi,
> 
> I have taken note of the last two comments with regard
> to the server freezing! In response to the queue
> implementation, I think that the implementation of the
> Scheduler, ObjectQueue, and ProcessQueue involved in
> dispatching requests is fine. There are no problems
> here.
> 
> However, the simple.http.connect implementation may be
> at fault, and I have no doubt whatsoever that the
> java.lang.OutOfMemory error will cause the server to
> stop responding under very heavy load. 
> 
> I have uploaded my test framework for anyone wishing
> to  use it to see if it will freeze their server. If
> it does then great! let me know what happened.
> However, I have put through well over a 100 million
> requests using my test framework today alone and, with
> the catch(Throwable) amendments there is no deadlock,
> not even a lost thread.
> 
> The test framework can be downloaded at
> http://simpleweb.sf.net/simple-test.tar.gz.
> 
> So, in conclusion, the application is thread safe and
> working properly with catch(Throwable) amendments.
> However, the TCP related issues have not been thought
> out carefully! And I believe the simple.http.connect
> package needs to be reworked! I will release all the
> bug fixes as soon as I am convinced everything is
> working.
> 
> Thanks for the bug reports, if anyone thinks the
> freeze is related to somthing else please let me know
> so that I can explore it!
> 
> Niall
> 
> 
> Niall Gallagher
> 
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection
> around 
> http://mail.yahoo.com 
> 
> 
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration
> Strategies
> from IBM. Find simple to follow Roadmaps, straightforward
> articles,
> informative Webcasts and more! Get everything you need to get
> up to
> speed, fast.
> http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> Simpleweb-Support mailing list
> Sim...@li...
> https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> 

RE: [Simpleweb-Support] FIX: Simple stops responding

[Niall G. <gal...@ya...>]

Hi Gail

> Your test framework doesn't seem to test these
> real-world use cases of an
> http server:
> 	* requests for very large documents
> 	* requests for streamed content
> 	* interrupted or cancelled client requests (i.e.
> the CLOSE_WAIT
> scenario)
> 	* failures! where are the simulations of common
> http failures?

Exactly, there are no tests of TCP whatsoever. My goal
is to test the engine. If the engine works and there
is no failures then the engine cannot be at fault.

> Simply testing 100 million "friendly" HTTP requests
> (requests for existing
> content of small size) doesn't test aggressive
> applications of your product.

Yes, I am not testing I/O here, I am testing for a
deadlock situation.

> Because Simple is so extensible, it's reasonable to
> assume that it will be
> embedded into applications that are far more
> interesting than just serving
> HTML...

Yes, I would hope so!

> Since the CLOSE_WAIT fix, Simple is working very
> well for me now. If I ever
> need to create a testing framework that stresses the
> use cases I mentioned
> above, I'll contribute it back to your project...

I would really appreciate it, I have written so many
test frameworks in the past and gotten rid of them
all. Mabye I should start a simple-test package??

Thank you for your feedback, especially with regard to
the simple.http.connect suggestion on SO_LINGER.

Regards,
Niall


Niall Gallagher


		
____________________________________________________
Start your day with Yahoo! - make it your home page 
http://www.yahoo.com/r/hs 
 

RE: [Simpleweb-Support] FIX: Simple stops responding

[Gail R. F. <ga...@sc...>]

Hi Niall,

Your test framework doesn't seem to test these real-world use cases of =
an
http server:
	* requests for very large documents
	* requests for streamed content
	* interrupted or cancelled client requests (i.e. the CLOSE_WAIT
scenario)
	* failures! where are the simulations of common http failures?

Simply testing 100 million "friendly" HTTP requests (requests for =
existing
content of small size) doesn't test aggressive applications of your =
product.
Because Simple is so extensible, it's reasonable to assume that it will =
be
embedded into applications that are far more interesting than just =
serving
HTML...

Since the CLOSE_WAIT fix, Simple is working very well for me now. If I =
ever
need to create a testing framework that stresses the use cases I =
mentioned
above, I'll contribute it back to your project...

-- Gail.

-------------
Gail Rahn Frederick
ga...@sc...
503.260.0910

Re: [Simpleweb-Support] FIX: Simple stops responding

[Niall G. <gal...@ya...>]

Hi,

I have taken note of the last two comments with regard
to the server freezing! In response to the queue
implementation, I think that the implementation of the
Scheduler, ObjectQueue, and ProcessQueue involved in
dispatching requests is fine. There are no problems
here.

However, the simple.http.connect implementation may be
at fault, and I have no doubt whatsoever that the
java.lang.OutOfMemory error will cause the server to
stop responding under very heavy load. 

I have uploaded my test framework for anyone wishing
to  use it to see if it will freeze their server. If
it does then great! let me know what happened.
However, I have put through well over a 100 million
requests using my test framework today alone and, with
the catch(Throwable) amendments there is no deadlock,
not even a lost thread.

The test framework can be downloaded at
http://simpleweb.sf.net/simple-test.tar.gz.

So, in conclusion, the application is thread safe and
working properly with catch(Throwable) amendments.
However, the TCP related issues have not been thought
out carefully! And I believe the simple.http.connect
package needs to be reworked! I will release all the
bug fixes as soon as I am convinced everything is
working.

Thanks for the bug reports, if anyone thinks the
freeze is related to somthing else please let me know
so that I can explore it!

Niall


Niall Gallagher

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [Simpleweb-Support] FIX: Simple stops responding

[Andrew M. <and...@ya...>]

Two parts to this message, the second part is more important.

Hmm, I've been reading these for a while and this caught my attention.
I haven't actively used Simple in about a month but I seem to remember
that I started having a problem with a specific application that I
wrote to run on Simple. The problem was that the server eventually
stopped responding and I found the problem very hard to track down.
Then I noticed the blocking queue that seemed to deal with reading data
from requests.

The presence of this queue made no sense to me but I eventually found
(by inserting debugging lines) that this was where the server was
stopping. To fix it I replaced the queue implementation with my own and
poof it worked. Maybe these two are unrelated and maybe I'm crazy, but
it sounds like a similar problem. The error was also platform
dependent, I only noticed it on (debian) linux.

Second Part:
I note that this problem does not seem to happen with Tomcat and I can
find no part of the API that allows for this setting in Coyote. Since
Simple and Coyote seem to have similar APIs this seems strange somehow.


--- Gail Rahn Frederick <ga...@sc...> wrote:

> Hi,
> 
> I am a colleague of Michael "Luni" Libes, who recently reported a bug
> in
> Simple where the server "hangs" when many sockets get stuck in the
> CLOSE_WAIT state. We have consistently been able to reproduce this
> behavior
> from J2ME and wireline clients (i.e. Firefox).
> 
> Niall's suggestion to catch Throwables instead of Exceptions in
> simple.http.Processor and simple.util.process.Daemon didn't affect
> our
> problem.
> 
> I wanted to share my modification to Simple that fixed our issue. But
> first,
> I want to stress the OS-dependent nature of my solution (and TCP
> stacks in
> particular). My fix works on Red Hat Linux ES 3. I have not tested
> nor do I
> assume it works on any other platform.
> 
> The underlying bug here is what happens when a server attempts to
> write to a
> socket that is already closed by the client. That is the CLOSE_WAIT
> scenario. The TCP standard is ambiguous here. The OS and application
> must
> decide, when a client doesn't ACK, whether to block forever until the
> write
> queue is drained OR timeout after some duration. (IMHO, the obviously
> choice
> is NOT to block forever but alas ...)
> 
> On Linux, the default behavior seems to block forever when attempting
> to
> write() to a socket that is closed on the other end. The application
> must
> adjust the SO_LINGER socket property to force a timeout.
> 
> I updated one line of the configure() method of
> simple.http.connect.Processor. 
> 
> This line (line ~154):
>       sock.setSoLinger(false,0);
> 
> Became this line:
>       sock.setSoLinger(true,5);
> 
> This change means that if sock.close() is called with a non-empty
> write
> queue, after 5 seconds the server-side socket will dump its queue and
> be
> forced to close. This change forces clients to be greedy (read data
> when
> received) or lose the data. I also understand this bugfix to change
> Linux's
> socket behavior from "don't notice a lingering idle socket" to
> "notice the
> socket, handle it after N seconds of lingering".
> 
> With this change, our CLOSE_WAITs all disappeared.
> 
> This behavior is surely platform-dependent and application-dependent
> (your
> app might allow more latency) so the best choice for Simple may be to
> allow
> external configuration of this properly.
> 
> Thanks for reading all the way to the bottom,
> 
> -- Gail.
> 
> -------------
> Gail Rahn Frederick
> ga...@sc...
> 503.260.0910
> 
> 
> 
> 
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration
> Strategies
> from IBM. Find simple to follow Roadmaps, straightforward articles,
> informative Webcasts and more! Get everything you need to get up to
> speed, fast. http://ads.osdn.com/?ad_idt77&alloc_id492&op=click
> _______________________________________________
> Simpleweb-Support mailing list
> Sim...@li...
> https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> 


Mozilla Firefox - Fast, Secure, Powerful Web Browser
Get it at www.mozilla.org for Free!
Firefox Users - Put Firefox in your tagline/signature!

[Simpleweb-Support] FIX: Simple stops responding

[Gail R. F. <ga...@sc...>]

Hi,

I am a colleague of Michael "Luni" Libes, who recently reported a bug in
Simple where the server "hangs" when many sockets get stuck in the
CLOSE_WAIT state. We have consistently been able to reproduce this =
behavior
from J2ME and wireline clients (i.e. Firefox).

Niall's suggestion to catch Throwables instead of Exceptions in
simple.http.Processor and simple.util.process.Daemon didn't affect our
problem.

I wanted to share my modification to Simple that fixed our issue. But =
first,
I want to stress the OS-dependent nature of my solution (and TCP stacks =
in
particular). My fix works on Red Hat Linux ES 3. I have not tested nor =
do I
assume it works on any other platform.

The underlying bug here is what happens when a server attempts to write =
to a
socket that is already closed by the client. That is the CLOSE_WAIT
scenario. The TCP standard is ambiguous here. The OS and application =
must
decide, when a client doesn't ACK, whether to block forever until the =
write
queue is drained OR timeout after some duration. (IMHO, the obviously =
choice
is NOT to block forever but alas ...)

On Linux, the default behavior seems to block forever when attempting to
write() to a socket that is closed on the other end. The application =
must
adjust the SO_LINGER socket property to force a timeout.

I updated one line of the configure() method of
simple.http.connect.Processor.=20

This line (line ~154):
      sock.setSoLinger(false,0);

Became this line:
      sock.setSoLinger(true,5);

This change means that if sock.close() is called with a non-empty write
queue, after 5 seconds the server-side socket will dump its queue and be
forced to close. This change forces clients to be greedy (read data when
received) or lose the data. I also understand this bugfix to change =
Linux's
socket behavior from "don't notice a lingering idle socket" to "notice =
the
socket, handle it after N seconds of lingering".

With this change, our CLOSE_WAITs all disappeared.

This behavior is surely platform-dependent and application-dependent =
(your
app might allow more latency) so the best choice for Simple may be to =
allow
external configuration of this properly.

Thanks for reading all the way to the bottom,

-- Gail.

-------------
Gail Rahn Frederick
ga...@sc...
503.260.0910

Re: [Simpleweb-Support] Too many open files

[Niall G. <gal...@ya...>]

Hi Jan,

I have been examining the source of simple quite a bit
recently. I can see no cause for your problem, except
mabye one! Try to change the Exception in the
simple.http.Processor.execute() method to Throwable,
this may fix your problem! This change will be
available in the next release soon!

Niall

--- Jan Mikkelsen <ja...@ja...> wrote:

> Hi 
> 
>  
> 
> I have been running out of filehandles since I
> incorporated Simple into
> my application. I'm running on Red Hat and max.
> number of filehandles Is
> 1024. My program uses 100 normally, but suddenly the
> number starts
> rising until it reaches 1024. I get lots of lines
> like this with lsof: 
> 
>  
> 
> java-ak 10197 root  949u  sock        0,0         
> 53806164 can't
> identify protocol
> 
> java-ak 10197 root  950u  sock        0,0         
> 53806165 can't
> identify protocol
> 
> java-ak 10197 root  951u  sock        0,0         
> 53814036 can't
> identify protocol
> 
> java-ak 10197 root  952u  sock        0,0         
> 53814037 can't
> identify protocol
> 
> java-ak 10197 root  953u  sock        0,0         
> 53824098 can't
> identify protocol
> 
> java-ak 10197 root  954u  sock        0,0         
> 53814058 can't
> identify protocol
> 
> java-ak 10197 root  955u  sock        0,0         
> 53823442 can't
> identify protocol
> 
>  
> 
> Does anybody know if Simple can cause "leakage of
> filehandles" ?
> 
>  
> 
> Any input is welcome here :-)
> 
>  
> 
> Jan Mikkelsen
> 
> 


Niall Gallagher

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: FW: [Simpleweb-Support] Simple stops responding

[Niall G. <gal...@ya...>]

Hi Guys,

I think I have figured out why there is a freeze under
heavy load. Its due to an uncaught
java.lang.OutOfMemoryError, which is thrown out of the
Processor.run method, this stops the thread running.
To fix this go into the simple.http.Processor object
and replace all lines that match

catch(Exception

with 

catch(Throwable

This will solve your problem! Also it might be a good
idea to replace the same within the
simple.util.process.Daemon object. I am continuing to
do the tests, but this fix seems to be at the root of
the problem.

Niall

--- Jorge CAMPOS <Jor...@ge...> wrote:

> Hi Niall!
> 
> I'm attaching the console output (removed a lot of
> lines before the end to
> shrink the file size, I've sent you the zip file to
> Yahoo) of our server
> with (at the end) a thread dump... After the thread
> dump you will find that
> there is another request (just to see where is it
> hanging)
> 
> I've attached the classes with the println I've
> added, and the
> Thread.setName (BTW, can you name the threads? it
> helps a lot when debugging
> and profiling)
> 
> It seems that there are no more dequeuers in the
> SchedulerQueue.java and
> then the lock.notify() is not executed (I'll check
> this today).
> 
> All the threads are in wait(), but they don't seem
> to be deadlocked. I
> failed to see our code in the waiting threads, so it
> seems they are not
> waiting for some request to be finished (at least it
> doesn't seems so to
> me).
> 
> I hope this helps you in your research and thansk
> again for your support.
> 
> Jorge
> 
> 
> -----Original Message-----
> From: sim...@li...
>
[mailto:sim...@li...]On
> Behalf Of Niall
> Gallagher
> Sent: Friday, July 29, 2005 5:21 AM
> To: sim...@li...
> Subject: Re: [Simpleweb-Support] Simple stops
> responding
> 
> 
> Hi Michael,
> 
> Thank you for this feedback! I am going to be
> testing
> this over the next few days to track down this
> problem, which is obviously pretty severe...I will
> hopefully have a solution pretty soon! At which time
> I
> will upload the updated version!
> 
> Regards,
> Niall
> 
> --- "Michael \"Luni\" Libes"
> <lun...@ya...> wrote:
> 
> > Niall,
> >
> > I'm successfully tested Simple with web stress
> > tools.  However,
> > I've found a case that does match what others are
> > seeing, and
> > this bug can be triggered with a handful of
> > requests.
> >
> > I have not yet tracked down exactly what is going
> > on, but the
> > server "hangs" whenver there are some connections
> in
> > the
> > CLOSE_WAIT state.  We were able to trigger this
> via
> > a connection
> > initiated by a J2ME handset when the HTTP
> connection
> > was not
> > properly closed, and similarly with a J2SE
> > monitoring tool using
> > java.net.URL.
> >
> > For those not aware, CLOSE_WAIT is a TCP state
> where
> > the server
> > has closed the connection but the client has not
> > acknolwedged
> > and closed it's side of the connection.  (See
> >
>
http://www.mail-archive.com/com...@ja.../msg043
> 38.html
> > and
> >
> http://www.tek-tips.com/viewthread.cfm?qid=669906).
> >
> > Is there any way within Java to detect when a
> socket
> > is in this
> > state?  If so, is there any way to get the socket
> to
> > get in a
> > TIME_WAIT or CLOSED state?
> >
> >   Luni
> >
> >
> > --- Niall Gallagher <gal...@ya...>
> > wrote:
> >
> > > Hi,
> > >
> > > This is interesting, I would like to know what
> > > conclusions you come to! However, I have tested
> > simple
> > > with really enormous loads, for a long period of
> > time,
> > > the only problem I could find was with the log
> > files.
> > > They were too big and the system ran out of disk
> > > space! Still requests were being processed fine!
> > >
> > > Just downloaded the simple-demo-1.0.tar.gz from
> > the
> > > site with no problems. I will send it to you
> > directly.
> > >
> > > Niall
> > >
> > > --- Jorge CAMPOS <Jor...@ge...>
> wrote:
> > >
> > > > Hi Niall!
> > > >
> > > > First of all thank you for your support.
> > > >
> > > > The problem we have is not an HTTP problem. In
> a
> > > > network environment is
> > > > common that connections get broken and I agree
> > that
> > > > it is probably logical
> > > > to close the connection.
> > > >
> > > > But our issue is that after stressing Simple
> > with
> > > > JMeter, the Simple stops
> > > > responding. I mean, the ServerSocket continues
> > to
> > > > accept connections, but
> > > > the requests are not treated. The rest of the
> > > > application continues to run
> > > > normally.
> > > >
> > > > It seems like there are no more resources to
> > handle
> > > > the incoming requests
> > > > (all previous requests were finished, so the
> > pool
> > > > should be able to process
> > > > more, but nothing happens).
> > > >
> > > > Henry thought it could be related to the
> > > > Poller.close(), thinking perhaps a
> > > > thread was removed from the pool and none was
> > > > created to replace it.
> > > >
> > > > I'm trying to download your demo server to
> > reproduce
> > > > the problem, but I
> > > > can't download it from your download section
> > > > (timeout, broken file, etc).
> > > > Can you put the files in the files section of
> > > > sourceforge (I guess at least
> > > > one mirror should work for me).
> > > >
> > > > Thank you very much for your support.
> > > >
> > > > Jorge
> > > >
> > > >
> > > > -----Original Message-----
> > > > From:
> > sim...@li...
> > > >
> > >
> >
>
[mailto:sim...@li...]On
> > > > Behalf Of Niall
> > > > Gallagher
> > > > Sent: Tuesday, July 26, 2005 3:24 PM
> > > > To: sim...@li...
> > > > Subject: RE: [Simpleweb-Support] Problem when
> > stress
> > > > testing an
> > > > application using SimpleWeb
> > > >
> 
=== message truncated ===


Niall Gallagher

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

FW: [Simpleweb-Support] Simple stops responding

[Jorge C. <Jor...@ge...>]

Hi Niall!

I'm attaching the console output (removed a lot of lines before the end to
shrink the file size, I've sent you the zip file to Yahoo) of our server
with (at the end) a thread dump... After the thread dump you will find that
there is another request (just to see where is it hanging)

I've attached the classes with the println I've added, and the
Thread.setName (BTW, can you name the threads? it helps a lot when debugging
and profiling)

It seems that there are no more dequeuers in the SchedulerQueue.java and
then the lock.notify() is not executed (I'll check this today).

All the threads are in wait(), but they don't seem to be deadlocked. I
failed to see our code in the waiting threads, so it seems they are not
waiting for some request to be finished (at least it doesn't seems so to
me).

I hope this helps you in your research and thansk again for your support.

Jorge


-----Original Message-----
From: sim...@li...
[mailto:sim...@li...]On Behalf Of Niall
Gallagher
Sent: Friday, July 29, 2005 5:21 AM
To: sim...@li...
Subject: Re: [Simpleweb-Support] Simple stops responding


Hi Michael,

Thank you for this feedback! I am going to be testing
this over the next few days to track down this
problem, which is obviously pretty severe...I will
hopefully have a solution pretty soon! At which time I
will upload the updated version!

Regards,
Niall

--- "Michael \"Luni\" Libes"
<lun...@ya...> wrote:

> Niall,
>
> I'm successfully tested Simple with web stress
> tools.  However,
> I've found a case that does match what others are
> seeing, and
> this bug can be triggered with a handful of
> requests.
>
> I have not yet tracked down exactly what is going
> on, but the
> server "hangs" whenver there are some connections in
> the
> CLOSE_WAIT state.  We were able to trigger this via
> a connection
> initiated by a J2ME handset when the HTTP connection
> was not
> properly closed, and similarly with a J2SE
> monitoring tool using
> java.net.URL.
>
> For those not aware, CLOSE_WAIT is a TCP state where
> the server
> has closed the connection but the client has not
> acknolwedged
> and closed it's side of the connection.  (See
>
http://www.mail-archive.com/com...@ja.../msg043
38.html
> and
> http://www.tek-tips.com/viewthread.cfm?qid=669906).
>
> Is there any way within Java to detect when a socket
> is in this
> state?  If so, is there any way to get the socket to
> get in a
> TIME_WAIT or CLOSED state?
>
>   Luni
>
>
> --- Niall Gallagher <gal...@ya...>
> wrote:
>
> > Hi,
> >
> > This is interesting, I would like to know what
> > conclusions you come to! However, I have tested
> simple
> > with really enormous loads, for a long period of
> time,
> > the only problem I could find was with the log
> files.
> > They were too big and the system ran out of disk
> > space! Still requests were being processed fine!
> >
> > Just downloaded the simple-demo-1.0.tar.gz from
> the
> > site with no problems. I will send it to you
> directly.
> >
> > Niall
> >
> > --- Jorge CAMPOS <Jor...@ge...> wrote:
> >
> > > Hi Niall!
> > >
> > > First of all thank you for your support.
> > >
> > > The problem we have is not an HTTP problem. In a
> > > network environment is
> > > common that connections get broken and I agree
> that
> > > it is probably logical
> > > to close the connection.
> > >
> > > But our issue is that after stressing Simple
> with
> > > JMeter, the Simple stops
> > > responding. I mean, the ServerSocket continues
> to
> > > accept connections, but
> > > the requests are not treated. The rest of the
> > > application continues to run
> > > normally.
> > >
> > > It seems like there are no more resources to
> handle
> > > the incoming requests
> > > (all previous requests were finished, so the
> pool
> > > should be able to process
> > > more, but nothing happens).
> > >
> > > Henry thought it could be related to the
> > > Poller.close(), thinking perhaps a
> > > thread was removed from the pool and none was
> > > created to replace it.
> > >
> > > I'm trying to download your demo server to
> reproduce
> > > the problem, but I
> > > can't download it from your download section
> > > (timeout, broken file, etc).
> > > Can you put the files in the files section of
> > > sourceforge (I guess at least
> > > one mirror should work for me).
> > >
> > > Thank you very much for your support.
> > >
> > > Jorge
> > >
> > >
> > > -----Original Message-----
> > > From:
> sim...@li...
> > >
> >
>
[mailto:sim...@li...]On
> > > Behalf Of Niall
> > > Gallagher
> > > Sent: Tuesday, July 26, 2005 3:24 PM
> > > To: sim...@li...
> > > Subject: RE: [Simpleweb-Support] Problem when
> stress
> > > testing an
> > > application using SimpleWeb
> > >
> > >
> > > Hi Henry,
> > >
> > > The reason the connection is closed is because
> RFC
> > > 2616 says that this is the way this must be
> done!
> > > For
> > > example, that the scenario where the request
> > > conatins
> > > a larger chunked encoded message body, and then
> you
> > > read only half of it. If the poller is not
> closed,
> > > where do you read the next request header from?
> Also
> > > if you send some wrong information how is the
> > > browser
> > > supposed to interpret it?
> > >
> > > Anyway, there is a large section in the HTTP/1.1
> > > specification that discusses pipelining and its
> > > semantics. It should provide some more detail.
> > >
> > > Niall
> > >
> > > --- Henry Voyer <hen...@ge...> wrote:
> > >
> > > > Re Hi Neil,
> > > >
> > > > Well i found the problem and fixed it. And i
> would
> > > > like your input about it.
> > > > The problem is when the client suddently close
> the
> > > > stream and this happens
> > > > often in highly concurrent transactions.
> > > >
> > > > java.io.IOException: Stream closed
> > > > 	at
> > > >
> > >
> >
>
java.io.PushbackInputStream.ensureOpen(PushbackInputStream.java:57)
> > > > 	at
> > > >
> > >
> >
>
java.io.PushbackInputStream.unread(PushbackInputStream.java:181)
> > > > 	at
> > > >
> > >
> >
>
simple.http.PollerInputStream.available(PollerInputStream.java:114)
> > > > 	at
> > > >
> > >
> >
>
simple.http.BasicPoller.process(BasicPoller.java:117)
> > > > 	at
> > > >
> simple.http.Processor.execute(Processor.java:121)
> > > > 	at
> simple.http.Processor.run(Processor.java:83)
> > > > 	at java.lang.Thread.run(Thread.java:595)
> > > >
> > > > Then the exception makes the closes the poller
> > > >
> > > > This happens, in the class :
> simple.http.Processor
> > > >
> > > >   private void execute() throws
> > > InterruptedException
> > > >  {
> > > >        Object top = queue.dequeue();
>
=== message truncated ===


Niall Gallagher

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO September
19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
Simpleweb-Support mailing list
Sim...@li...
https://lists.sourceforge.net/lists/listinfo/simpleweb-support

Re: [Simpleweb-Support] Simple stops responding

[Niall G. <gal...@ya...>]

Hi Michael,

Thank you for this feedback! I am going to be testing
this over the next few days to track down this
problem, which is obviously pretty severe...I will
hopefully have a solution pretty soon! At which time I
will upload the updated version!

Regards,
Niall

--- "Michael \"Luni\" Libes"
<lun...@ya...> wrote:

> Niall,
> 
> I'm successfully tested Simple with web stress
> tools.  However,
> I've found a case that does match what others are
> seeing, and
> this bug can be triggered with a handful of
> requests.
> 
> I have not yet tracked down exactly what is going
> on, but the
> server "hangs" whenver there are some connections in
> the
> CLOSE_WAIT state.  We were able to trigger this via
> a connection
> initiated by a J2ME handset when the HTTP connection
> was not
> properly closed, and similarly with a J2SE
> monitoring tool using
> java.net.URL.
> 
> For those not aware, CLOSE_WAIT is a TCP state where
> the server
> has closed the connection but the client has not
> acknolwedged
> and closed it's side of the connection.  (See
>
http://www.mail-archive.com/com...@ja.../msg04338.html
> and
> http://www.tek-tips.com/viewthread.cfm?qid=669906).
> 
> Is there any way within Java to detect when a socket
> is in this
> state?  If so, is there any way to get the socket to
> get in a
> TIME_WAIT or CLOSED state?
> 
>   Luni
> 
> 
> --- Niall Gallagher <gal...@ya...>
> wrote:
> 
> > Hi,
> > 
> > This is interesting, I would like to know what
> > conclusions you come to! However, I have tested
> simple
> > with really enormous loads, for a long period of
> time,
> > the only problem I could find was with the log
> files.
> > They were too big and the system ran out of disk
> > space! Still requests were being processed fine!
> > 
> > Just downloaded the simple-demo-1.0.tar.gz from
> the
> > site with no problems. I will send it to you
> directly.
> > 
> > Niall
> > 
> > --- Jorge CAMPOS <Jor...@ge...> wrote:
> > 
> > > Hi Niall!
> > > 
> > > First of all thank you for your support.
> > > 
> > > The problem we have is not an HTTP problem. In a
> > > network environment is
> > > common that connections get broken and I agree
> that
> > > it is probably logical
> > > to close the connection.
> > > 
> > > But our issue is that after stressing Simple
> with
> > > JMeter, the Simple stops
> > > responding. I mean, the ServerSocket continues
> to
> > > accept connections, but
> > > the requests are not treated. The rest of the
> > > application continues to run
> > > normally.
> > > 
> > > It seems like there are no more resources to
> handle
> > > the incoming requests
> > > (all previous requests were finished, so the
> pool
> > > should be able to process
> > > more, but nothing happens).
> > > 
> > > Henry thought it could be related to the
> > > Poller.close(), thinking perhaps a
> > > thread was removed from the pool and none was
> > > created to replace it.
> > > 
> > > I'm trying to download your demo server to
> reproduce
> > > the problem, but I
> > > can't download it from your download section
> > > (timeout, broken file, etc).
> > > Can you put the files in the files section of
> > > sourceforge (I guess at least
> > > one mirror should work for me).
> > > 
> > > Thank you very much for your support.
> > > 
> > > Jorge
> > > 
> > > 
> > > -----Original Message-----
> > > From:
> sim...@li...
> > >
> >
>
[mailto:sim...@li...]On
> > > Behalf Of Niall
> > > Gallagher
> > > Sent: Tuesday, July 26, 2005 3:24 PM
> > > To: sim...@li...
> > > Subject: RE: [Simpleweb-Support] Problem when
> stress
> > > testing an
> > > application using SimpleWeb
> > > 
> > > 
> > > Hi Henry,
> > > 
> > > The reason the connection is closed is because
> RFC
> > > 2616 says that this is the way this must be
> done!
> > > For
> > > example, that the scenario where the request
> > > conatins
> > > a larger chunked encoded message body, and then
> you
> > > read only half of it. If the poller is not
> closed,
> > > where do you read the next request header from?
> Also
> > > if you send some wrong information how is the
> > > browser
> > > supposed to interpret it?
> > > 
> > > Anyway, there is a large section in the HTTP/1.1
> > > specification that discusses pipelining and its
> > > semantics. It should provide some more detail.
> > > 
> > > Niall
> > > 
> > > --- Henry Voyer <hen...@ge...> wrote:
> > > 
> > > > Re Hi Neil,
> > > >
> > > > Well i found the problem and fixed it. And i
> would
> > > > like your input about it.
> > > > The problem is when the client suddently close
> the
> > > > stream and this happens
> > > > often in highly concurrent transactions.
> > > >
> > > > java.io.IOException: Stream closed
> > > > 	at
> > > >
> > >
> >
>
java.io.PushbackInputStream.ensureOpen(PushbackInputStream.java:57)
> > > > 	at
> > > >
> > >
> >
>
java.io.PushbackInputStream.unread(PushbackInputStream.java:181)
> > > > 	at
> > > >
> > >
> >
>
simple.http.PollerInputStream.available(PollerInputStream.java:114)
> > > > 	at
> > > >
> > >
> >
>
simple.http.BasicPoller.process(BasicPoller.java:117)
> > > > 	at
> > > >
> simple.http.Processor.execute(Processor.java:121)
> > > > 	at
> simple.http.Processor.run(Processor.java:83)
> > > > 	at java.lang.Thread.run(Thread.java:595)
> > > >
> > > > Then the exception makes the closes the poller
> > > >
> > > > This happens, in the class :
> simple.http.Processor
> > > >
> > > >   private void execute() throws
> > > InterruptedException
> > > >  {
> > > >        Object top = queue.dequeue();
> 
=== message truncated ===


Niall Gallagher

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [Simpleweb-Support] Simple stops responding

[Michael \Luni\ L. <lun...@ya...>]

Niall,

I'm successfully tested Simple with web stress tools.  However,
I've found a case that does match what others are seeing, and
this bug can be triggered with a handful of requests.

I have not yet tracked down exactly what is going on, but the
server "hangs" whenver there are some connections in the
CLOSE_WAIT state.  We were able to trigger this via a connection
initiated by a J2ME handset when the HTTP connection was not
properly closed, and similarly with a J2SE monitoring tool using
java.net.URL.

For those not aware, CLOSE_WAIT is a TCP state where the server
has closed the connection but the client has not acknolwedged
and closed it's side of the connection.  (See
http://www.mail-archive.com/com...@ja.../msg04338.html
and http://www.tek-tips.com/viewthread.cfm?qid=669906).

Is there any way within Java to detect when a socket is in this
state?  If so, is there any way to get the socket to get in a
TIME_WAIT or CLOSED state?

  Luni


--- Niall Gallagher <gal...@ya...> wrote:

> Hi,
> 
> This is interesting, I would like to know what
> conclusions you come to! However, I have tested simple
> with really enormous loads, for a long period of time,
> the only problem I could find was with the log files.
> They were too big and the system ran out of disk
> space! Still requests were being processed fine!
> 
> Just downloaded the simple-demo-1.0.tar.gz from the
> site with no problems. I will send it to you directly.
> 
> Niall
> 
> --- Jorge CAMPOS <Jor...@ge...> wrote:
> 
> > Hi Niall!
> > 
> > First of all thank you for your support.
> > 
> > The problem we have is not an HTTP problem. In a
> > network environment is
> > common that connections get broken and I agree that
> > it is probably logical
> > to close the connection.
> > 
> > But our issue is that after stressing Simple with
> > JMeter, the Simple stops
> > responding. I mean, the ServerSocket continues to
> > accept connections, but
> > the requests are not treated. The rest of the
> > application continues to run
> > normally.
> > 
> > It seems like there are no more resources to handle
> > the incoming requests
> > (all previous requests were finished, so the pool
> > should be able to process
> > more, but nothing happens).
> > 
> > Henry thought it could be related to the
> > Poller.close(), thinking perhaps a
> > thread was removed from the pool and none was
> > created to replace it.
> > 
> > I'm trying to download your demo server to reproduce
> > the problem, but I
> > can't download it from your download section
> > (timeout, broken file, etc).
> > Can you put the files in the files section of
> > sourceforge (I guess at least
> > one mirror should work for me).
> > 
> > Thank you very much for your support.
> > 
> > Jorge
> > 
> > 
> > -----Original Message-----
> > From: sim...@li...
> >
> [mailto:sim...@li...]On
> > Behalf Of Niall
> > Gallagher
> > Sent: Tuesday, July 26, 2005 3:24 PM
> > To: sim...@li...
> > Subject: RE: [Simpleweb-Support] Problem when stress
> > testing an
> > application using SimpleWeb
> > 
> > 
> > Hi Henry,
> > 
> > The reason the connection is closed is because RFC
> > 2616 says that this is the way this must be done!
> > For
> > example, that the scenario where the request
> > conatins
> > a larger chunked encoded message body, and then you
> > read only half of it. If the poller is not closed,
> > where do you read the next request header from? Also
> > if you send some wrong information how is the
> > browser
> > supposed to interpret it?
> > 
> > Anyway, there is a large section in the HTTP/1.1
> > specification that discusses pipelining and its
> > semantics. It should provide some more detail.
> > 
> > Niall
> > 
> > --- Henry Voyer <hen...@ge...> wrote:
> > 
> > > Re Hi Neil,
> > >
> > > Well i found the problem and fixed it. And i would
> > > like your input about it.
> > > The problem is when the client suddently close the
> > > stream and this happens
> > > often in highly concurrent transactions.
> > >
> > > java.io.IOException: Stream closed
> > > 	at
> > >
> >
>
java.io.PushbackInputStream.ensureOpen(PushbackInputStream.java:57)
> > > 	at
> > >
> >
>
java.io.PushbackInputStream.unread(PushbackInputStream.java:181)
> > > 	at
> > >
> >
>
simple.http.PollerInputStream.available(PollerInputStream.java:114)
> > > 	at
> > >
> >
> simple.http.BasicPoller.process(BasicPoller.java:117)
> > > 	at
> > > simple.http.Processor.execute(Processor.java:121)
> > > 	at simple.http.Processor.run(Processor.java:83)
> > > 	at java.lang.Thread.run(Thread.java:595)
> > >
> > > Then the exception makes the closes the poller
> > >
> > > This happens, in the class : simple.http.Processor
> > >
> > >   private void execute() throws
> > InterruptedException
> > >  {
> > >        Object top = queue.dequeue();
> > >        Poller poller = (Poller)top;
> > >        try{
> > >           poller.process(proc);
> > >        }catch(Exception e){
> > >           poller.close();
> > >        }
> > >     }
> > >
> > > Why close the poller?
> > >
> > >
> > > My fix consist in removing the try catch, and
> > > removing the line of the
> > > pooler.close
> > >
> > > the code is :
> > >
> > >    private void execute() throws IOException,
> > > InterruptedException {
> > >       Object top = queue.dequeue();
> > >       Poller poller = (Poller) top;
> > >       poller.process(proc);
> > >    }
> > >
> > > then is catched in the run method
> > >
> > >  public void run() {
> > >       while (alive) {
> > >          try {
> > >             execute();
> > >          }
> > >          catch (IOException e) {
> > >             l
> > >          }
> > >          catch (InterruptedException e) {
> > >             logger.error("Interrupted exception
> > > while executing an request",
> > > e);
> > >          }
> > >       }
> > >    }
> > >
> > >
> > > -----Original Message-----
> > > From:
> > sim...@li...
> > >
> >
> [mailto:sim...@li...]On
> > > Behalf Of Henry
> > > Voyer
> > > Sent: Tuesday, July 26, 2005 1:06 PM
> > > To: sim...@li...
> > > Cc: jor...@ge...
> > > Subject: RE: [Simpleweb-Support] Problem when
> > stress
> > > testing an
> > > application using SimpleWeb
> > >
> > >
> > > Hi Neil,
> > >
> > > Thanks for answering.
> > >
> > > We know thats part of the problem. But for the
> > HTTP
> > > processing that doesnt
> > > mean that simple has to stop processing more
> > > requests comming from the same
> > > or other clients. It looks like the poller.close()
> > > does just that. One error
> > > and everything stops.
> > >
> > > Il continue checking to see exactly at what
> > > request/ratio it happens and
> > > exacly why.
> > >
> > > Regards
> > > And thanks
> > >
> > > henry
> > >
> > >
> > >
> > >
> > > -----Original Message-----
> > > From:
> > sim...@li...
> > >
> >
> [mailto:sim...@li...]On
> > > Behalf Of Niall
> > > Gallagher
> > 
> === message truncated ===
> 
> 
> Niall Gallagher
> 
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection
> around 
> http://mail.yahoo.com 
> 
> 
> -------------------------------------------------------
> SF.Net email is Sponsored by the Better Software Conference &
> EXPO September
> 19-22, 2005 * San Francisco, CA * Development Lifecycle
> Practices
> Agile & Plan-Driven Development * Managing Projects & Teams *
> Testing & QA
> Security * Process Improvement & Measurement *
> http://www.sqe.com/bsce5sf
> _______________________________________________
> Simpleweb-Support mailing list
> Sim...@li...
> https://lists.sourceforge.net/lists/listinfo/simpleweb-support
> 

Re: [Simpleweb-Support] Simple stops responding

[Niall G. <gal...@ya...>]

Hi,

This is interesting, I would like to know what
conclusions you come to! However, I have tested simple
with really enormous loads, for a long period of time,
the only problem I could find was with the log files.
They were too big and the system ran out of disk
space! Still requests were being processed fine!

Just downloaded the simple-demo-1.0.tar.gz from the
site with no problems. I will send it to you directly.

Niall

--- Jorge CAMPOS <Jor...@ge...> wrote:

> Hi Niall!
> 
> First of all thank you for your support.
> 
> The problem we have is not an HTTP problem. In a
> network environment is
> common that connections get broken and I agree that
> it is probably logical
> to close the connection.
> 
> But our issue is that after stressing Simple with
> JMeter, the Simple stops
> responding. I mean, the ServerSocket continues to
> accept connections, but
> the requests are not treated. The rest of the
> application continues to run
> normally.
> 
> It seems like there are no more resources to handle
> the incoming requests
> (all previous requests were finished, so the pool
> should be able to process
> more, but nothing happens).
> 
> Henry thought it could be related to the
> Poller.close(), thinking perhaps a
> thread was removed from the pool and none was
> created to replace it.
> 
> I'm trying to download your demo server to reproduce
> the problem, but I
> can't download it from your download section
> (timeout, broken file, etc).
> Can you put the files in the files section of
> sourceforge (I guess at least
> one mirror should work for me).
> 
> Thank you very much for your support.
> 
> Jorge
> 
> 
> -----Original Message-----
> From: sim...@li...
>
[mailto:sim...@li...]On
> Behalf Of Niall
> Gallagher
> Sent: Tuesday, July 26, 2005 3:24 PM
> To: sim...@li...
> Subject: RE: [Simpleweb-Support] Problem when stress
> testing an
> application using SimpleWeb
> 
> 
> Hi Henry,
> 
> The reason the connection is closed is because RFC
> 2616 says that this is the way this must be done!
> For
> example, that the scenario where the request
> conatins
> a larger chunked encoded message body, and then you
> read only half of it. If the poller is not closed,
> where do you read the next request header from? Also
> if you send some wrong information how is the
> browser
> supposed to interpret it?
> 
> Anyway, there is a large section in the HTTP/1.1
> specification that discusses pipelining and its
> semantics. It should provide some more detail.
> 
> Niall
> 
> --- Henry Voyer <hen...@ge...> wrote:
> 
> > Re Hi Neil,
> >
> > Well i found the problem and fixed it. And i would
> > like your input about it.
> > The problem is when the client suddently close the
> > stream and this happens
> > often in highly concurrent transactions.
> >
> > java.io.IOException: Stream closed
> > 	at
> >
>
java.io.PushbackInputStream.ensureOpen(PushbackInputStream.java:57)
> > 	at
> >
>
java.io.PushbackInputStream.unread(PushbackInputStream.java:181)
> > 	at
> >
>
simple.http.PollerInputStream.available(PollerInputStream.java:114)
> > 	at
> >
>
simple.http.BasicPoller.process(BasicPoller.java:117)
> > 	at
> > simple.http.Processor.execute(Processor.java:121)
> > 	at simple.http.Processor.run(Processor.java:83)
> > 	at java.lang.Thread.run(Thread.java:595)
> >
> > Then the exception makes the closes the poller
> >
> > This happens, in the class : simple.http.Processor
> >
> >   private void execute() throws
> InterruptedException
> >  {
> >        Object top = queue.dequeue();
> >        Poller poller = (Poller)top;
> >        try{
> >           poller.process(proc);
> >        }catch(Exception e){
> >           poller.close();
> >        }
> >     }
> >
> > Why close the poller?
> >
> >
> > My fix consist in removing the try catch, and
> > removing the line of the
> > pooler.close
> >
> > the code is :
> >
> >    private void execute() throws IOException,
> > InterruptedException {
> >       Object top = queue.dequeue();
> >       Poller poller = (Poller) top;
> >       poller.process(proc);
> >    }
> >
> > then is catched in the run method
> >
> >  public void run() {
> >       while (alive) {
> >          try {
> >             execute();
> >          }
> >          catch (IOException e) {
> >             l
> >          }
> >          catch (InterruptedException e) {
> >             logger.error("Interrupted exception
> > while executing an request",
> > e);
> >          }
> >       }
> >    }
> >
> >
> > -----Original Message-----
> > From:
> sim...@li...
> >
>
[mailto:sim...@li...]On
> > Behalf Of Henry
> > Voyer
> > Sent: Tuesday, July 26, 2005 1:06 PM
> > To: sim...@li...
> > Cc: jor...@ge...
> > Subject: RE: [Simpleweb-Support] Problem when
> stress
> > testing an
> > application using SimpleWeb
> >
> >
> > Hi Neil,
> >
> > Thanks for answering.
> >
> > We know thats part of the problem. But for the
> HTTP
> > processing that doesnt
> > mean that simple has to stop processing more
> > requests comming from the same
> > or other clients. It looks like the poller.close()
> > does just that. One error
> > and everything stops.
> >
> > Il continue checking to see exactly at what
> > request/ratio it happens and
> > exacly why.
> >
> > Regards
> > And thanks
> >
> > henry
> >
> >
> >
> >
> > -----Original Message-----
> > From:
> sim...@li...
> >
>
[mailto:sim...@li...]On
> > Behalf Of Niall
> > Gallagher
> 
=== message truncated ===


Niall Gallagher

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

[Simpleweb-Support] Simple stops responding

[Jorge C. <Jor...@ge...>]

Hi Niall!

First of all thank you for your support.

The problem we have is not an HTTP problem. In a network environment is
common that connections get broken and I agree that it is probably logical
to close the connection.

But our issue is that after stressing Simple with JMeter, the Simple stops
responding. I mean, the ServerSocket continues to accept connections, but
the requests are not treated. The rest of the application continues to run
normally.

It seems like there are no more resources to handle the incoming requests
(all previous requests were finished, so the pool should be able to process
more, but nothing happens).

Henry thought it could be related to the Poller.close(), thinking perhaps a
thread was removed from the pool and none was created to replace it.

I'm trying to download your demo server to reproduce the problem, but I
can't download it from your download section (timeout, broken file, etc).
Can you put the files in the files section of sourceforge (I guess at least
one mirror should work for me).

Thank you very much for your support.

Jorge


-----Original Message-----
From: sim...@li...
[mailto:sim...@li...]On Behalf Of Niall
Gallagher
Sent: Tuesday, July 26, 2005 3:24 PM
To: sim...@li...
Subject: RE: [Simpleweb-Support] Problem when stress testing an
application using SimpleWeb


Hi Henry,

The reason the connection is closed is because RFC
2616 says that this is the way this must be done! For
example, that the scenario where the request conatins
a larger chunked encoded message body, and then you
read only half of it. If the poller is not closed,
where do you read the next request header from? Also
if you send some wrong information how is the browser
supposed to interpret it?

Anyway, there is a large section in the HTTP/1.1
specification that discusses pipelining and its
semantics. It should provide some more detail.

Niall

--- Henry Voyer <hen...@ge...> wrote:

> Re Hi Neil,
>
> Well i found the problem and fixed it. And i would
> like your input about it.
> The problem is when the client suddently close the
> stream and this happens
> often in highly concurrent transactions.
>
> java.io.IOException: Stream closed
> 	at
>
java.io.PushbackInputStream.ensureOpen(PushbackInputStream.java:57)
> 	at
>
java.io.PushbackInputStream.unread(PushbackInputStream.java:181)
> 	at
>
simple.http.PollerInputStream.available(PollerInputStream.java:114)
> 	at
>
simple.http.BasicPoller.process(BasicPoller.java:117)
> 	at
> simple.http.Processor.execute(Processor.java:121)
> 	at simple.http.Processor.run(Processor.java:83)
> 	at java.lang.Thread.run(Thread.java:595)
>
> Then the exception makes the closes the poller
>
> This happens, in the class : simple.http.Processor
>
>   private void execute() throws InterruptedException
>  {
>        Object top = queue.dequeue();
>        Poller poller = (Poller)top;
>        try{
>           poller.process(proc);
>        }catch(Exception e){
>           poller.close();
>        }
>     }
>
> Why close the poller?
>
>
> My fix consist in removing the try catch, and
> removing the line of the
> pooler.close
>
> the code is :
>
>    private void execute() throws IOException,
> InterruptedException {
>       Object top = queue.dequeue();
>       Poller poller = (Poller) top;
>       poller.process(proc);
>    }
>
> then is catched in the run method
>
>  public void run() {
>       while (alive) {
>          try {
>             execute();
>          }
>          catch (IOException e) {
>             l
>          }
>          catch (InterruptedException e) {
>             logger.error("Interrupted exception
> while executing an request",
> e);
>          }
>       }
>    }
>
>
> -----Original Message-----
> From: sim...@li...
>
[mailto:sim...@li...]On
> Behalf Of Henry
> Voyer
> Sent: Tuesday, July 26, 2005 1:06 PM
> To: sim...@li...
> Cc: jor...@ge...
> Subject: RE: [Simpleweb-Support] Problem when stress
> testing an
> application using SimpleWeb
>
>
> Hi Neil,
>
> Thanks for answering.
>
> We know thats part of the problem. But for the HTTP
> processing that doesnt
> mean that simple has to stop processing more
> requests comming from the same
> or other clients. It looks like the poller.close()
> does just that. One error
> and everything stops.
>
> Il continue checking to see exactly at what
> request/ratio it happens and
> exacly why.
>
> Regards
> And thanks
>
> henry
>
>
>
>
> -----Original Message-----
> From: sim...@li...
>
[mailto:sim...@li...]On
> Behalf Of Niall
> Gallagher
> Sent: Tuesday, July 26, 2005 10:53 AM
> To: sim...@li...
> Subject: Re: [Simpleweb-Support] Problem when stress
> testing an
> application using SimpleWeb
>
>
> Hi Henry,
>
> The stream has been closed because of an error! An
> error has occured probably because the HTTP request
> contained a message body. At any rate there is some
> HTTP/1.1 violation with delimeters.
>
> Has your request got a message body that you have
> failed to read?
>
> Niall
>
> --- Henry Voyer <hen...@ge...> wrote:
>
> > Hi everyone
> >
> > I have been stress testing an extensive
> transaction
> > oriented application
> > that uses simple.
> > The stress test consist of using jmeter with
> around
> > 20 threads sending 500
> > requests each to my server.
> >
> > After a couple of minutes the system stops.
> >
> > So i started putting logs and found the section
> > where the errors occurs.
> > The code :
> >
> > simple.http.Processor
> >
> >  private void execute() throws
> InterruptedException
> > {
> >       logger.debug("Reading a process");
> >       Object top = queue.dequeue();
> >       logger.debug("Processing " +
> > ToStringBuilder.reflectionToString(top));
> >       Poller poller = (Poller)top;
> >       try{
> >          logger.debug("Who is being processed " +
> > ToStringBuilder.reflectionToString(proc));
> >          poller.process(proc);
> >          logger.debug("Finishing pooler");
> >       }catch(Exception e){
> >          logger.error("Closing the pooler due to:
> ",
> > e);
> >          poller.close();
> >       }
> >    }
> >
> > The exception is:
> >
> > ERROR 2005-07-26 10:02:00,125 Processor.execute -
> > Closing the pooler due to:
> > java.io.IOException: Stream closed
> > 	at
> >
>
java.io.PushbackInputStream.ensureOpen(PushbackInputStream.java:57)
> > 	at
> >
>
java.io.PushbackInputStream.unread(PushbackInputStream.java:181)
> > 	at
> simple.http.PollerInputStream.available(Unknown
> > Source)
> > 	at simple.http.BasicPoller.process(Unknown
> Source)
> > 	at simple.http.Processor.execute(Unknown Source)
> > 	at simple.http.Processor.run(Unknown Source)
> > 	at java.lang.Thread.run(Thread.java:595)
> >
> > My configuration is:
> >
> >   	System.setProperty("simple.http.load.mapper",
> > custommapper);
> >       System.setProperty("simple.http.poller",
> > "simple.http.GranularPoller");
>
=== message truncated ===


Niall Gallagher

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
Simpleweb-Support mailing list
Sim...@li...
https://lists.sourceforge.net/lists/listinfo/simpleweb-support

RE: [Simpleweb-Support] Problem when stress testing an application using SimpleWeb

[Niall G. <gal...@ya...>]

Hi,

I had a further look at this, the reason for the
poller.close() is because if the header is too large,
and cannot be fully read the server will shut down the
connection. It has been a while since I wrote this,
but the sematics seem right.

If the header was too large, and the poller was not
closed, then the next poll would read from within the
center of an already half read request header, which
could result in an undesired result. At any rate, if
there is an error reading the request, and the state
cannot be determined, it is safest and best to close
the connection. RFC 2616 has guidelines on how clients
should behave on an asynchronous close!

Niall

--- Niall Gallagher <gal...@ya...> wrote:

> Hi Henry,
> 
> The reason the connection is closed is because RFC
> 2616 says that this is the way this must be done!
> For
> example, that the scenario where the request
> conatins
> a larger chunked encoded message body, and then you
> read only half of it. If the poller is not closed,
> where do you read the next request header from? Also
> if you send some wrong information how is the
> browser
> supposed to interpret it? 
> 
> Anyway, there is a large section in the HTTP/1.1
> specification that discusses pipelining and its
> semantics. It should provide some more detail.
> 
> Niall
> 
> --- Henry Voyer <hen...@ge...> wrote:
> 
> > Re Hi Neil,
> > 
> > Well i found the problem and fixed it. And i would
> > like your input about it.
> > The problem is when the client suddently close the
> > stream and this happens
> > often in highly concurrent transactions.
> > 
> > java.io.IOException: Stream closed
> > 	at
> >
>
java.io.PushbackInputStream.ensureOpen(PushbackInputStream.java:57)
> > 	at
> >
>
java.io.PushbackInputStream.unread(PushbackInputStream.java:181)
> > 	at
> >
>
simple.http.PollerInputStream.available(PollerInputStream.java:114)
> > 	at
> >
>
simple.http.BasicPoller.process(BasicPoller.java:117)
> > 	at
> > simple.http.Processor.execute(Processor.java:121)
> > 	at simple.http.Processor.run(Processor.java:83)
> > 	at java.lang.Thread.run(Thread.java:595)
> > 
> > Then the exception makes the closes the poller
> > 
> > This happens, in the class : simple.http.Processor
> > 
> >   private void execute() throws
> InterruptedException
> >  {
> >        Object top = queue.dequeue();
> >        Poller poller = (Poller)top;
> >        try{
> >           poller.process(proc);
> >        }catch(Exception e){
> >           poller.close();
> >        }
> >     }
> > 
> > Why close the poller?
> > 
> > 
> > My fix consist in removing the try catch, and
> > removing the line of the
> > pooler.close
> > 
> > the code is :
> > 
> >    private void execute() throws IOException,
> > InterruptedException {
> >       Object top = queue.dequeue();
> >       Poller poller = (Poller) top;
> >       poller.process(proc);
> >    }
> > 
> > then is catched in the run method
> > 
> >  public void run() {
> >       while (alive) {
> >          try {
> >             execute();
> >          }
> >          catch (IOException e) {
> >             l
> >          }
> >          catch (InterruptedException e) {
> >             logger.error("Interrupted exception
> > while executing an request",
> > e);
> >          }
> >       }
> >    }
> > 
> > 
> > -----Original Message-----
> > From:
> sim...@li...
> >
>
[mailto:sim...@li...]On
> > Behalf Of Henry
> > Voyer
> > Sent: Tuesday, July 26, 2005 1:06 PM
> > To: sim...@li...
> > Cc: jor...@ge...
> > Subject: RE: [Simpleweb-Support] Problem when
> stress
> > testing an
> > application using SimpleWeb
> > 
> > 
> > Hi Neil,
> > 
> > Thanks for answering.
> > 
> > We know thats part of the problem. But for the
> HTTP
> > processing that doesnt
> > mean that simple has to stop processing more
> > requests comming from the same
> > or other clients. It looks like the poller.close()
> > does just that. One error
> > and everything stops.
> > 
> > Il continue checking to see exactly at what
> > request/ratio it happens and
> > exacly why.
> > 
> > Regards
> > And thanks
> > 
> > henry
> > 
> > 
> > 
> > 
> > -----Original Message-----
> > From:
> sim...@li...
> >
>
[mailto:sim...@li...]On
> > Behalf Of Niall
> > Gallagher
> > Sent: Tuesday, July 26, 2005 10:53 AM
> > To: sim...@li...
> > Subject: Re: [Simpleweb-Support] Problem when
> stress
> > testing an
> > application using SimpleWeb
> > 
> > 
> > Hi Henry,
> > 
> > The stream has been closed because of an error! An
> > error has occured probably because the HTTP
> request
> > contained a message body. At any rate there is
> some
> > HTTP/1.1 violation with delimeters.
> > 
> > Has your request got a message body that you have
> > failed to read?
> > 
> > Niall
> > 
> > --- Henry Voyer <hen...@ge...> wrote:
> > 
> > > Hi everyone
> > >
> > > I have been stress testing an extensive
> > transaction
> > > oriented application
> > > that uses simple.
> > > The stress test consist of using jmeter with
> > around
> > > 20 threads sending 500
> > > requests each to my server.
> > >
> > > After a couple of minutes the system stops.
> > >
> > > So i started putting logs and found the section
> > > where the errors occurs.
> > > The code :
> > >
> > > simple.http.Processor
> > >
> > >  private void execute() throws
> > InterruptedException
> > > {
> > >       logger.debug("Reading a process");
> > >       Object top = queue.dequeue();
> > >       logger.debug("Processing " +
> > > ToStringBuilder.reflectionToString(top));
> > >       Poller poller = (Poller)top;
> > >       try{
> > >          logger.debug("Who is being processed "
> +
> 
=== message truncated ===


Niall Gallagher

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

RE: [Simpleweb-Support] Problem when stress testing an application using SimpleWeb

[Niall G. <gal...@ya...>]

Hi Henry,

The reason the connection is closed is because RFC
2616 says that this is the way this must be done! For
example, that the scenario where the request conatins
a larger chunked encoded message body, and then you
read only half of it. If the poller is not closed,
where do you read the next request header from? Also
if you send some wrong information how is the browser
supposed to interpret it? 

Anyway, there is a large section in the HTTP/1.1
specification that discusses pipelining and its
semantics. It should provide some more detail.

Niall

--- Henry Voyer <hen...@ge...> wrote:

> Re Hi Neil,
> 
> Well i found the problem and fixed it. And i would
> like your input about it.
> The problem is when the client suddently close the
> stream and this happens
> often in highly concurrent transactions.
> 
> java.io.IOException: Stream closed
> 	at
>
java.io.PushbackInputStream.ensureOpen(PushbackInputStream.java:57)
> 	at
>
java.io.PushbackInputStream.unread(PushbackInputStream.java:181)
> 	at
>
simple.http.PollerInputStream.available(PollerInputStream.java:114)
> 	at
>
simple.http.BasicPoller.process(BasicPoller.java:117)
> 	at
> simple.http.Processor.execute(Processor.java:121)
> 	at simple.http.Processor.run(Processor.java:83)
> 	at java.lang.Thread.run(Thread.java:595)
> 
> Then the exception makes the closes the poller
> 
> This happens, in the class : simple.http.Processor
> 
>   private void execute() throws InterruptedException
>  {
>        Object top = queue.dequeue();
>        Poller poller = (Poller)top;
>        try{
>           poller.process(proc);
>        }catch(Exception e){
>           poller.close();
>        }
>     }
> 
> Why close the poller?
> 
> 
> My fix consist in removing the try catch, and
> removing the line of the
> pooler.close
> 
> the code is :
> 
>    private void execute() throws IOException,
> InterruptedException {
>       Object top = queue.dequeue();
>       Poller poller = (Poller) top;
>       poller.process(proc);
>    }
> 
> then is catched in the run method
> 
>  public void run() {
>       while (alive) {
>          try {
>             execute();
>          }
>          catch (IOException e) {
>             l
>          }
>          catch (InterruptedException e) {
>             logger.error("Interrupted exception
> while executing an request",
> e);
>          }
>       }
>    }
> 
> 
> -----Original Message-----
> From: sim...@li...
>
[mailto:sim...@li...]On
> Behalf Of Henry
> Voyer
> Sent: Tuesday, July 26, 2005 1:06 PM
> To: sim...@li...
> Cc: jor...@ge...
> Subject: RE: [Simpleweb-Support] Problem when stress
> testing an
> application using SimpleWeb
> 
> 
> Hi Neil,
> 
> Thanks for answering.
> 
> We know thats part of the problem. But for the HTTP
> processing that doesnt
> mean that simple has to stop processing more
> requests comming from the same
> or other clients. It looks like the poller.close()
> does just that. One error
> and everything stops.
> 
> Il continue checking to see exactly at what
> request/ratio it happens and
> exacly why.
> 
> Regards
> And thanks
> 
> henry
> 
> 
> 
> 
> -----Original Message-----
> From: sim...@li...
>
[mailto:sim...@li...]On
> Behalf Of Niall
> Gallagher
> Sent: Tuesday, July 26, 2005 10:53 AM
> To: sim...@li...
> Subject: Re: [Simpleweb-Support] Problem when stress
> testing an
> application using SimpleWeb
> 
> 
> Hi Henry,
> 
> The stream has been closed because of an error! An
> error has occured probably because the HTTP request
> contained a message body. At any rate there is some
> HTTP/1.1 violation with delimeters.
> 
> Has your request got a message body that you have
> failed to read?
> 
> Niall
> 
> --- Henry Voyer <hen...@ge...> wrote:
> 
> > Hi everyone
> >
> > I have been stress testing an extensive
> transaction
> > oriented application
> > that uses simple.
> > The stress test consist of using jmeter with
> around
> > 20 threads sending 500
> > requests each to my server.
> >
> > After a couple of minutes the system stops.
> >
> > So i started putting logs and found the section
> > where the errors occurs.
> > The code :
> >
> > simple.http.Processor
> >
> >  private void execute() throws
> InterruptedException
> > {
> >       logger.debug("Reading a process");
> >       Object top = queue.dequeue();
> >       logger.debug("Processing " +
> > ToStringBuilder.reflectionToString(top));
> >       Poller poller = (Poller)top;
> >       try{
> >          logger.debug("Who is being processed " +
> > ToStringBuilder.reflectionToString(proc));
> >          poller.process(proc);
> >          logger.debug("Finishing pooler");
> >       }catch(Exception e){
> >          logger.error("Closing the pooler due to:
> ",
> > e);
> >          poller.close();
> >       }
> >    }
> >
> > The exception is:
> >
> > ERROR 2005-07-26 10:02:00,125 Processor.execute -
> > Closing the pooler due to:
> > java.io.IOException: Stream closed
> > 	at
> >
>
java.io.PushbackInputStream.ensureOpen(PushbackInputStream.java:57)
> > 	at
> >
>
java.io.PushbackInputStream.unread(PushbackInputStream.java:181)
> > 	at
> simple.http.PollerInputStream.available(Unknown
> > Source)
> > 	at simple.http.BasicPoller.process(Unknown
> Source)
> > 	at simple.http.Processor.execute(Unknown Source)
> > 	at simple.http.Processor.run(Unknown Source)
> > 	at java.lang.Thread.run(Thread.java:595)
> >
> > My configuration is:
> >
> >   	System.setProperty("simple.http.load.mapper",
> > custommapper);
> >       System.setProperty("simple.http.poller",
> > "simple.http.GranularPoller");
> 
=== message truncated ===


Niall Gallagher

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com