Re: [Jython-users] Concurrency question

 Re: [Jython-users] Concurrency question From: Ype Kingma - 2002-04-07 14:36:25 ```[posting the reply already sent to Frank] Frank, >I wrote a script that creates 1001 threads. Each thread creates an HTTP >connection object and requests a random page from a selection of 10 possible >pages. The thread records the starting time and ending time. Once all 1001 >agents have completed I have a simple evaluation function to count the >number of threads with start and end times that overlap. > >I then calculate the total possible overlapping threads as each thread could >overlap every other but not itself. For 1001 threads that would be 1001 x >1001 - 1001 = 1,001,000 possible overlaps. > >So if I count a total of 991,246 overlaps out of a possible 1,001,000 then I >can say that 99.03% of the agents ran concurrently. > >Does this method make sense? Due to the symmettry of overlaps you could make the final computation roughly twice as fast by using the range(i, ...) instead of range(1, ...) for j. You would then also have to divide the max. nr. of overlaps by two ( n * n - n == n * (n - 1) is always even.) You might also consider using range(0, ...) for i, (python indexes lists from zero.) The appends to stime and duration are not thread safe. You might collect the stats with something like: stats = [] # init stats.append((starttime, duration)) # append a tuple for each thread and get them back afterwards by: for i in range(len(stats)): st1, dr1 = stats[i] for j in range(i + 1, len(stats)): st2, dr2 = stats[j] .... I'd also add assert agentcount >= 2 somewhere. Have fun, Ype >-Frank > >-- > ># Test script written in TestMaker - an open-source utility for testing ># Web Services for functionality, scalability and performance. Download ># TestMaker for free at http://www.PushToTest.com > ># The from statements import objects from the TestMaker objects (TOOL.) > >print "Concurrency is a test to learn how many agents a machine may run >concurrently." >print >print "This version of Concurrency creates local threads that make HTTP" >print "requests to the remote Apache host. The threads records their start" >print "time and duration. The values are compared to determine the number" >print "of threads that overlapped." >print >print "(c) 2002 Frank Cohen. All rights reserved." >print "for details see http://www.PushToTest.com"; >print > >import thread >import time >import random >import sys >from com.pushtotest.tool.protocolhandler import Header >from com.pushtotest.tool.protocolhandler import HTTPHeader >from com.pushtotest.tool.protocolhandler import Body >from com.pushtotest.tool.protocolhandler import HTTPBody >from com.pushtotest.tool.protocolhandler import HTTPProtocol >from com.pushtotest.tool.response import Response > >agentcount = 1000 >min = 10.0 # shortest time a thread took to run >avg = 0.0 # average time >max = 0.0 # maximum time it took for a thread to run >count = 0.0 # count of threads reporting results >runtime = time.time() >stime = [ 0 ] >dur = [ 0 ] > ># This thread is called and returned > >def dominic( me ): > > global min, avg, max, count > > start = time.time(); > > # Create the needed objects to communicate with the host > httphandler = HTTPProtocol() > httphandler.setHost( "192.168.1.100" ) > > httphandler.setPath( "frank" + random.choice("0123456789") + ".html" ) > > # Request the document from the host > response = httphandler.connect() > > stop = time.time() > > duration = stop - start > > if duration<0: > print "gotcha" > > if duration min=duration > > if duration>max: > max=duration > > avg = avg + duration > count = count + 1 > > stime.append( start ) > dur.append( duration ) > >print "Initializing ", agentcount, "threads" > ># reset the values >min = 10.0 >avg = 0.0 >max = 0.0 >count = 1.0 > >for j in range( agentcount ): > > #start_new_thread is a handy feature of Python > #it calls a function in a new thread, when the > #function is complete the thread terminates silently > > thread.start_new_thread( dominic, ( 0, ) ) > >print "running" > >while count < ( agentcount-100 ): > time.sleep(1) > print "Min = %9.6f Max = %9.6f Avg = %9.6f count=%d duration=%d" % (min, >max, avg/count, count, (time.time()-runtime) ) > >ccount = 0 > >print "calculating concurrent threads" > >for i in range( 1, len( stime ) ): > for j in range( 1, len( stime ) ): > if i != j: > start1 = stime[i] > end1 = start1 + dur[i] > start2 = stime[j] > end2 = start2 + dur[j] > if ( ( ( start2 > start1 ) and ( end1 <= end2 ) ) or ( ( start1 >> start2 ) and ( end2 <= end1 ) ) ): > ccount = ccount + 1 > >maxoverlaps = (len(stime)*len(stime))-len(stime) >curper = ( float( ccount) / maxoverlaps ) * 100 >print "Total number of agents counted: ", len(stime) >print "Overlap counts: ", ccount, " out of a total possible overlaps ", >maxoverlaps >print "percentage of agents that ran concurrently: %3.2f" % curper >print "Total duration: ", time.time() - runtime > > > >-- >Frank Cohen, CEO, PushToTest, http://www.pushtotest.com, phone: 408 374 7426 >Come to PushToTest for free open-source Active Security solutions that test, >monitor and automate Web Service systems for functionality, scalability and >performance. > > > >_______________________________________________ >Jython-users mailing list >Jython-users@... >https://lists.sourceforge.net/lists/listinfo/jython-users -- ```

 Re: [Jython-users] Concurrency question From: Ype Kingma - 2002-04-07 14:36:25 ```[posting the reply already sent to Frank] Frank, >I wrote a script that creates 1001 threads. Each thread creates an HTTP >connection object and requests a random page from a selection of 10 possible >pages. The thread records the starting time and ending time. Once all 1001 >agents have completed I have a simple evaluation function to count the >number of threads with start and end times that overlap. > >I then calculate the total possible overlapping threads as each thread could >overlap every other but not itself. For 1001 threads that would be 1001 x >1001 - 1001 = 1,001,000 possible overlaps. > >So if I count a total of 991,246 overlaps out of a possible 1,001,000 then I >can say that 99.03% of the agents ran concurrently. > >Does this method make sense? Due to the symmettry of overlaps you could make the final computation roughly twice as fast by using the range(i, ...) instead of range(1, ...) for j. You would then also have to divide the max. nr. of overlaps by two ( n * n - n == n * (n - 1) is always even.) You might also consider using range(0, ...) for i, (python indexes lists from zero.) The appends to stime and duration are not thread safe. You might collect the stats with something like: stats = [] # init stats.append((starttime, duration)) # append a tuple for each thread and get them back afterwards by: for i in range(len(stats)): st1, dr1 = stats[i] for j in range(i + 1, len(stats)): st2, dr2 = stats[j] .... I'd also add assert agentcount >= 2 somewhere. Have fun, Ype >-Frank > >-- > ># Test script written in TestMaker - an open-source utility for testing ># Web Services for functionality, scalability and performance. Download ># TestMaker for free at http://www.PushToTest.com > ># The from statements import objects from the TestMaker objects (TOOL.) > >print "Concurrency is a test to learn how many agents a machine may run >concurrently." >print >print "This version of Concurrency creates local threads that make HTTP" >print "requests to the remote Apache host. The threads records their start" >print "time and duration. The values are compared to determine the number" >print "of threads that overlapped." >print >print "(c) 2002 Frank Cohen. All rights reserved." >print "for details see http://www.PushToTest.com"; >print > >import thread >import time >import random >import sys >from com.pushtotest.tool.protocolhandler import Header >from com.pushtotest.tool.protocolhandler import HTTPHeader >from com.pushtotest.tool.protocolhandler import Body >from com.pushtotest.tool.protocolhandler import HTTPBody >from com.pushtotest.tool.protocolhandler import HTTPProtocol >from com.pushtotest.tool.response import Response > >agentcount = 1000 >min = 10.0 # shortest time a thread took to run >avg = 0.0 # average time >max = 0.0 # maximum time it took for a thread to run >count = 0.0 # count of threads reporting results >runtime = time.time() >stime = [ 0 ] >dur = [ 0 ] > ># This thread is called and returned > >def dominic( me ): > > global min, avg, max, count > > start = time.time(); > > # Create the needed objects to communicate with the host > httphandler = HTTPProtocol() > httphandler.setHost( "192.168.1.100" ) > > httphandler.setPath( "frank" + random.choice("0123456789") + ".html" ) > > # Request the document from the host > response = httphandler.connect() > > stop = time.time() > > duration = stop - start > > if duration<0: > print "gotcha" > > if duration min=duration > > if duration>max: > max=duration > > avg = avg + duration > count = count + 1 > > stime.append( start ) > dur.append( duration ) > >print "Initializing ", agentcount, "threads" > ># reset the values >min = 10.0 >avg = 0.0 >max = 0.0 >count = 1.0 > >for j in range( agentcount ): > > #start_new_thread is a handy feature of Python > #it calls a function in a new thread, when the > #function is complete the thread terminates silently > > thread.start_new_thread( dominic, ( 0, ) ) > >print "running" > >while count < ( agentcount-100 ): > time.sleep(1) > print "Min = %9.6f Max = %9.6f Avg = %9.6f count=%d duration=%d" % (min, >max, avg/count, count, (time.time()-runtime) ) > >ccount = 0 > >print "calculating concurrent threads" > >for i in range( 1, len( stime ) ): > for j in range( 1, len( stime ) ): > if i != j: > start1 = stime[i] > end1 = start1 + dur[i] > start2 = stime[j] > end2 = start2 + dur[j] > if ( ( ( start2 > start1 ) and ( end1 <= end2 ) ) or ( ( start1 >> start2 ) and ( end2 <= end1 ) ) ): > ccount = ccount + 1 > >maxoverlaps = (len(stime)*len(stime))-len(stime) >curper = ( float( ccount) / maxoverlaps ) * 100 >print "Total number of agents counted: ", len(stime) >print "Overlap counts: ", ccount, " out of a total possible overlaps ", >maxoverlaps >print "percentage of agents that ran concurrently: %3.2f" % curper >print "Total duration: ", time.time() - runtime > > > >-- >Frank Cohen, CEO, PushToTest, http://www.pushtotest.com, phone: 408 374 7426 >Come to PushToTest for free open-source Active Security solutions that test, >monitor and automate Web Service systems for functionality, scalability and >performance. > > > >_______________________________________________ >Jython-users mailing list >Jython-users@... >https://lists.sourceforge.net/lists/listinfo/jython-users -- ```
 Re: [Jython-users] Concurrency question From: Frank Cohen - 2002-04-08 16:11:04 ```Thank you for the tips. I'll change the script, run the tests and post the results here for everyone's info. On a related topic, I ran this script on Jython 2.1 on RedHat Linux 7.2. In this environment is start_new_thread creating a new Java thread? A new native Linux process? -Frank -- Frank Cohen, CEO, PushToTest, http://www.pushtotest.com, phone: 408 374 7426 Come to PushToTest for free open-source Active Security solutions that test, monitor and automate Web Service systems for functionality, scalability and performance. > From: Ype Kingma > Date: Sun, 7 Apr 2002 16:48:30 +0100 > To: jython-users@... > Subject: Re: [Jython-users] Concurrency question > > [posting the reply already sent to Frank] > > Frank, > > > >> I wrote a script that creates 1001 threads. Each thread creates an HTTP >> connection object and requests a random page from a selection of 10 possible >> pages. The thread records the starting time and ending time. Once all 1001 >> agents have completed I have a simple evaluation function to count the >> number of threads with start and end times that overlap. >> >> I then calculate the total possible overlapping threads as each thread could >> overlap every other but not itself. For 1001 threads that would be 1001 x >> 1001 - 1001 = 1,001,000 possible overlaps. >> >> So if I count a total of 991,246 overlaps out of a possible 1,001,000 then I >> can say that 99.03% of the agents ran concurrently. >> >> Does this method make sense? > > Due to the symmettry of overlaps > you could make the final computation roughly twice as fast > by using the range(i, ...) instead of range(1, ...) for j. > You would then also have to divide the max. nr. of overlaps by two > ( n * n - n == n * (n - 1) is always even.) > > You might also consider using range(0, ...) for i, > (python indexes lists from zero.) > > The appends to stime and duration are not thread safe. > You might collect the stats with something like: > > stats = [] # init > > stats.append((starttime, duration)) # append a tuple for each thread > > and get them back afterwards by: > > for i in range(len(stats)): > st1, dr1 = stats[i] > for j in range(i + 1, len(stats)): > st2, dr2 = stats[j] > .... > > I'd also add > > assert agentcount >= 2 > > somewhere. > > > Have fun, > Ype > > >> -Frank >> >> -- >> >> # Test script written in TestMaker - an open-source utility for testing >> # Web Services for functionality, scalability and performance. Download >> # TestMaker for free at http://www.PushToTest.com >> >> # The from statements import objects from the TestMaker objects (TOOL.) >> >> print "Concurrency is a test to learn how many agents a machine may run >> concurrently." >> print >> print "This version of Concurrency creates local threads that make HTTP" >> print "requests to the remote Apache host. The threads records their start" >> print "time and duration. The values are compared to determine the number" >> print "of threads that overlapped." >> print >> print "(c) 2002 Frank Cohen. All rights reserved." >> print "for details see http://www.PushToTest.com"; >> print >> >> import thread >> import time >> import random >> import sys >> from com.pushtotest.tool.protocolhandler import Header >> from com.pushtotest.tool.protocolhandler import HTTPHeader >> from com.pushtotest.tool.protocolhandler import Body >> from com.pushtotest.tool.protocolhandler import HTTPBody >> from com.pushtotest.tool.protocolhandler import HTTPProtocol >> from com.pushtotest.tool.response import Response >> >> agentcount = 1000 >> min = 10.0 # shortest time a thread took to run >> avg = 0.0 # average time >> max = 0.0 # maximum time it took for a thread to run >> count = 0.0 # count of threads reporting results >> runtime = time.time() >> stime = [ 0 ] >> dur = [ 0 ] >> >> # This thread is called and returned >> >> def dominic( me ): >> >> global min, avg, max, count >> >> start = time.time(); >> >> # Create the needed objects to communicate with the host >> httphandler = HTTPProtocol() >> httphandler.setHost( "192.168.1.100" ) >> >> httphandler.setPath( "frank" + random.choice("0123456789") + ".html" ) >> >> # Request the document from the host >> response = httphandler.connect() >> >> stop = time.time() >> >> duration = stop - start >> >> if duration<0: >> print "gotcha" >> >> if duration> min=duration >> >> if duration>max: >> max=duration >> >> avg = avg + duration >> count = count + 1 >> >> stime.append( start ) >> dur.append( duration ) >> >> print "Initializing ", agentcount, "threads" >> >> # reset the values >> min = 10.0 >> avg = 0.0 >> max = 0.0 >> count = 1.0 >> >> for j in range( agentcount ): >> >> #start_new_thread is a handy feature of Python >> #it calls a function in a new thread, when the >> #function is complete the thread terminates silently >> >> thread.start_new_thread( dominic, ( 0, ) ) >> >> print "running" >> >> while count < ( agentcount-100 ): >> time.sleep(1) >> print "Min = %9.6f Max = %9.6f Avg = %9.6f count=%d duration=%d" % (min, >> max, avg/count, count, (time.time()-runtime) ) >> >> ccount = 0 >> >> print "calculating concurrent threads" >> >> for i in range( 1, len( stime ) ): >> for j in range( 1, len( stime ) ): >> if i != j: >> start1 = stime[i] >> end1 = start1 + dur[i] >> start2 = stime[j] >> end2 = start2 + dur[j] >> if ( ( ( start2 > start1 ) and ( end1 <= end2 ) ) or ( ( start1 >>> start2 ) and ( end2 <= end1 ) ) ): >> ccount = ccount + 1 >> >> maxoverlaps = (len(stime)*len(stime))-len(stime) >> curper = ( float( ccount) / maxoverlaps ) * 100 >> print "Total number of agents counted: ", len(stime) >> print "Overlap counts: ", ccount, " out of a total possible overlaps ", >> maxoverlaps >> print "percentage of agents that ran concurrently: %3.2f" % curper >> print "Total duration: ", time.time() - runtime >> >> >> >> -- >> Frank Cohen, CEO, PushToTest, http://www.pushtotest.com, phone: 408 374 7426 >> Come to PushToTest for free open-source Active Security solutions that test, >> monitor and automate Web Service systems for functionality, scalability and >> performance. >> >> >> >> _______________________________________________ >> Jython-users mailing list >> Jython-users@... >> https://lists.sourceforge.net/lists/listinfo/jython-users > > > -- > > _______________________________________________ > Jython-users mailing list > Jython-users@... > https://lists.sourceforge.net/lists/listinfo/jython-users ```
 Re: [Jython-users] Concurrency question From: Michel Pelletier - 2002-04-09 10:57:06 ```On Monday 08 April 2002 06:10 pm, Frank Cohen wrote: > Thank you for the tips. I'll change the script, run the tests and post the > results here for everyone's info. > > On a related topic, I ran this script on Jython 2.1 on RedHat Linux 7.2. In > this environment is start_new_thread creating a new Java thread? A new > native Linux process? I'm not sure what native linux processes are, I haven't heard the term before. A thread in linux is very similar to a process but not identical, Linux calls threads "Light Weight Processes". These links might be of help to you and others: http://www.linuxdoc.org/FAQ/Threads-FAQ/Types.html http://www.linuxjournal.com/article.php?sid=5211 -Michel ```
 Re: [Jython-users] Concurrency question From: Kevin J. Butler - 2002-04-09 23:08:47 ```Frank Cohen wrote: > Thank you for the tips. I'll change the script, run the tests and post the > results here for everyone's info. > > On a related topic, I ran this script on Jython 2.1 on RedHat Linux 7.2. In > this environment is start_new_thread creating a new Java thread? Always. > A new native Linux process? This depends almost totally on the JVM you're using. The Volano performance report (biased toward network services) has a lot of good info. http://www.volano.com/report/index.html kb ```