From: Charlie G. <cha...@gm...> - 2007-07-23 06:49:47
Attachments:
test_socket.out
test_select_new.out
|
It looks like test_socket and test_select_new are failing on the Mac on trunk. I'm not sure how long they've been failing; their unittests weren't run by regrtest's runner, so failures didn't show up for me. They'll show up there now though. In any case, I'm not expert enough in socket or select to have any idea what's going wrong without further investigation. I've attached their output in case someone else wants to take a look. From the buildbot, it looks like they're ok on Windows, but I'm not sure about Linux since that slave is down. I'm going to start digging into these in earnest tomorrow night since this is the only thing holding back rc3. Charlie |
From: Pekka L. <pe...@ik...> - 2007-07-23 07:07:07
|
2007/7/23, Charlie Groves <cha...@gm...>: > It looks like test_socket and test_select_new are failing on the Mac > on trunk. I'm not sure how long they've been failing; their unittests > weren't run by regrtest's runner, so failures didn't show up for me. > They'll show up there now though. > > In any case, I'm not expert enough in socket or select to have any > idea what's going wrong without further investigation. I've attached > their output in case someone else wants to take a look. From the > buildbot, it looks like they're ok on Windows, but I'm not sure about > Linux since that slave is down. I'm going to start digging into these > in earnest tomorrow night since this is the only thing holding back > rc3. I tested them on Ubuntu 7.04 and got quite a lot of similar failures too. I can send outputs later if needed (the machine is not on net currently) but I got to go now... Cheers, .peke |
From: Oti <oh...@gm...> - 2007-07-23 14:08:45
Attachments:
standalone.out
|
Charlie, if I run the tests with regrtest.py, test_socket fails, but both test_select and test_select_new pass. The error message is: test_socket test test_socket produced unexpected output: ********************************************************************** *** line 2 of expected output missing: - socket.error ********************************************************************** But if I run the tests standalone (jython test_socket.py), they both pass (see attached output). >From your stack I see that you are calling them using unittest.py How do you exactly call them ? I suspect it has to do with the way the tests are called. best wishes, Oti. On 7/23/07, Charlie Groves <cha...@gm...> wrote: > It looks like test_socket and test_select_new are failing on the Mac > on trunk. I'm not sure how long they've been failing; their unittests > weren't run by regrtest's runner, so failures didn't show up for me. > They'll show up there now though. > > In any case, I'm not expert enough in socket or select to have any > idea what's going wrong without further investigation. I've attached > their output in case someone else wants to take a look. From the > buildbot, it looks like they're ok on Windows, but I'm not sure about > Linux since that slave is down. I'm going to start digging into these > in earnest tomorrow night since this is the only thing holding back > rc3. > > Charlie > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > > > |
From: Oti <oh...@gm...> - 2007-07-23 14:35:22
|
My fault: I did not update build.xml to the newest revision, so I had test/output/test_socket in my dist folder. If I delete this file, regrtest.py neither complains about test_socket nor test_select* sorry for the confusion, Oti. On 7/23/07, Oti <oh...@gm...> wrote: > Charlie, > > if I run the tests with regrtest.py, test_socket fails, but both > test_select and test_select_new pass. The error message is: > > test_socket > test test_socket produced unexpected output: > ********************************************************************** > *** line 2 of expected output missing: > - socket.error > ********************************************************************** > > But if I run the tests standalone (jython test_socket.py), they both > pass (see attached output). > > From your stack I see that you are calling them using unittest.py > How do you exactly call them ? > I suspect it has to do with the way the tests are called. > > best wishes, > Oti. > > On 7/23/07, Charlie Groves <cha...@gm...> wrote: > > It looks like test_socket and test_select_new are failing on the Mac > > on trunk. I'm not sure how long they've been failing; their unittests > > weren't run by regrtest's runner, so failures didn't show up for me. > > They'll show up there now though. > > > > In any case, I'm not expert enough in socket or select to have any > > idea what's going wrong without further investigation. I've attached > > their output in case someone else wants to take a look. From the > > buildbot, it looks like they're ok on Windows, but I'm not sure about > > Linux since that slave is down. I'm going to start digging into these > > in earnest tomorrow night since this is the only thing holding back > > rc3. > > > > Charlie > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by: Splunk Inc. > > Still grepping through log files to find problems? Stop. > > Now Search log events and configuration files using AJAX and a browser. > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > _______________________________________________ > > Jython-dev mailing list > > Jyt...@li... > > https://lists.sourceforge.net/lists/listinfo/jython-dev > > > > > > > > |
From: Alan K. <jyt...@xh...> - 2007-07-23 17:36:38
|
[Charlie] > It looks like test_socket and test_select_new are failing on the Mac > on trunk. I'm not sure how long they've been failing; their unittests > weren't run by regrtest's runner, so failures didn't show up for me. > They'll show up there now though. > > In any case, I'm not expert enough in socket or select to have any > idea what's going wrong without further investigation. I've attached > their output in case someone else wants to take a look. From the > buildbot, it looks like they're ok on Windows, but I'm not sure about > Linux since that slave is down. I'm going to start digging into these > in earnest tomorrow night since this is the only thing holding back > rc3. Can I get access to the Mac on which these are failing? It looks like most of those failures are timing related. Looking at test_socket: both of the failures are tests that I introduced. 1. testTCPClientTimeout was introduced to test timeouts on client connects. The test tries to connect to a port (in a client thread), but the server (in the other thread), while it has bound and opened the socket, is not accepting new connections. Which should cause the client connect to timeout and generate a socket.timeout exception. Strange that it generates "connection refused" instead; this indicates that the server thread has completely failed to bind to the socket. 2. testBindException tries to bind to the same address twice, to check that an address in use exception results. But only the second bind should fail, because the first bind should already have bound the address. But the first bind is failing, which indicates the address is bound before testBindException even starts. Perhaps this is an issue with reuseaddress? Putting in a pause before the first bind and examing the state of the ports with netstat should throw some light. Are these two failing tests running directly one after the other by any chance? Both the failures relate to binding to the same single specific address, i.e. (localhost, 50007) Looking at test_select: These tests must all run in sequence, and are very timing dependent. If one fails, then it is virtually guaranteed that all of the tests after that will fail as well. The first test that fails is test430, which fails with "Exceeded alloted time (5.001 > 5.000) to read 98000 bytes: got 80000". Increasing the value of READ_TIMEOUT in the test module should solve the problem, but increase the running time of the test. Unfortunately, the timing of the select_new tests is heavily dependent on the underlying socket library, how large its buffers are, how quickly it drains buffers, etc. As I mentioned, if I can get access to the mac on which they're being run, I'll have a look. Regards, Alan. |
From: Charlie G. <cha...@gm...> - 2007-07-24 08:44:50
|
On 7/23/07, Alan Kennedy <jyt...@xh...> wrote: > [Charlie] > > In any case, I'm not expert enough in socket or select to have any > > idea what's going wrong without further investigation. I've attached > > their output in case someone else wants to take a look. From the > > buildbot, it looks like they're ok on Windows, but I'm not sure about > > Linux since that slave is down. I'm going to start digging into these > > in earnest tomorrow night since this is the only thing holding back > > rc3. > > > Looking at test_socket: both of the failures are tests that I introduced. > > 1. testTCPClientTimeout was introduced to test timeouts on client > connects. The test tries to connect to a port (in a client thread), but > the server (in the other thread), while it has bound and opened the > socket, is not accepting new connections. Which should cause the client > connect to timeout and generate a socket.timeout exception. Strange that > it generates "connection refused" instead; this indicates that the > server thread has completely failed to bind to the socket. I may be totally misunderstanding how the test was supposed to work, but I don't think it was doing this. Since testTCPClientTimeout just returned immediately, __tearDown was called by ThreadedTest as soon as it returned which closed the server socket. My Mac is dual-core, and the buildbot Mac has dual G5s, so they would both see the server socket closed before the client socket timeout expired. I committed a change to testTCPClientTimeout that I think fixes this. It just waits for the client thread to call done.set before returning and closing the server socket. However, the test still fails with this change, just with a different error. Now there's no exception thrown at all, Mac or PC, single or dual processor. If I've managed to understand the way the inheritance and calls map through ThreadedTCPSocketTest into TCPClientTimeoutTest, the test code is really running the following: import socket serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM) serv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) serv.bind(('localhost', 50007)) serv.listen(1) cli = socket.socket(socket.AF_INET, socket.SOCK_STREAM) cli.settimeout(0.1) cli.connect(('localhost', 50007)) This doesn't cause an exception on cli.connect under CPython or Jython. > 2. testBindException tries to bind to the same address twice, to check > that an address in use exception results. But only the second bind > should fail, because the first bind should already have bound the > address. But the first bind is failing, which indicates the address is > bound before testBindException even starts. Perhaps this is an issue > with reuseaddress? It looks like if a previous test doesn't close its socket bound to the port and address we use in the tests, the bind fails in testBindException. I was able to get testBindException to pass when just the TestJythonExceptions class tests are run by themselves by adding a setUp/tearDown to make sure the socket is closed every time, but it still fails when run with the other tests. I guess one of them is leaving the socket as well and it causes the failure. Maybe that's what you mean by reuseaddress? That's all the time I have to look at this tonight though. More tomorrow! Charlie |
From: Alan K. <jyt...@xh...> - 2007-07-24 18:21:03
|
[Alan] >>1. testTCPClientTimeout was introduced to test timeouts on client >>connects. The test tries to connect to a port (in a client thread), but >>the server (in the other thread), while it has bound and opened the >>socket, is not accepting new connections. [Charlie] > I may be totally misunderstanding how the test was supposed to work, > but I don't think it was doing this. Since testTCPClientTimeout just > returned immediately, __tearDown was called by ThreadedTest as soon as > it returned which closed the server socket. You're right Charlie, the server thread finishes too early and closes the socket. I think it's only an accident of thread-timing that the test was passing on Windows. [Charlie] > I committed a change to testTCPClientTimeout that I think fixes this. But you're proposed fix won't work either. The problem is that, although the server thread is not accept'ing new connections, it is listen'ing, meaning that it does "accept" incoming connections; the accept() call simply peels the next connected socket of the backlog queue (I think). To make the server thread refuse incoming connections, and thus make the test pass with the correct timeout exception, it needs to bind to but not listen to the port. So this definition of the test should work def testTCPClientTimeout(self): self.serv.close() # Currently can't do this with jython socket # because of delayed socket creation import java.net.ServerSocket self.serv = java.net.ServerSocket(PORT) # Server port is bound, but not listening self.done.wait() def _testTCPClientTimeout(self): try: self.cli.settimeout(0.1) self.cli.connect( (HOST, PORT) ) except socket.timeout, st: pass except Exception, x: self.fail("Client socket timeout should have raised socket.timeout, not %s" % str(x)) else: self.fail("Client socket timeout should have raised socket.timeout") Note the direct use of java.net.ServerSocket in the server thread. This is necessary because it is currently not possible to bind but not listen using the current jython implementation, due to the necessity of deferring socket creation until it is known if the socket is a server or client socket. [Charlie] > It just waits for the client thread to call done.set before returning > and closing the server socket. This is definitely the right thing to do; the test could never properly pass without it. [Alan] >>2. testBindException tries to bind to the same address twice, to check >>that an address in use exception results. [Charlie] > It looks like if a previous test doesn't close its socket bound to the > port and address we use in the tests, the bind fails in > testBindException. I think you're right there, but I'm out of time now to look further. Let's get the other test sorted out first; this one might fall into line. Regards, Alan. |
From: Pekka L. <pe...@ik...> - 2007-07-24 21:46:14
Attachments:
test_socket.out
test_select_new.out
|
Hello, I run socket and select unit test on my Linux machine and got some errors. Outputs are attached and below is a brief summary and some details about my machine. Note that I needed to remove some content from test_socket.out to make it small enough for this list. I also tried sending outputs zipped but .zip extension seems to be totally blocked... When running select tests there's either one or two failures. test410_FullChannels seems to be failing every time and test430_EmptyChannels most of the time but not always. >From socket tests I got normally 44 errors but sometimes few less. All of them seem to have message "error: (98, 'Address already in use')". I run tests using latest code from trunk (r3355) and used following commands to execute them. $ ./jytip dist/Lib/test/test_socket.py > test_socket.out $ ./jytip dist/Lib/test/test_select_new.py > test_select_new.out My machine is Lenovo Thinkpad T60 5600 with Intel dual core CPU. I'm running normal Ubuntu 7.04 and Java version information is below. $ java -version java version "1.6.0" Java(TM) SE Runtime Environment (build 1.6.0-b105) Java HotSpot(TM) Client VM (build 1.6.0-b105, mixed mode, sharing) Let me know if some more information is needed, I should be running these tests differently, etc. Cheers, .peke |
From: Charlie G. <cha...@gm...> - 2007-07-25 06:31:09
|
On 7/24/07, Alan Kennedy <jyt...@xh...> wrote: > To make the server thread refuse incoming connections, and thus make the > test pass with the correct timeout exception, it needs to bind to but > not listen to the port. So this definition of the test should work > > def testTCPClientTimeout(self): > self.serv.close() > # Currently can't do this with jython socket > # because of delayed socket creation > import java.net.ServerSocket > self.serv = java.net.ServerSocket(PORT) > # Server port is bound, but not listening > self.done.wait() > > def _testTCPClientTimeout(self): > try: > self.cli.settimeout(0.1) > self.cli.connect( (HOST, PORT) ) > except socket.timeout, st: > pass > except Exception, x: > self.fail("Client socket timeout should have raised > socket.timeout, not %s" % str(x)) > else: > self.fail("Client socket timeout should have raised socket.timeout") > > Note the direct use of java.net.ServerSocket in the server thread. This > is necessary because it is currently not possible to bind but not listen > using the current jython implementation, due to the necessity of > deferring socket creation until it is known if the socket is a server or > client socket. This still listens for incoming connections; bind on a ServerSocket calls listen internally. I don't see a way to separate out binding from listening from Java, so I'm thinking we may just have to drop this test. Charlie |
From: Alan K. <jyt...@xh...> - 2007-07-25 18:48:47
|
[Alan] >> To make the server thread refuse incoming connections, and thus make the >> test pass with the correct timeout exception, it needs to bind to but >> not listen to the port. So this definition of the test should work [Charlie] > This still listens for incoming connections; bind on a ServerSocket > calls listen internally. I don't see a way to separate out binding > from listening from Java, so I'm thinking we may just have to drop > this test. Yes, indeed there seems to be no way to bind but not listen on a port. The test as-it-was was passing on my machine because the timeout I used was too short for a connection to be successfully formed. When I lengthen the timeout to 1 second, the connect succeeds, thus failing the test. We could try to find a timeout value that is short enough to guarantee failure on every platform. But that might not be possible, given the multi-threading nature of the test, and the fact that it's being run on multi-cpu machines. Or we could try to connect to a machine on the internet, e.g. www.python.org, with a very low timeout, which is pretty much guaranteed to fail. But that would require our test machine having internet access, and thus our tests would not be self-contained. Maybe we need to drop this test. Which is a pity, because the use case is something that needs testing. I introduced the test because it tests for a bug I found. When in timeout mode, the socket implementation needs to use the java.net.Socket.connect() method, which takes a timeout parameter, instead of the java.nio.channels.SocketChannel.connect() method, which does not take a timeout parameter. My original implementation used the latter method instead of the former, thus timeouts on client connects were not being honoured, as I recorded in the checkin comments for the fix + unit test. Still, rather than simply delete the test, I'd prefer to leave it in place, but commented out. At least that way it can serve as documentation that timeout on client connects is something that needs to be manually tested, or at least noted as a potential source of problems. Regards, Alan. |
From: <bob...@ai...> - 2007-07-26 02:31:46
|
Charlie, Alan, For what its worth, I spent some time looking into this, and with Java, it is virtually impossible to? cause a connect timeout to a server on the same system that the connect (client) is running on.? The minimum value that you can set for a connect timeout in Java is one millisecond, and my connect tests succeeded within 1 millisecond way over 99 percent of the time.? Also, I could not get connects to reliably fail with a timeout even on my LAN.? Setting the timeout to 1 ms, from 50 to 75 percent of the connections still succeeded.? This was on both Windows XP and Linux with both running Java 1.6.? So, unless there is some trick, I don't think that you are going to be able to reliably get this test to pass, without connecting to a server on the Internet.? For the record,? the Java code appeared to do better over the Internet than on the Lan, so I speculate that there is some fault tolerance going on in Java by looking at the address, but I do not know this for sure.? Java really didn't get a good clock until Java 1.5, so this might be something that could be requested from Sun if they are not already planning on upgrading this.? I ran my same tests in Python 2.5 and Python is able to be much more accurate on connect timeouts. Also, I noticed that timeout was added to Python in Python 2.3, so it may be a good idea to leave it off of the Jython 2.2 and add it the next release.? Thanks, ? Bob -----Original Message----- From: Alan Kennedy <jyt...@xh...> To: Charlie Groves <cha...@gm...> Cc: JythonDevelopers <jyt...@li...> Sent: Wed, 25 Jul 2007 1:48 pm Subject: Re: [Jython-dev] test_socket and test_select_new failures on Mac on trunk [Alan] >> To make the server thread refuse incoming connections, and thus make the >> test pass with the correct timeout exception, it needs to bind to but >> not listen to the port. So this definition of the test should work [Charlie] > This still listens for incoming connections; bind on a ServerSocket > calls listen internally. I don't see a way to separate out binding > from listening from Java, so I'm thinking we may just have to drop > this test. Yes, indeed there seems to be no way to bind but not listen on a port. The test as-it-was was passing on my machine because the timeout I used was too short for a connection to be successfully formed. When I lengthen the timeout to 1 second, the connect succeeds, thus failing the test. We could try to find a timeout value that is short enough to guarantee failure on every platform. But that might not be possible, given the multi-threading nature of the test, and the fact that it's being run on multi-cpu machines. Or we could try to connect to a machine on the internet, e.g. www.python.org, with a very low timeout, which is pretty much guaranteed to fail. But that would require our test machine having internet access, and thus our tests would not be self-contained. Maybe we need to drop this test. Which is a pity, because the use case is something that needs testing. I introduced the test because it tests for a bug I found. When in timeout mode, the socket implementation needs to use the java.net.Socket.connect() method, which takes a timeout parameter, instead of the java.nio.channels.SocketChannel.connect() method, which does not take a timeout parameter. My original implementation used the latter method instead of the former, thus timeouts on client connects were not being honoured, as I recorded in the checkin comments for the fix + unit test. Still, rather than simply delete the test, I'd prefer to leave it in place, but commented out. At least that way it can serve as documentation that timeout on client connects is something that needs to be manually tested, or at least noted as a potential source of problems. Regards, Alan. ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Jython-dev mailing list Jyt...@li... https://lists.sourceforge.net/lists/listinfo/jython-dev ________________________________________________________________________ Check Out the new free AIM(R) Mail -- Unlimited storage and industry-leading spam and email virus protection. |
From: Raghuram D. <dra...@gm...> - 2007-07-26 19:32:48
|
On 7/25/07, Alan Kennedy <jyt...@xh...> wrote: > Maybe we need to drop this test. > Which is a pity, because the use case is something that needs testing. I I googled for a way to test connection timeouts and found this link: http://www.developerweb.net/forum/archive/index.php/t-3608.html Following up on the suggestion in that thread, I tried connecting to IPs 213.168.179.251 and 192.168.179.251 and consistenly got timeouts (with time out set to 0.1). Is there a way we can use this methodology in "TCPClientTimeoutTest"? Thanks, Raghu |
From: Alan K. <jyt...@xh...> - 2007-07-26 19:47:25
|
[Raghuram] > I googled for a way to test connection timeouts and found this link: > > http://www.developerweb.net/forum/archive/index.php/t-3608.html > > Following up on the suggestion in that thread, I tried connecting to > IPs 213.168.179.251 and 192.168.179.251 and consistenly got timeouts > (with time out set to 0.1). Is there a way we can use this methodology > in "TCPClientTimeoutTest"? The technique is sound. However, there is always the remote possibility that the chosen address will actually exist, in which case the test will fail. IMO, the probability of the address existing is small enough that I can live with it: if the test fails, then the tester will probably spot the problem fairly quickly, especially if we document it clearly. I'm +1 on this solution, but would like to wait for possible disagreements before we go ahead with it. If anyone thinks this is not a good approach, speak up! Regards, Alan. |
From: <bob...@ai...> - 2007-07-26 20:38:47
|
Hi Alan, This looks good to me.? I would suggest lowering the timeout to 0.001 from 0.1 .? On Windows XP there is a significant difference in the amount of time the code will actually wait to timeout between 0.1 - approx 160 milliseconds - and 0.01? - approx 60 milliseconds.? This will dramatically improve the chances of the test passing even if the IP address happens to exist.? The measured difference between a timeout setting of 0.01 and 0.001 is negligible (on Windows XP anyway), but I would think that 0.001 would translate to a timeout of 1 millisecond in the java code and may become relevant in the future. -----Original Message----- From: Alan Kennedy <jyt...@xh...> To: Raghuram Devarakonda <dra...@gm...> Cc: JythonDevelopers <jyt...@li...>; Charlie Groves <cha...@gm...> Sent: Thu, 26 Jul 2007 2:47 pm Subject: Re: [Jython-dev] test_socket and test_select_new failures on Mac on trunk [Raghuram] > I googled for a way to test connection timeouts and found this link: > > http://www.developerweb.net/forum/archive/index.php/t-3608.html > > Following up on the suggestion in that thread, I tried connecting to > IPs 213.168.179.251 and 192.168.179.251 and consistenly got timeouts > (with time out set to 0.1). Is there a way we can use this methodology > in "TCPClientTimeoutTest"? The technique is sound. However, there is always the remote possibility that the chosen address will actually exist, in which case the test will fail. IMO, the probability of the address existing is small enough that I can live with it: if the test fails, then the tester will probably spot the problem fairly quickly, especially if we document it clearly. I'm +1 on this solution, but would like to wait for possible disagreements before we go ahead with it. If anyone thinks this is not a good approach, speak up! Regards, Alan. ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Jython-dev mailing list Jyt...@li... https://lists.sourceforge.net/lists/listinfo/jython-dev ________________________________________________________________________ Check Out the new free AIM(R) Mail -- Unlimited storage and industry-leading spam and email virus protection. |
From: Charlie G. <cha...@gm...> - 2007-07-28 07:51:25
|
On 7/26/07, Alan Kennedy <jyt...@xh...> wrote: > [Raghuram] > > I googled for a way to test connection timeouts and found this link: > > > > http://www.developerweb.net/forum/archive/index.php/t-3608.html > > > > Following up on the suggestion in that thread, I tried connecting to > > IPs 213.168.179.251 and 192.168.179.251 and consistenly got timeouts > > (with time out set to 0.1). Is there a way we can use this methodology > > in "TCPClientTimeoutTest"? > > I'm +1 on this solution, but would like to wait for possible > disagreements before we go ahead with it. I didn't see any objections, so I went ahead and did this in r3161. r3161 also fixes the testBindException problem I was seeing; the socket created in that test wasn't getting SO_REUSEADDR set on it so the initial bind would fail. With these changes all of test_socket passes for me now. Onto the test_select_new failures... Charlie |
From: <bob...@ai...> - 2007-07-28 19:20:49
|
I tested this on both Suse Linux 10.2 and Windows XP and it ran fine. I still suggest that the timeout be changed from its current value of 0.1 seconds (or 100 milliseconds) to? 0.001 seconds (or 1 millisecond).? Its an easy change, 1 millisecond is a valid timeout value in Java and it will dramatically improve the chances of the test passing if the made up test ip address exists somehow on the testers system.? I don't want to make a big deal out of this, so I will just leave it as a suggestion. -----Original Message----- From: Charlie Groves <cha...@gm...> To: Alan Kennedy <jyt...@xh...> Cc: JythonDevelopers <jyt...@li...> Sent: Sat, 28 Jul 2007 2:50 am Subject: Re: [Jython-dev] test_socket and test_select_new failures on Mac on trunk On 7/26/07, Alan Kennedy <jyt...@xh...> wrote: > [Raghuram] > > I googled for a way to test connection timeouts and found this link: > > > > http://www.developerweb.net/forum/archive/index.php/t-3608.html > > > > Following up on the suggestion in that thread, I tried connecting to > > IPs 213.168.179.251 and 192.168.179.251 and consistenly got timeouts > > (with time out set to 0.1). Is there a way we can use this methodology > > in "TCPClientTimeoutTest"? > > I'm +1 on this solution, but would like to wait for possible > disagreements before we go ahead with it. I didn't see any objections, so I went ahead and did this in r3161. r3161 also fixes the testBindException problem I was seeing; the socket created in that test wasn't getting SO_REUSEADDR set on it so the initial bind would fail. With these changes all of test_socket passes for me now. Onto the test_select_new failures... Charlie ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Jython-dev mailing list Jyt...@li... https://lists.sourceforge.net/lists/listinfo/jython-dev ________________________________________________________________________ Check Out the new free AIM(R) Mail -- Unlimited storage and industry-leading spam and email virus protection. |
From: <bob...@ai...> - 2007-07-28 19:41:52
|
On my systems, Test_Select passed on both Windows XP and Suse Linux 10.2. Test_Select_New passed on Windows XP, but failed in test210_FullChannel and then had an error in? test430_EmptyChannels on my SuseLinux 10.2. -----Original Message----- From: Charlie Groves <cha...@gm...> To: Alan Kennedy <jyt...@xh...> Cc: JythonDevelopers <jyt...@li...> Sent: Sat, 28 Jul 2007 2:50 am Subject: Re: [Jython-dev] test_socket and test_select_new failures on Mac on trunk On 7/26/07, Alan Kennedy <jyt...@xh...> wrote: > [Raghuram] > > I googled for a way to test connection timeouts and found this link: > > > > http://www.developerweb.net/forum/archive/index.php/t-3608.html > > > > Following up on the suggestion in that thread, I tried connecting to > > IPs 213.168.179.251 and 192.168.179.251 and consistenly got timeouts > > (with time out set to 0.1). Is there a way we can use this methodology > > in "TCPClientTimeoutTest"? > > I'm +1 on this solution, but would like to wait for possible > disagreements before we go ahead with it. I didn't see any objections, so I went ahead and did this in r3161. r3161 also fixes the testBindException problem I was seeing; the socket created in that test wasn't getting SO_REUSEADDR set on it so the initial bind would fail. With these changes all of test_socket passes for me now. Onto the test_select_new failures... Charlie ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Jython-dev mailing list Jyt...@li... https://lists.sourceforge.net/lists/listinfo/jython-dev ________________________________________________________________________ Check Out the new free AIM(R) Mail -- Unlimited storage and industry-leading spam and email virus protection. |
From: Alan K. <jyt...@xh...> - 2007-07-23 19:12:13
|
[Alan] > Are these two failing tests running directly one after the other by any > chance? Both the failures relate to binding to the same single specific > address, i.e. (localhost, 50007) Also, I need to point out that the problem may be nothing to do with the socket code, but perhaps be caused by the JVM version its running on, or the patch level of the OS. Back when I started developing these modules, I had problems with bind (on Windows). Although all the unit-tests always ran fine when running singly, I had problems running them all as one suite: The problem was *always* with *bind*ing the server socket, which would occasionally hang, for no fathomable reason. You can read about my trials and tribulations with it here http://www.nabble.com/Non-blocking-IO-update.-tf389917.html#a1074861 The problem intermittently happened, even though I was certain I was executing the right series of socket operations. I tried all kinds of variations; that's still why there are 4 close methods in the current socket module. Then I left the modules untouched for over a year. The next time I ran it, it ran flawlessly! So either an OS upgrade or a JVM upgrade caused the problem to disappear. I'm not going to unroll all my OS upgrades just to find out which one. So I highly recommend checking that both the OS and the JVM are up-to-date on that Mac. If they are not, I would recommend first upgrading the JVM, to the latest 1.4.x. If that doesn't work, then upgrade the OS. Just in case; I wasted days trying to find the source of an almost identical problem with the same code, only to find the problem wasn't in the code, it was in the platform. Regards, Alan. |
From: Charlie G. <cha...@gm...> - 2007-07-24 08:49:36
|
On 7/23/07, Alan Kennedy <jyt...@xh...> wrote: > [Alan] > > Are these two failing tests running directly one after the other by any > > chance? Both the failures relate to binding to the same single specific > > address, i.e. (localhost, 50007) > So I highly recommend checking that both the OS and the JVM are > up-to-date on that Mac. If they are not, I would recommend first > upgrading the JVM, to the latest 1.4.x. If that doesn't work, then > upgrade the OS. Both of the Macs are running OS X 10.4 and are completely patched up. The test fails in the same fashion on 1.5.0_07 and 1.4.2_12, the two VMs available with nio. I don't have Java 6 on them because I think the developer preview Apple has available obliterates any other version on the machine. I'd be interested to see how the tests fare for anyone out there running Java 6 on the Mac. Charlie |
From: Pekka L. <pe...@ik...> - 2007-07-24 09:06:07
|
Charlie, How are you running these tests? As I wrote yesterday I got some errors on Linux too and would like to know are we running tests the same way. What I did was simply running "dist/Lib/test/test_socket.py" (and test_select_new.py) using Jython I build from trunk. I'll anyway rerun those tests when I get home from work. I'll also send outputs here (unless there's some better place to put them) and will check my machine details (it's Thinkpad T60 something). Cheers, .peke |
From: Oti <oh...@gm...> - 2007-07-24 09:24:29
Attachments:
3354-test_socket.out
|
Charlie, test_socket rev 3354 now also fails on my machine. testTCPClientTimeout has an AssertionError: Client socket timeout should have raised socket.timeout I did a full build and ran the test standalone (details attached). best wishes, Oti. On 7/24/07, Charlie Groves <cha...@gm...> wrote: > On 7/23/07, Alan Kennedy <jyt...@xh...> wrote: > > [Alan] > > > Are these two failing tests running directly one after the other by any > > > chance? Both the failures relate to binding to the same single specific > > > address, i.e. (localhost, 50007) > > So I highly recommend checking that both the OS and the JVM are > > up-to-date on that Mac. If they are not, I would recommend first > > upgrading the JVM, to the latest 1.4.x. If that doesn't work, then > > upgrade the OS. > > Both of the Macs are running OS X 10.4 and are completely patched up. > The test fails in the same fashion on 1.5.0_07 and 1.4.2_12, the two > VMs available with nio. I don't have Java 6 on them because I think > the developer preview Apple has available obliterates any other > version on the machine. I'd be interested to see how the tests fare > for anyone out there running Java 6 on the Mac. > > Charlie > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > |
From: Alan K. <jyt...@xh...> - 2007-07-25 18:59:11
|
[Pekka] > I run socket and select unit test on my Linux machine and got some errors. >> From socket tests I got normally 44 errors but sometimes few less. All > Let me know if some more information is needed, I should be running > these tests differently, etc. Try commenting out the "TCPClientTimeoutTest" class in the test_main() function at the bottom of the test_socket.py file. It not doing what I originally expected when I wrote it, and may be disturbing the rest of the tests, by colliding with the sequence of bind calls to the server socket. Regards, Alan. |
From: Pekka L. <pe...@ik...> - 2007-07-25 21:45:09
|
2007/7/25, Alan Kennedy <jyt...@xh...>: > [Pekka] > > I run socket and select unit test on my Linux machine and got some errors. > > >> From socket tests I got normally 44 errors but sometimes few less. All > > > Let me know if some more information is needed, I should be running > > these tests differently, etc. > > Try commenting out the "TCPClientTimeoutTest" class in the test_main() > function at the bottom of the test_socket.py file. > > It not doing what I originally expected when I wrote it, and may be > disturbing the rest of the tests, by colliding with the sequence of bind > calls to the server socket. Tried that but still got 43 errors most of the time and sometimes 32 or 35. Errors seem to be same as before and the error message is still "error: (98, 'Address already in use')". Btw, when I looked at these tests I noticed following code where the message in else branch looks pretty weird. Shouldn't it say "Send on unconnected socket didn't raise exception" since isn't else branch executed when there's no exception? Having "except Exception, x" is also probably not needed here since uncaught exceptions are caught by unittest runner anyway. There were also some places where checking only exception type is enough and assertRaises could be used instead of more verbose try/except/else. try: self.s.send(MSG) except socket.error, se: self.failUnlessEqual(se[0], errno.ENOTCONN) except Exception, x: self.fail("Send on unconnected socket raised wrong exception: %s" % x) else: self.fail("Send on unconnected socket raised exception") Cheers, .peke |
From: Alan K. <jyt...@xh...> - 2007-07-26 20:36:47
|
[Pekka] >>>I run socket and select unit test on my Linux machine and got some errors. [Alan] >>Try commenting out the "TCPClientTimeoutTest" class in the test_main() >>function at the bottom of the test_socket.py file. [Pekka] > Tried that but still got 43 errors most of the time and sometimes 32 > or 35. Errors seem to be same as before and the error message is still > "error: (98, 'Address already in use')". Are there any known problems with reuse_address on your platform? Every single test should be self contained, in that each opens and closes the server socket, in setUp and tearDown respectively. If you change the definition of the SocketTCPTest to be like this, you'll see what I mean # -=-=-=-=-=-= class SocketTCPTest(unittest.TestCase): def setUp(self): self.serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.serv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) self.failUnless(self.serv.getsockopt(None, socket.SO_REUSEADDR)) self.serv.bind((HOST, PORT)) self.serv.listen(1) def tearDown(self): jsocket = self.serv._get_jsocket() self.serv.close() self.failUnless(jsocket.isClosed()) del jsocket self.serv = None # -=-=-=-=-=-= It would be good to see the state of the server socket between tests, by looking at the output of netstat. Insert a delay or a pause in the setUp method of the class above, and examine the state of port 50007 using netstat. If you see sockets in TIME_WAIT or FIN_WAIT_? states, then there may be problems with the socket implementation. Did you say that you're running on a dual-cpu machine? Perhaps the threads being distributed across two cpus might be causing timing issues. Although the test_socket module uses threading.Event's to avoid this. Try inserting a 1-10 second time.sleep at the beginning of the SocketTCPTest.setUp method, to see if that makes a difference. Lastly, googling "serversocket bindexception ubuntu" yields other possibilities 1. Port number is priviledged, i.e. < 1024. I presume that you have not changed the test port number from 50007. 2. The 'localhost' interface is not working properly. http://threebit.net/mail-archive/tomcat-users/msg17755.html http://saloon.javaranch.com/cgi-bin/ubb/ultimatebb.cgi?ubb=next_topic&f=56&t=005381&go=newer Lastly, what JVM are you using? Regards, Alan. |
From: <bob...@ai...> - 2007-07-26 20:59:46
|
Alan, I am having problems with SocketServer having to wait for a timeout period though I've set allow_reuse_address = True .? I am running this on Linux with JVM 1.6.? It sounds related to Pekkas problem.?? I can put together an example for you quite easily using SocketServer.? Would this be helpful to you, or would you rather have one using socket which will take me a little longer. -----Original Message----- From: Alan Kennedy <jyt...@xh...> To: Pekka Laukkanen <pe...@ik...> Cc: JythonDevelopers <jyt...@li...> Sent: Thu, 26 Jul 2007 3:36 pm Subject: Re: [Jython-dev] test_socket and test_select_new failures on Mac on trunk [Pekka] >>>I run socket and select unit test on my Linux machine and got some errors. [Alan] >>Try commenting out the "TCPClientTimeoutTest" class in the test_main() >>function at the bottom of the test_socket.py file. [Pekka] > Tried that but still got 43 errors most of the time and sometimes 32 > or 35. Errors seem to be same as before and the error message is still > "error: (98, 'Address already in use')". Are there any known problems with reuse_address on your platform? Every single test should be self contained, in that each opens and closes the server socket, in setUp and tearDown respectively. If you change the definition of the SocketTCPTest to be like this, you'll see what I mean # -=-=-=-=-=-= class SocketTCPTest(unittest.TestCase): def setUp(self): self.serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.serv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) self.failUnless(self.serv.getsockopt(None, socket.SO_REUSEADDR)) self.serv.bind((HOST, PORT)) self.serv.listen(1) def tearDown(self): jsocket = self.serv._get_jsocket() self.serv.close() self.failUnless(jsocket.isClosed()) del jsocket self.serv = None # -=-=-=-=-=-= It would be good to see the state of the server socket between tests, by looking at the output of netstat. Insert a delay or a pause in the setUp method of the class above, and examine the state of port 50007 using netstat. If you see sockets in TIME_WAIT or FIN_WAIT_? states, then there may be problems with the socket implementation. Did you say that you're running on a dual-cpu machine? Perhaps the threads being distributed across two cpus might be causing timing issues. Although the test_socket module uses threading.Event's to avoid this. Try inserting a 1-10 second time.sleep at the beginning of the SocketTCPTest.setUp method, to see if that makes a difference. Lastly, googling "serversocket bindexception ubuntu" yields other possibilities 1. Port number is priviledged, i.e. < 1024. I presume that you have not changed the test port number from 50007. 2. The 'localhost' interface is not working properly. http://threebit.net/mail-archive/tomcat-users/msg17755.html http://saloon.javaranch.com/cgi-bin/ubb/ultimatebb.cgi?ubb=next_topic&f=56&t=005381&go=newer Lastly, what JVM are you using? Regards, Alan. ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Jython-dev mailing list Jyt...@li... https://lists.sourceforge.net/lists/listinfo/jython-dev ________________________________________________________________________ Check Out the new free AIM(R) Mail -- Unlimited storage and industry-leading spam and email virus protection. |