#18 pcap.next() does not allow thread yield

closed-fixed
None
5
2012-01-05
2008-08-28
Jim
No

For whatever reason, a Python thread will not yield when waiting for a packet on next(). For instance,

self.p = pcap.pcapObject()
self.p.open_live( self.interfaceName, 65535, True, 0 )

while True:
packetStringGlob = self.p.next()
print "I got one!"

If you open this on an interface which receives no traffic (not hard to do if you open on all interfaces and one of them isn't configured) on many threads, your program will hang on next(), even though you put it in a thread and it will wait on a blocking call.

A workaround is as follows:

self.p = pcap.pcapObject()
# return after 1ms
self.p.open_live( self.interfaceName, 65535, True, 1 )

while True:

if packetStringGlob == None:
time.sleep( 1 )

packetStringGlob = self.p.next()
if packetStringGlob == None:
continue

print "I got one!"

The upshot is that if the buffer fills up, it won't immediately respond until the sleep() is over. If there are packets, it'll return packets, and therefore won't sleep before it emptys the buffer. It's a lot like the behavior of the top snippet that doesn't work... only it does work in threaded environments.

Hope this helps someone, and hopefully it'll get fixed at some point.

Discussion

  • Zheng Zhang

    Zheng Zhang - 2009-03-02

    Hi,

    I run into a similar problem and figured out why.

    The Python thread that calls pcap.next() does not yield to another Python thread, while the call is blocked waiting for packet. This is because all your Python threads are actually one OS thread. When you call pcap.next(), the underlying libpcap blocks the entire OS thread. So all your Python threads get stuck until pcap.next() returns.

     
  • Wim Lewis

    Wim Lewis - 2012-01-05
    • assigned_to: nobody --> wiml
     
  • Wim Lewis

    Wim Lewis - 2012-01-05

    Yes: pylibpcap doesn't release the GIL when calling out to possibly-blocking pcap functions. (It basically isn't thread-aware at all, which means it ends up blocking other threads when it needn't.)

     
  • Wim Lewis

    Wim Lewis - 2012-01-05
    • status: open --> closed-fixed
     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks