Ad

Proper Way Of Cancelling Accept And Closing A Python Processing/multiprocessing Listener Connection

- 1 answer

(I'm using the pyprocessing module in this example, but replacing processing with multiprocessing should probably work if you run python 2.6 or use the multiprocessing backport)

I currently have a program that listens to a unix socket (using a processing.connection.Listener), accept connections and spawns a thread handling the request. At a certain point I want to quit the process gracefully, but since the accept()-call is blocking and I see no way of cancelling it in a nice way. I have one way that works here (OS X) at least, setting a signal handler and signalling the process from another thread like so:

import processing
from processing.connection import Listener
import threading
import time
import os
import signal
import socket
import errno

# This is actually called by the connection handler.
def closeme():
    time.sleep(1)
    print 'Closing socket...'
    listener.close()
    os.kill(processing.currentProcess().getPid(), signal.SIGPIPE)

oldsig = signal.signal(signal.SIGPIPE, lambda s, f: None)

listener = Listener('/tmp/asdf', 'AF_UNIX')
# This is a thread that handles one already accepted connection, left out for brevity
threading.Thread(target=closeme).start()
print 'Accepting...'
try:
    listener.accept()
except socket.error, e:
    if e.args[0] != errno.EINTR:
        raise
# Cleanup here...
print 'Done...'

The only other way I've thought about is reaching deep into the connection (listener._listener._socket) and setting the non-blocking option...but that probably has some side effects and is generally really scary.

Does anyone have a more elegant (and perhaps even correct!) way of accomplishing this? It needs to be portable to OS X, Linux and BSD, but Windows portability etc is not necessary.

Clarification: Thanks all! As usual, ambiguities in my original question are revealed :)

  • I need to perform cleanup after I have cancelled the listening, and I don't always want to actually exit that process.
  • I need to be able to access this process from other processes not spawned from the same parent, which makes Queues unwieldy
  • The reasons for threads are that:
    • They access a shared state. Actually more or less a common in-memory database, so I suppose it could be done differently.
    • I must be able to have several connections accepted at the same time, but the actual threads are blocking for something most of the time. Each accepted connection spawns a new thread; this in order to not block all clients on I/O ops.

Regarding threads vs. processes, I use threads for making my blocking ops non-blocking and processes to enable multiprocessing.

Ad

Answer

Isnt that what select is for??

Only call accept on the socket if the select indicates it will not block...

The select has a timeout, so you can break out occasionally occasionally to check if its time to shut down....

Ad
source: stackoverflow.com
Ad