Writing Multithreaded GUI Applications with Python
Date: Jan 31, 2003
Article is provided courtesy of InformIT.
Multithreading is a programming paradigm that is useful in a number of situations. GUI applications often need to present the appearance of doing several things at once, such as accepting user input while printing, showing a progress bar while computing, and downloading stuff while rendering what has already arrived. Complex games need threads to keep track of both player characters and nonplayer characters. Networking applications need to be able to serve more than one request at a time. Database applications need to monitor the application state while storing and retrieving data.
Because multithreading is so important in so many areas, it's a pity that working with it is often difficult. Debugging a multithreaded application is what programmer's nightmares are made of. The number of subtle timing bugs that can occur in even a simple multithreaded application is hard to underestimate. Threads also are a limited commodity on any operating system: Running out of available threads seldom leads to a pretty result.
Of course, there are alternatives to using threads. In Python, you have the option to use the asyncore module if you are working on a networking application. If you potentially need thousands of threads, you can turn to Stackless Python (see my InformIT article on this topic). A similar package is available for Java in the form of the JRockit JVM from Bea. And for less complex purposes—say, a progress bar or a simple periodical check of some state in your application—using a timer probably is sufficient.
But for your daily bread-and-butter programming, you will probably need threading at some point. Since the early days of NextSTEP, multithreading has grown up and it is now a mature technology.
In this article, I briefly outline the choices you have when you want to work with multithreading in Python while working on a GUI application. Then, using the PyQt toolkit, I will show you how to work around the various obstacles you can encounter when programming a multithreaded GUI.
Thread Basics
Threads are singularly aptly named: A thread is a single strand of execution within the process of your application. That means that inside your application, more than one thing can happen at the same time—virtually, if you've only got one processor, and really if you've got a multiprocessor box. All threads can have direct access to the data in you application.
That doesn't mean that it's a good idea to access that data from several threads at the same time, though. Consider, for instance, what would happen if Thread 1 places the cursor at the position 10,10 on your widget because it wants to paint a button there. At almost the same time, Thread 2 moves that same cursor to position 100,100 because it wants to paint some text. Now several things can happen: Both the button and the text are painted at 10,10 or at 100,100, or the text can be painted at 10,10 and the button can be painted at 100,100. Chances are slim that both will be painted at the right place, but you never know. It's also quite likely that your application will crash horribly.
That's why, in a GUI application, generally one thread paints the screen and receives user events. Any multithreaded application needs safeguards around data that is shared by threads.
Native Threads and a GUI
If you think your application might one day get a different GUI but will remain firmly rooted in Python, using Python's native threading support is a good bet. You can use Python's native threads with any GUI library you like, but you should not use more than one threading system in the same application. That means that you cannot use threading and QThread at the same moment. Doing so will invariably result in crashes.
In a GUI application that uses native threads, it is also not possible to directly influence the GUI thread. PyQt's thread support and wxPython's thread support offer functions you can use to send messages or events to the GUI thread or to lock the main GUI before accessing shared data.
If you use Python's native thread, your worker threads must fill a Queue object and your GUI thread must install a timer that every so often polls the Queue and retrieves new data.
Let's briefly look at Listing 1, a small recipe that uses PyQt, derived from a similar script by Jacob Hallen (from the ActiveState Python Cookbook).
Listing 1: Using Python Threads and a PyQt GUI
import sys, time, threading, random, Queue, qt
class GuiPart(qt.QMainWindow):
def __init__(self, queue, endcommand, *args):
qt.QMainWindow.__init__(self, *args)
self.queue = queue
self.editor = qt.QMultiLineEdit(self)
self.setCentralWidget(self.editor)
self.endcommand = endcommand
def closeEvent(self, ev):
"""
We just call the endcommand when the window is closed
instead of presenting a button for that purpose.
"""
self.endcommand()
def processIncoming(self):
"""
Handle all the messages currently in the queue (if any).
"""
while self.queue.qsize():
try:
msg = self.queue.get(0)
# Check contents of message and do what it says
# As a test, we simply print it
self.editor.insertLine(str(msg))
except Queue.Empty:
pass
class ThreadedClient:
"""
Launch the main part of the GUI and the worker thread.
periodicCall and endApplication could reside in the GUI part,
but putting them here means that you have all the thread
controls in a single place.
"""
def __init__(self):
# Create the queue
self.queue = Queue.Queue()
# Set up the GUI part
self.gui=GuiPart(self.queue, self.endApplication)
self.gui.show()
# A timer to periodically call periodicCall :-)
self.timer = qt.QTimer()
qt.QObject.connect(self.timer,
qt.SIGNAL("timeout()"),
self.periodicCall)
# Start the timer -- this replaces the initial call
# to periodicCall
self.timer.start(100)
# Set up the thread to do asynchronous I/O
# More can be made if necessary
self.running = 1
_self.thread1 = threading.Thread(target=self.workerThread1)
self.thread1.start()
def periodicCall(self):
"""
Check every 100 ms if there is something new in the queue.
"""
self.gui.processIncoming()
if not self.running:
root.quit()
def endApplication(self):
self.running = 0
def workerThread1(self):
"""
This is where we handle the asynchronous I/O. For example,
it may be a 'select()'.
One important thing to remember is that the thread has to
yield control.
"""
while self.running:
# To simulate asynchronous I/O, we create a random number
# at random intervals. Replace the following 2 lines
# with the real thing.
time.sleep(rand.random() * 0.3)
msg = rand.random()
self.queue.put(msg)
rand = random.Random()
root = qt.QApplication(sys.argv)
client = ThreadedClient()
root.exec_loop()
Important in this example are the queue and the timer. Every time the QTimer ticks, a signal is sent to the periodicCall method. This method asks the GUI to process everything that has been put in the Queue by one of the worker threads.
This example is a direct translation of the Tkinter example given in the ActiveState Python Cookbook, and it shows how easy it is to adapt threaded Tkinter code to PyQt.
Qt Threading
Qt is a large but well-designed and well-documented library. Originally, it just provided the wherewithal to create GUIs—buttons, windows, and so on. Then, bit by bit, other features were added: networking classes, SQL classes, generic data containers. A String class has always been part of Qt. And now there are thread classes. Qt's thread classes have been well designed and are easy to use.
But you still have to keep in mind that classes that derive from QObject (such as all widgets) are not thread-safe. The same holds for classes that deal with the OS, such as QProcess and QSocket. The class QRegExp uses static data, and you cannot even create different instances for different threads. The same problem holds with Python's standard random module and Python's own threads, by the way.
Qt offers the following four classes:
QThread—Encapsulates a platform thread
QMutex—Serializes access to data between threads
QSemaphore—A mutex that can be accessed from more than one thread at a time
QWaitCondition—Allows a thread to signal to one or more other threads that are waiting
Apart from these classes, the class QApplication offers five functions that allow you to serialize access to the GUI thread:
lock()—Locks the GUI thread. If the GUI thread is currently locked, waits to obtain the lock.
unlock()—Unlocks the GUI thread.
locked()—Determines whether the GUI thread is currently locked.
tryLock()—Tries to lock, but returns if it's not possible.
wakeUpGUIThread()—Wakes up the GUI thread.
guiThreadAwake()—Determines whether the GUI thread is awake.
An Example
In Listing 2, we re-create the previous example, but now using PyQt threads and no Queue. This is more tricky because the threads try to send events directly to the GUI. The QThread class has the special function postEvent() for that, and every QObject class has the function customEvent(), which you must reimplement to catch those events.
Listing 2: Qt Threads and Event Posting
import sys, time, random
from qt import *
rand = random.Random()
class WorkerThread(QThread):
def __init__(self, name, receiver):
QThread.__init__(self)
self.name = name
self.receiver = receiver
self.stopped = 0
def run(self):
while not self.stopped:
time.sleep(rand.random() * 0.3)
msg = rand.random()
event = QCustomEvent(10000)
event.setData("%s: %f" % (self.name, msg))
QThread.postEvent(self.receiver, event)
def stop(self):
self.stopped = 1
class ThreadExample(QMultiLineEdit):
def __init__(self, *args):
QMultiLineEdit.__init__(self, *args)
self.setCaption("Threading Example")
self.threads = []
for name in ["t1", "t2", "t3"]:
t = WorkerThread(name, self)
t.start()
self.threads.append(t)
def customEvent(self,event):
if event.type() == 10000:
s = event.data()
self.append(s)
def __del__(self):
for t in self.threads:
running = t.running()
t.stop()
if not t.finished():
t.wait()
app = QApplication(sys.argv)
threadExample = ThreadExample()
app.setMainWidget(threadExample)
threadExample.show()
sys.exit(app.exec_loop())
As you can see, the code in Listing 2 is considerably more simple than the previous code, even if we have to carefully wait() on the running threads before exiting the application. If you remove the __del__() method, you can be sure of getting a segmentation fault every so often, or something like:
boud@calcifer:~/doc/articles/informit/threading> python qtthreads.py Traceback (most recent call last): File "qtthreads.py", line 20, in run event = QCustomEvent(10000) TypeError: 'NoneType' object is not callable Segmentation fault
Another interesting thing to note is that you cannot stop a thread forcibly. No, you cannot. Java used to have a function for that, but it's deprecated, and no Python threading library offers anything like kill for processes. The reason is that a thread can access shared data and leave that data in an undefined state when the thread is stopped. You must do the graceful thing: Set a stop flag that is checked in the run() method. If your thread hangs somewhere in the run() method, you've got a basically insoluble problem. Be aware of this.
Now we'll modify the WorkerThread class so that the threads can directly mess with the state of the GUI. To do this, they must first acquire a lock on the GUI thread, and that's done with the qApp.lock() method:
class WorkerThread(QThread):
def __init__(self, name, receiver):
QThread.__init__(self)
self.name = name
self.receiver = receiver
self.stopped = 0
def run(self):
while not self.stopped:
time.sleep(rand.random() * 0.3)
msg = rand.random()
qApp.lock()
self.receiver.append("%s: %f" %
(self.name, msg))
qApp.unlock()
def stop(self):
self.stopped = 1
The qApp object is the singleton instance of QApplication that can be accessed globally in your application.
Conclusion
There is more to threading than can be discussed in the space of an article like this; it is a subject for complete books. One thing I haven't discussed is the infamous GIL, or Global Interpreter Lock, a construct that makes multithreaded Python applications ever so much more robust, but also prohibits access to Python objects from more than one thread at a time. In effect, all access to Python objects is serialized through this construct.
Even so, I hope I have given you reason to consider using threading in your applications. Which of the possible libraries you should use depends on your requirements. If you're using a PyQt GUI anyway and don't plan on separating the GUI from the application, then QThread and friends are a fast and easy-to-program choice. On the other hand, the threading module is a Python standard.