Home > Articles > Programming > Python

  • Print
  • + Share This
This chapter is from the book

4.5. The threading Module

We will now introduce the higher-level threading module, which gives you not only a Thread class but also a wide variety of synchronization mechanisms to use to your heart’s content. Table 4-2 presents a list of all the objects available in the threading module.

Table 4-2. threading Module Objects

Object

Description

Thread

Object that represents a single thread of execution

Lock

Primitive lock object (same lock as in thread module)

RLock

Re-entrant lock object provides ability for a single thread to (re)acquire an already-held lock (recursive locking)

Condition

Condition variable object causes one thread to wait until a certain “condition” has been satisfied by another thread, such as changing of state or of some data value

Event

General version of condition variables, whereby any number of threads are waiting for some event to occur and all will awaken when the event happens

Semaphore

Provides a “counter” of finite resources shared between threads; block when none are available

BoundedSemaphore

Similar to a Semaphore but ensures that it never exceeds its initial value

Timer

Similar to Thread, except that it waits for an allotted period of time before running

Barriera

Creates a “barrier,” at which a specified number of threads must all arrive before they’re all allowed to continue

170fig01.jpg

In this section, we will examine how to use the Thread class to implement threading. Because we have already covered the basics of locking, we will not cover the locking primitives here. The Thread() class also contains a form of synchronization, so explicit use of locking primitives is not necessary.

4.5.1. The Thread Class

The Thread class of the threading module is your primary executive object. It has a variety of functions not available to the thread module. Table 4-3 presents a list of attributes and methods.

Table 4-3. Thread Object Attributes and Methods

Attribute

Description

Thread object data attributes

 

name

The name of a thread.

ident

The identifier of a thread.

daemon

Boolean flag indicating whether a thread is daemonic.

Thread object methods

 

__init__(group=None, target=None, name=None, args=(), kwargs={}, verbose=None, daemon=None)c

Instantiate a Thread object, taking target callable and any args or kwargs. A name or group can also be passed but the latter is unimplemented. A verbose flag is also accepted. Any daemon value sets the thread.daemon attribute/flag.

start()

Begin thread execution.

run()

Method defining thread functionality (usually overridden by application writer in a subclass).

join(timeout=None)

Suspend until the started thread terminates; blocks unless timeout (in seconds) is given.

getName()a

Return name of thread.

setName(name)a

Set name of thread.

isAlive/is_alive()b

Boolean flag indicating whether thread is still running.

isDaemon()c

Return True if thread daemonic, False otherwise.

setDaemon(daemonic)c

Set the daemon flag to the given Boolean daemonic value (must be called before thread start().

There are a variety of ways by which you can create threads using the Thread class. We cover three of them here, all quite similar. Pick the one you feel most comfortable with, not to mention the most appropriate for your application and future scalability (we like the final choice the best):

  • Create Thread instance, passing in function
  • Create Thread instance, passing in callable class instance
  • Subclass Thread and create subclass instance

You’ll discover that you will pick either the first or third option. The latter is chosen when a more object-oriented interface is desired and the former, otherwise. The second, honestly, is a bit more awkward and slightly harder to read, as you’ll discover.

Create Thread Instance, Passing in Function

In our first example, we will just instantiate Thread, passing in our function (and its arguments) in a manner similar to our previous examples. This function is what will be executed when we direct the thread to begin execution. Taking our mtsleepB.py script from Example 4-3 and tweaking it by adding the use of Thread objects, we have mtsleepC.py, as shown in Example 4-4.

Example 4-4. Using the threading Module (mtsleepC.py)

The Thread class from the threading module has a join() method that lets the main thread wait for thread completion.

1    #!/usr/bin/env python
2
3    import threading
4    from time import sleep, ctime
5
6    loops = [4,2]
7
8    def loop(nloop, nsec):
9        print 'start loop', nloop, 'at:', ctime()
10       sleep(nsec)
11       print 'loop', nloop, 'done at:', ctime()
12
13   def main():
14       print 'starting at:', ctime()
15       threads = []
16       nloops = range(len(loops))
17
18       for i in nloops:
19           t = threading.Thread(target=loop,
20               args=(i, loops[i]))
21           threads.append(t)
22
23       for i in nloops:          # start threads
24           threads[i].start()
25
26       for i in nloops:          # wait for all
27           threads[i].join()     # threads to finish
28
29       print 'all DONE at:', ctime()
30
31   if __name__ == '__main__':
32       main()

When we run the script in Example 4-4, we see output similar to that of its predecessors:

$ mtsleepC.py
starting at: Sun Aug 13 18:16:38 2006
start loop 0 at: Sun Aug 13 18:16:38 2006
start loop 1 at: Sun Aug 13 18:16:38 2006
loop 1 done at: Sun Aug 13 18:16:40 2006
loop 0 done at: Sun Aug 13 18:16:42 2006
all DONE at: Sun Aug 13 18:16:42 2006

So what did change? Gone are the locks that we had to implement when using the thread module. Instead, we create a set of Thread objects. When each Thread is instantiated, we dutifully pass in the function (target) and arguments (args) and receive a Thread instance in return. The biggest difference between instantiating Thread (calling Thread()) and invoking thread.start_new_thread() is that the new thread does not begin execution right away. This is a useful synchronization feature, especially when you don’t want the threads to start immediately.

Once all the threads have been allocated, we let them go off to the races by invoking each thread’s start() method, but not a moment before that. And rather than having to manage a set of locks (allocating, acquiring, releasing, checking lock state, etc.), we simply call the join() method for each thread. join() will wait until a thread terminates, or, if provided, a timeout occurs. Use of join() appears much cleaner than an infinite loop that waits for locks to be released (which is why these locks are sometimes known as spin locks).

One other important aspect of join() is that it does not need to be called at all. Once threads are started, they will execute until their given function completes, at which point, they will exit. If your main thread has things to do other than wait for threads to complete (such as other processing or waiting for new client requests), it should do so. join() is useful only when you want to wait for thread completion.

Create Thread Instance, Passing in Callable Class Instance

A similar offshoot to passing in a function when creating a thread is having a callable class and passing in an instance for execution—this is the more object-oriented approach to MT programming. Such a callable class embodies an execution environment that is much more flexible than a function or choosing from a set of functions. You now have the power of a class object behind you, as opposed to a single function or a list/tuple of functions.

Adding our new class ThreadFunc to the code and making other slight modifications to mtsleepC.py, we get mtsleepD.py, shown in Example 4-5.

Example 4-5. Using Callable Classes (mtsleepD.py)

In this example, we pass in a callable class (instance) as opposed to just a function. It presents more of an object-oriented approach than mtsleepC.py.

1    #!/usr/bin/env python
2
3    import threading
4    from time import sleep, ctime
5
6    loops = [4,2]
7
8    class ThreadFunc(object):
9
10       def __init__(self, func, args, name=''):
11           self.name = name
12           self.func = func
13           self.args = args
14
15       def __call__(self):
16           self.func(*self.args)
17
18   def loop(nloop, nsec):
19       print 'start loop', nloop, 'at:', ctime()
20       sleep(nsec)
21       print 'loop', nloop, 'done at:', ctime()
22
23   def main():
24       print 'starting at:', ctime()
25       threads = []
26       nloops = range(len(loops))
27
28       for i in nloops:  # create all threads
29           t = threading.Thread(
30               target=ThreadFunc(loop, (i, loops[i]),
31               loop.__name__))
32           threads.append(t)
33
34       for i in nloops:  # start all threads
35           threads[i].start()
36
37       for i in nloops:  # wait for completion
38           threads[i].join()
39
40       print 'all DONE at:', ctime()
41
42   if __name__ == '__main__':
43       main()

When we run mtsleepD.py, we get the expected output:

$ mtsleepD.py
starting at: Sun Aug 13 18:49:17 2006
start loop 0 at: Sun Aug 13 18:49:17 2006
start loop 1 at: Sun Aug 13 18:49:17 2006
loop 1 done at: Sun Aug 13 18:49:19 2006
loop 0 done at: Sun Aug 13 18:49:21 2006
all DONE at: Sun Aug 13 18:49:21 2006

So what are the changes this time? The addition of the ThreadFunc class and a minor change to instantiate the Thread object, which also instantiates ThreadFunc, our callable class. In effect, we have a double instantiation going on here. Let’s take a closer look at our ThreadFunc class.

We want to make this class general enough to use with functions other than our loop() function, so we added some new infrastructure, such as having this class hold the arguments for the function, the function itself, and also a function name string. The constructor __init__() just sets all the values.

When the Thread code calls our ThreadFunc object because a new thread is created, it will invoke the __call__() special method. Because we already have our set of arguments, we do not need to pass it to the Thread() constructor and can call the function directly.

Subclass Thread and Create Subclass Instance

The final introductory example involves subclassing Thread(), which turns out to be extremely similar to creating a callable class as in the previous example. Subclassing is a bit easier to read when you are creating your threads (lines 29–30). We will present the code for mtsleepE.py in Example 4-6 as well as the output obtained from its execution, and leave it as an exercise for you to compare mtsleepE.py to mtsleepD.py.

Example 4-6. Subclassing Thread (mtsleepE.py)

Rather than instantiating the Thread class, we subclass it. This gives us more flexibility in customizing our threading objects and simplifies the thread creation call.

1    #!/usr/bin/env python
2
3    import threading
4    from time import sleep, ctime
5
6    loops = (4, 2)
7
8    class MyThread(threading.Thread):
9        def __init__(self, func, args, name=''):
10           threading.Thread.__init__(self)
11           self.name = name
12           self.func = func
13           self.args = args
14
15       def run(self):
16           self.func(*self.args)
17
18   def loop(nloop, nsec):
19       print 'start loop', nloop, 'at:', ctime()
20       sleep(nsec)
21       print 'loop', nloop, 'done at:', ctime()
22
23   def main():
24       print 'starting at:', ctime()
25       threads = []
26       nloops = range(len(loops))
27
28       for i in nloops:
29           t = MyThread(loop, (i, loops[i]),
30               loop.__name__)
31           threads.append(t)
32
33       for i in nloops:
34           threads[i].start()
35
36       for i in nloops:
37           threads[i].join()
38
39       print 'all DONE at:', ctime()'
40
41   if __name__ == '__main__':
42       main()

Here is the output for mtsleepE.py. Again, it’s just as we expected:

$ mtsleepE.py
starting at: Sun Aug 13 19:14:26 2006
start loop 0 at: Sun Aug 13 19:14:26 2006
start loop 1 at: Sun Aug 13 19:14:26 2006
loop 1 done at: Sun Aug 13 19:14:28 2006
loop 0 done at: Sun Aug 13 19:14:30 2006
all DONE at: Sun Aug 13 19:14:30 2006

While you compare the source between the mtsleep4 and mtsleep5 modules, we want to point out the most significant changes: 1) our MyThread subclass constructor must first invoke the base class constructor (line 9), and 2) the former special method __call__() must be called run() in the subclass.

We now modify our MyThread class with some diagnostic output and store it in a separate module called myThread (look ahead to Example 4-7) and import this class for the upcoming examples. Rather than simply calling our functions, we also save the result to instance attribute self.res, and create a new method to retrieve that value, getResult().

Example 4-7. MyThread Subclass of Thread (myThread.py)

To generalize our subclass of Thread from mtsleepE.py, we move the subclass to a separate module and add a getResult() method for callables that produce return values.

1    #!/usr/bin/env python
2
3    import threading
4    from time import ctime
5
6    class MyThread(threading.Thread):
7       def __init__(self, func, args, name=''):
8           threading.Thread.__init__(self)
9           self.name = name
10          self.func = func
11          self.args = args
12
13      def getResult(self):
14          return self.res
15
16      def run(self):
17          print 'starting', self.name, 'at:', 18              ctime()
19          self.res = self.func(*self.args)
20          print self.name, 'finished at:', 21              ctime()

4.5.2. Other Threading Module Functions

In addition to the various synchronization and threading objects, the Threading module also has some supporting functions, as detailed in Table 4-4.

Table 4-4. threading Module Functions

Function

Description

activeCount/active_count()a

Number of currently active Thread objects

currentThread()/current_threada

Returns the current Thread object

enumerate()

Returns list of all currently active Threads

settrace(func)b

Sets a trace function for all threads

setprofile(func)b

Sets a profile function for all threads

stack_size(size=0)c

Returns stack size of newly created threads; optional size can be set for subsequently created threads

  • + Share This
  • 🔖 Save To Your Account

InformIT Promotional Mailings & Special Offers

I would like to receive exclusive offers and hear about products from InformIT and its family of brands. I can unsubscribe at any time.

Overview


Pearson Education, Inc., 221 River Street, Hoboken, New Jersey 07030, (Pearson) presents this site to provide information about products and services that can be purchased through this site.

This privacy notice provides an overview of our commitment to privacy and describes how we collect, protect, use and share personal information collected through this site. Please note that other Pearson websites and online products and services have their own separate privacy policies.

Collection and Use of Information


To conduct business and deliver products and services, Pearson collects and uses personal information in several ways in connection with this site, including:

Questions and Inquiries

For inquiries and questions, we collect the inquiry or question, together with name, contact details (email address, phone number and mailing address) and any other additional information voluntarily submitted to us through a Contact Us form or an email. We use this information to address the inquiry and respond to the question.

Online Store

For orders and purchases placed through our online store on this site, we collect order details, name, institution name and address (if applicable), email address, phone number, shipping and billing addresses, credit/debit card information, shipping options and any instructions. We use this information to complete transactions, fulfill orders, communicate with individuals placing orders or visiting the online store, and for related purposes.

Surveys

Pearson may offer opportunities to provide feedback or participate in surveys, including surveys evaluating Pearson products, services or sites. Participation is voluntary. Pearson collects information requested in the survey questions and uses the information to evaluate, support, maintain and improve products, services or sites, develop new products and services, conduct educational research and for other purposes specified in the survey.

Contests and Drawings

Occasionally, we may sponsor a contest or drawing. Participation is optional. Pearson collects name, contact information and other information specified on the entry form for the contest or drawing to conduct the contest or drawing. Pearson may collect additional personal information from the winners of a contest or drawing in order to award the prize and for tax reporting purposes, as required by law.

Newsletters

If you have elected to receive email newsletters or promotional mailings and special offers but want to unsubscribe, simply email information@informit.com.

Service Announcements

On rare occasions it is necessary to send out a strictly service related announcement. For instance, if our service is temporarily suspended for maintenance we might send users an email. Generally, users may not opt-out of these communications, though they can deactivate their account information. However, these communications are not promotional in nature.

Customer Service

We communicate with users on a regular basis to provide requested services and in regard to issues relating to their account we reply via email or phone in accordance with the users' wishes when a user submits their information through our Contact Us form.

Other Collection and Use of Information


Application and System Logs

Pearson automatically collects log data to help ensure the delivery, availability and security of this site. Log data may include technical information about how a user or visitor connected to this site, such as browser type, type of computer/device, operating system, internet service provider and IP address. We use this information for support purposes and to monitor the health of the site, identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents and appropriately scale computing resources.

Web Analytics

Pearson may use third party web trend analytical services, including Google Analytics, to collect visitor information, such as IP addresses, browser types, referring pages, pages visited and time spent on a particular site. While these analytical services collect and report information on an anonymous basis, they may use cookies to gather web trend information. The information gathered may enable Pearson (but not the third party web trend services) to link information with application and system log data. Pearson uses this information for system administration and to identify problems, improve service, detect unauthorized access and fraudulent activity, prevent and respond to security incidents, appropriately scale computing resources and otherwise support and deliver this site and its services.

Cookies and Related Technologies

This site uses cookies and similar technologies to personalize content, measure traffic patterns, control security, track use and access of information on this site, and provide interest-based messages and advertising. Users can manage and block the use of cookies through their browser. Disabling or blocking certain cookies may limit the functionality of this site.

Do Not Track

This site currently does not respond to Do Not Track signals.

Security


Pearson uses appropriate physical, administrative and technical security measures to protect personal information from unauthorized access, use and disclosure.

Children


This site is not directed to children under the age of 13.

Marketing


Pearson may send or direct marketing communications to users, provided that

  • Pearson will not use personal information collected or processed as a K-12 school service provider for the purpose of directed or targeted advertising.
  • Such marketing is consistent with applicable law and Pearson's legal obligations.
  • Pearson will not knowingly direct or send marketing communications to an individual who has expressed a preference not to receive marketing.
  • Where required by applicable law, express or implied consent to marketing exists and has not been withdrawn.

Pearson may provide personal information to a third party service provider on a restricted basis to provide marketing solely on behalf of Pearson or an affiliate or customer for whom Pearson is a service provider. Marketing preferences may be changed at any time.

Correcting/Updating Personal Information


If a user's personally identifiable information changes (such as your postal address or email address), we provide a way to correct or update that user's personal data provided to us. This can be done on the Account page. If a user no longer desires our service and desires to delete his or her account, please contact us at customer-service@informit.com and we will process the deletion of a user's account.

Choice/Opt-out


Users can always make an informed choice as to whether they should proceed with certain services offered by InformIT. If you choose to remove yourself from our mailing list(s) simply visit the following page and uncheck any communication you no longer want to receive: www.informit.com/u.aspx.

Sale of Personal Information


Pearson does not rent or sell personal information in exchange for any payment of money.

While Pearson does not sell personal information, as defined in Nevada law, Nevada residents may email a request for no sale of their personal information to NevadaDesignatedRequest@pearson.com.

Supplemental Privacy Statement for California Residents


California residents should read our Supplemental privacy statement for California residents in conjunction with this Privacy Notice. The Supplemental privacy statement for California residents explains Pearson's commitment to comply with California law and applies to personal information of California residents collected in connection with this site and the Services.

Sharing and Disclosure


Pearson may disclose personal information, as follows:

  • As required by law.
  • With the consent of the individual (or their parent, if the individual is a minor)
  • In response to a subpoena, court order or legal process, to the extent permitted or required by law
  • To protect the security and safety of individuals, data, assets and systems, consistent with applicable law
  • In connection the sale, joint venture or other transfer of some or all of its company or assets, subject to the provisions of this Privacy Notice
  • To investigate or address actual or suspected fraud or other illegal activities
  • To exercise its legal rights, including enforcement of the Terms of Use for this site or another contract
  • To affiliated Pearson companies and other companies and organizations who perform work for Pearson and are obligated to protect the privacy of personal information consistent with this Privacy Notice
  • To a school, organization, company or government agency, where Pearson collects or processes the personal information in a school setting or on behalf of such organization, company or government agency.

Links


This web site contains links to other sites. Please be aware that we are not responsible for the privacy practices of such other sites. We encourage our users to be aware when they leave our site and to read the privacy statements of each and every web site that collects Personal Information. This privacy statement applies solely to information collected by this web site.

Requests and Contact


Please contact us about this Privacy Notice or if you have any requests or questions relating to the privacy of your personal information.

Changes to this Privacy Notice


We may revise this Privacy Notice through an updated posting. We will identify the effective date of the revision in the posting. Often, updates are made to provide greater clarity or to comply with changes in regulatory requirements. If the updates involve material changes to the collection, protection, use or disclosure of Personal Information, Pearson will provide notice of the change through a conspicuous notice on this site or other appropriate way. Continued use of the site after the effective date of a posted revision evidences acceptance. Please contact us if you have questions or concerns about the Privacy Notice or any objection to any revisions.

Last Update: November 17, 2020