- 3.1 Motivation for Virtual Threads
- 3.2 Virtual Thread Execution Model
- 3.3 Using Thread Class to Create Virtual Threads
- 3.4 Using Thread Builders to Create Virtual Threads
- 3.5 Using Thread Factory to Create Threads
- 3.6 Using Thread Executor Services
- 3.7 Scalability of Throughput with Virtual Threads
- 3.8 Best Practices for Using Virtual Threads
- Review Questions
This chapter is from the book
This chapter is from the book
This chapter is from the book
3.7 Scalability of Throughput with Virtual Threads
Figure 3.1b illustrates how a virtual thread executes its task by being mounted and unmounted on carrier threads during its lifetime. Example 3.7 demonstrates how virtual threads are executed by mounting on platform threads. In addition, it demonstrates the scalability of executing thousands of virtual threads.
The following line of code executed by a thread returns a string representation of the thread with pertinent information about the thread.
String vtInfo1 = Thread.currentThread().toString();
For a virtual thread, it returns pertinent information about the virtual thread in the following format:
VirtualThread[#22,VT_2]/runnable@ForkJoinPool-1-worker-2
From the string representation, we can see that the virtual thread with ID #22 is mounted on carrier thread worker-2. We can extract the name of carrier thread with this code:
String ctName1 = vtInfo1.substring(vtInfo1.indexOf('w')); // worker-2
If virtual thread #22 executes a blocking operation, it will be unmounted and when it resumes execution, it might be mounted on a different or the same carrier thread. We can check this from the string representation of the virtual thread after resumption:
String vtInfo2 = Thread.currentThread().toString();
If the string representation of the virtual thread is as below, we know that it was mounted on carrier thread worker-4.
VirtualThread[#22,VT_2]/runnable@ForkJoinPool-1-worker-4
We can extract the name of carrier thread with this code:
String ctName2 = vtInfo2.substring(vtInfo2.indexOf('w')); // worker-4
We can graphically represent the scheduling of a virtual thread from one carrier thread to another after a blocking operation as follows:
worker-2 -> worker-4
The getCarrierThreadName() static method at (7) in Example 3.7 extracts the name of the carrier thread the virtual thread is mounted on as outlined above. We call the getCarrierThreadName() method to extract the name of the carrier thread before and after each blocking operation in the task defined at (3):
String ctName1 = getCarrierThreadName(); // Carrier thread before. someBlockingOperation(); String ctName2 = getCarrierThreadName(); // Carrier thread after. ...
The scheduling of a virtual thread from one carrier thread to another after each blocking operation is printed in the same order as the blocking operations. In order to keep the output manageable, only execution of a limited number of virtual threads is printed (controlled by the INTERVAL value defined at (2)).
A sleeping operation with a duration of 1 second is implemented as the blocking operation by the method someBlockingOperation() declared at (8).
The main() method at (9) uses a one-virtual-thread-per-task executor service to submit and execute the task a fixed number of times (NUM_OF_VT defined at (1)). The main() method also computes the time the executor service used to execute the submitted tasks (i.e., the duration) at (10) and the throughput (i.e., number of tasks/duration) at (11).
From the output in Example 3.7, we can see the carrier threads that a virtual thread was mounted on to execute the task. At the resumption of execution after blocking, a virtual thread can be mounted on the same or a different carrier thread. Virtual thread #200000 was mounted consecutively on the same carrier thread twice:
Virtual Thread #200000: worker-2 -> worker-2 -> worker-5 -> worker-5
Whereas, virtual thread #300000 was mounted on different carrier threads after blocking during its lifetime.
Virtual Thread #300000: worker-3 -> worker-8 -> worker-4 -> worker-5
From the output in Example 3.7, we can see that the number of carrier threads employed by the executor service is 8 (the highest count on a carrier thread name in the output that is the same as the number of processors in this case).
Example 3.7 is running a million virtual threads (with a one-thread-per-task executor service taking a little over 54 seconds). The very high ratio of virtual threads executed to carrier threads employed to execute them results in formidable scaling of throughput, in this case a little over 18000 tasks/second. The curious reader is encouraged to experiment with different values for the number of tasks to execute, and to refactor the code to use platform threads and different executor services.
An important factor to keep in mind is that virtual threads can help to increase the throughput under the right circumstances, but they do not improve the latency—that is, they do not make each task execute faster.
Example 3.7 Virtual Thread Execution and Scalability
package vt;
import java.time.Duration;
import java.time.Instant;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.stream.IntStream;
public class VTExecutionDemo {
public static final int NUM_OF_VT = 1_000_000; // (1)
public static final int INTERVAL = NUM_OF_VT/10; // (2)
// Create the task: (3)
static final Runnable task = () -> {
// Obtain the names of carrier threads
// the virtual thread was mounted on: (4)
var ctName1 = getCarrierThreadName();
someBlockingOperation();
var ctName2 = getCarrierThreadName();
someBlockingOperation();
var ctName3 = getCarrierThreadName();
someBlockingOperation();
var ctName4 = getCarrierThreadName();
// ID of this virtual thread: (5)
var vtID = Thread.currentThread().threadId();
// Print carrier threads the virtual thread was mounted on: (6)
if (vtID % INTERVAL == 0) {
System.out.printf("Virtual Thread #%d: %s -> %s -> %s -> %s%n",
vtID, ctName1, ctName2, ctName3, ctName4);
}
};
// Get the name of the carrier thread (format: worker-n)
// the current virtual thread is mounted on. (7)
static String getCarrierThreadName() {
var vtInfo = Thread.currentThread().toString();
return vtInfo.substring(vtInfo.indexOf('w'));
}
// A blocking operation to unmount a virtual thread. (8)
static void someBlockingOperation() {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public static void main(String[] args) { // (9)
Instant start = Instant.now();
try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
IntStream.range(0, NUM_OF_VT).forEach(i -> executor.submit(task));
}
Instant finish = Instant.now();
long duration = Duration.between(start, finish).toMillis();// (10)
double throughput = (double) NUM_OF_VT / duration * 1000; // (11)
String format = """
Number of virtual threads: %d
Duration: %d ms
Throughput: %.2f tasks/s
""";
System.out.printf(format, NUM_OF_VT, duration, throughput);
}
}
Probable output from the program:
Virtual Thread #100000: worker-3 -> worker-7 -> worker-5 -> worker-7 Virtual Thread #200000: worker-2 -> worker-2 -> worker-5 -> worker-5 Virtual Thread #300000: worker-3 -> worker-8 -> worker-4 -> worker-5 Virtual Thread #400000: worker-2 -> worker-7 -> worker-8 -> worker-8 Virtual Thread #500000: worker-7 -> worker-4 -> worker-3 -> worker-2 Virtual Thread #600000: worker-1 -> worker-8 -> worker-4 -> worker-3 Virtual Thread #700000: worker-3 -> worker-7 -> worker-1 -> worker-8 Virtual Thread #800000: worker-2 -> worker-4 -> worker-5 -> worker-6 Virtual Thread #900000: worker-4 -> worker-4 -> worker-7 -> worker-8 Virtual Thread #1000000: worker-5 -> worker-5 -> worker-5 -> worker-7 Number of virtual threads: 1000000 Duration: 54407 ms Throughput: 18379.99 tasks/s
){kind=link}