Java’s threading model has been the same for twenty-five years. One Java thread maps to one OS thread. The OS schedules it, the JVM wraps it, and you pay the cost — roughly 512KB to 1MB of stack memory per thread, plus the overhead of OS-level context switches. For most applications this never mattered. Then came the era of microservices, where every service makes I/O calls — databases, HTTP clients, message queues — and threads spend most of their time waiting rather than computing.
This is not a Java problem. It is a fundamental mismatch between how threads were designed (to run continuously) and how most backend work actually behaves (run a little, wait a lot). Java 21 addresses this with virtual threads. To understand what changes and what does not, you need to understand what came before and why it was the right choice at the time.
Traditional Thread Creation: What the JVM Actually Does
When you call new Thread(runnable).start(), the JVM makes a system call to create an OS thread. The OS allocates a native stack — typically 512KB minimum on Linux — and the kernel scheduler manages it independently of what your application is doing. This works well when threads stay busy computing. The problem is blocking I/O. When a thread issues a JDBC query and waits for the response, it is parked by the OS. The OS still tracks it, maintains its stack, includes it in scheduling decisions. You are paying for a resource that is doing nothing.
Creating threads ad hoc has an obvious ceiling: at 512KB per thread, 10,000 threads consume 5GB of memory, and the OS scheduler degrades badly well before that point. The practical limit on most JVM deployments is 200 to 500 threads before throughput starts falling.
// Java 8+ — raw thread creation; acceptable for background tasks, not per-request work
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import javax.sql.DataSource;
public class PlatformThreadExample {
private static final Logger log = LoggerFactory.getLogger(PlatformThreadExample.class);
private final DataSource dataSource;
public PlatformThreadExample(DataSource dataSource) {
this.dataSource = dataSource;
}
public void handleRequest(int userId) {
Thread worker = new Thread(() -> {
log.info("Handling request on: {}", Thread.currentThread().getName());
// This JDBC call blocks the OS thread for its entire duration
// The thread is parked — stack memory still allocated, OS still tracking it
String result = fetchUser(userId);
log.info("Done: {}", result);
});
worker.start();
}
private String fetchUser(int userId) {
try (Connection conn = dataSource.getConnection();
PreparedStatement ps = conn.prepareStatement("SELECT name FROM users WHERE id = ?")) {
ps.setInt(1, userId);
ResultSet rs = ps.executeQuery(); // OS thread parks here waiting for DB response
return rs.next() ? rs.getString("name") : "unknown";
} catch (Exception e) {
log.error("DB error for user {}", userId, e);
return "error";
}
}
}Code language: Java (java)ExecutorService: The Right Abstraction for Platform Threads
Thread pools solved the per-request creation overhead problem. Instead of creating and destroying an OS thread per task, you maintain a fixed set and reuse them. ExecutorService manages this pool and queues work when all threads are busy. This is the correct model for platform threads and it remains so today.
// Java 8+ — fixed thread pool; correct model for CPU-bound or mixed workloads
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class ExecutorServiceExample {
private static final Logger log = LoggerFactory.getLogger(ExecutorServiceExample.class);
// cores * 2 is a common starting point for I/O-mixed workloads
// For pure CPU work: cores only — more threads adds contention, not throughput
private static final ExecutorService POOL = Executors.newFixedThreadPool(
Runtime.getRuntime().availableProcessors() * 2
);
public Future<String> submitRequest(int userId) {
return POOL.submit(() -> {
log.info("Processing userId={} on: {}", userId, Thread.currentThread().getName());
// If all pool threads are blocked on JDBC, new tasks queue here
// Pool size becomes the system-wide throughput ceiling
return fetchFromDatabase(userId);
});
}
private String fetchFromDatabase(int userId) {
// Blocking JDBC call — pool thread is occupied for the full query duration
return "user-" + userId;
}
}Code language: Java (java)The pool model works, but pool size becomes a system-wide ceiling. If all 200 threads are blocked waiting for database responses, the 201st request queues. Under sustained load the queue grows, latency rises, and requests hit timeout thresholds. Increasing pool size moves the ceiling rather than removing it. Reactive frameworks (Project Reactor, RxJava) solved this differently — by making I/O non-blocking at the call site — but that changes how you write and debug code significantly, and stack traces become largely useless when the call chain is assembled asynchronously.
Virtual Threads: How the JVM Takes Over Thread Scheduling
Java 21 introduces virtual threads (JEP 444) as a production-ready feature. Virtual threads are lightweight JVM-managed threads multiplexed onto a smaller pool of OS threads called carrier threads. The number of carrier threads defaults to the number of available CPU cores.
When a virtual thread performs a blocking operation — a JDBC query, an HTTP client call, a file read — the JVM detects the block and unmounts the virtual thread from its carrier thread. The virtual thread’s execution state is saved as a continuation on the heap. The carrier immediately picks up another runnable virtual thread. When the blocking operation completes, the original virtual thread is rescheduled onto the next available carrier and resumes from exactly where it left off. The memory cost is hundreds of bytes to a few kilobytes per virtual thread — orders of magnitude less than an OS thread. You can create millions without exhausting memory, and context switching happens in JVM space rather than kernel space.
// Java 21+ — identical blocking code; JVM handles multiplexing transparently
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class VirtualThreadExample {
private static final Logger log = LoggerFactory.getLogger(VirtualThreadExample.class);
public void handleRequests(List<Integer> userIds) throws Exception {
// newVirtualThreadPerTaskExecutor: one virtual thread per submitted task
// No pool size tuning — the JVM manages carrier threads automatically
try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
List<Future<String>> futures = new ArrayList<>();
for (int userId : userIds) {
futures.add(executor.submit(() -> processUser(userId)));
}
for (Future<String> f : futures) {
log.info("Result: {}", f.get());
}
}
}
private String processUser(int userId) {
log.debug("Processing userId={} on: {}", userId, Thread.currentThread());
// This JDBC call unmounts the virtual thread from its carrier
// Carrier is free to run other virtual threads during the DB wait
return fetchFromDatabase(userId);
}
private String fetchFromDatabase(int userId) {
// Blocking JDBC call — virtual thread parks, carrier does NOT park
return "user-" + userId;
}
}Code language: Java (java)Enabling Virtual Threads in Spring Boot
In Spring Boot 3.2+, switching the entire web layer to virtual threads is a single property. All HTTP request handling threads, @Async methods, and scheduled tasks use virtual threads automatically.
// Java 21+ / Spring Boot 3.2+
// application.properties:
// spring.threads.virtual.enabled=true
// For explicit control over specific executor beans:
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
@Configuration
public class ThreadConfig {
@Bean
public Executor applicationTaskExecutor() {
// @Async methods and Spring task scheduling use virtual threads
return Executors.newVirtualThreadPerTaskExecutor();
}
}Code language: Java (java)Where Virtual Threads Do Not Help — and Where They Can Hurt
Virtual threads solve one specific problem: threads blocked on I/O. For CPU-bound work — image processing, cryptography, sorting large in-memory datasets — a virtual thread provides no benefit over a platform thread. The thread is actively consuming CPU, not waiting. The JVM cannot run another virtual thread on the same carrier while it is executing. Creating millions of virtual threads for CPU-bound tasks increases overhead without increasing throughput.
Two production risks to be aware of. Pinning: if a virtual thread holds a synchronized lock and then encounters a blocking operation, it cannot unmount. The carrier thread stays blocked until both the lock is released and the operation returns. Under high concurrency this exhausts the carrier pool — CPU core count by default — and produces throughput degradation similar to thread starvation. Detect pinning events at runtime with the JVM flag -Djdk.tracePinnedThreads=full. The fix is replacing synchronized blocks that contain I/O with ReentrantLock, which allows unmounting.
ThreadLocal misuse: libraries that cache expensive objects in ThreadLocal assume threads are pooled and reused — one cached instance per thread, shared across many requests. With virtual threads created per task, each request gets its own ThreadLocal slot. At high concurrency this creates as many entries as concurrent tasks, consuming memory proportional to request rate rather than pool size. Java 21’s ScopedValue is the designed replacement for passing immutable context through a call chain without these thread-lifecycle assumptions.
Choosing the Right Threading Model
For I/O-bound work — web handlers, database queries, HTTP client calls, message consumers — use virtual threads. The thread-count ceiling disappears and the code stays synchronous. For CPU-bound work, use a fixed platform thread pool sized to available cores. Adding more threads than cores does not add compute; it adds context-switch overhead. For applications that mix both, keep them separate: virtual threads for the I/O layer, a bounded platform pool for the compute layer — submit CPU-heavy work to the platform pool and await the result from the virtual thread.
Interview Questions
What happens to a virtual thread when it calls a blocking JDBC method?
The JVM scheduler detects the blocking call and unmounts the virtual thread from its carrier OS thread. The continuation — the thread’s stack state — is saved on the heap. The carrier immediately picks up another runnable virtual thread. When the JDBC call completes, the original virtual thread is made runnable and scheduled onto the next available carrier. Execution resumes on the next line after the blocking call, on potentially a different carrier thread, transparently from the perspective of the application code.
How does the system behave when a virtual thread holds a synchronized lock and encounters a blocking I/O call?
The virtual thread is pinned to its carrier and cannot unmount because the JVM cannot relocate a thread holding a monitor lock. The carrier stays blocked until both the I/O completes and the synchronized block exits. Under concurrent load this exhausts the carrier pool — CPU core count by default — producing throughput collapse similar to platform thread starvation. The JVM logs pinning events when block duration exceeds a threshold, detectable with JVM trace flags. The fix is replacing synchronized blocks that contain I/O with ReentrantLock.
What issues arise when a library caches objects in ThreadLocal and the application switches to virtual threads?
Platform thread pools are small and threads are reused, so ThreadLocal effectively caches one object per thread across many requests. With virtual threads created per task, each task gets its own ThreadLocal slot. At high concurrency this produces as many cached instances as concurrent tasks — memory proportional to request rate rather than pool size. ScopedValue is the designed replacement for request-scoped immutable context without thread-lifecycle assumptions.
Summary
Platform threads and ExecutorService remain the right choice for CPU-bound work. Virtual threads are the right choice when your bottleneck is threads waiting on I/O — which describes the majority of backend Java applications. The programming model stays synchronous, debugging stays tractable, the thread-count ceiling disappears. The main risks to manage are synchronized blocks containing I/O (pinning) and ThreadLocal-heavy libraries — both are detectable with JVM tooling and fixable without rewriting your application.
