Netty Thread Model

Netty operates based on an EventLoop. Here, an EventLoop refers to an infinite loop thread for executing events.

Events generated by objects are loaded into an event queue, and the event loop retrieves and executes events from the event queue when they are present. This is the basic concept of an event loop. Event loops are divided into single-threaded event loops and multi-threaded event loops depending on the type of thread they support. This is known as the reactor pattern.

Single-Threaded Event Loop #

Currently, systems equipped with multi-core CPUs are common, so modern applications often utilize sophisticated multithreading techniques to efficiently leverage system resources. In the early days of Java, the multithreading system was rudimentary, creating and starting new threads whenever needed to execute concurrent units of work, leading to severe performance degradation under heavy load. This problem was resolved with the introduction of the Executor API in Java 5, which significantly improved performance through thread caching and reuse by supporting thread pools.

A single-threaded event loop refers to a state where there is only one thread processing events.

• To execute a requested task (an implementation of Runnable), one thread is selected from the pool's available list and assigned.
• Once the task is complete, the thread is returned to the list and can be reused.

While pooling and reusing threads is clearly an improvement over creating and deleting threads for each task, it does not eliminate context switching costs. These costs become evident as the number of threads increases and can become a serious problem under heavy load. Furthermore, due to application concurrency requirements and overall complexity, other threading issues may arise throughout the project lifecycle.

Multi-Threaded Event Loop #

A multi-threaded event loop is a model where multiple threads process events. Although its framework implementation is more complex than a single-threaded event loop, event loop threads execute event methods in parallel, efficiently utilizing multi-core CPUs.

Disadvantages include potential thread contention when multiple event loop threads access a single event queue, similar to when multiple threads share one resource. Also, because events are processed in parallel, there can be a mismatch between the order of event publication and execution.

Netty resolves the issue of mismatched publication and execution order, a drawback of multi-threaded event loops, as follows.

• Netty events are published by a Channel.
• Each EventLoop object has its own event queue.
• A Netty Channel is registered with a single EventLoop.
• Multiple Channels can be registered with a single EventLoop thread.
• All operations during a Channel's lifecycle are handled by a single thread.

In a multi-threaded event model, the reason for mismatched event execution order is that loops share an event queue. Netty addresses this by giving each EventLoop thread its own event queue, allowing the specific thread assigned to handle an event to process it, thus eliminating the need for threads to access a shared event queue.

EventLoop Interface #

Executing tasks that process events occurring during the lifetime of a connection is a fundamental function of networking frameworks. The programming construct representing this is called an Event Loop, and Netty applies this concept in its io.netty.channel.EventLoop interface.

Netty's EventLoop is designed leveraging two fundamental APIs: concurrency and networking.

1. The io.netty.util.concurrent package provides thread executors based on the JDK package java.util.concurrent.
2. Classes in the io.netty.channel package extend these APIs to perform Channel events and interactions.

In this model, an EventLoop operates with a single, immutable Thread, and tasks can be submitted directly to the EventLoop implementation for immediate or scheduled execution. Depending on the configuration and available cores, multiple EventLoops may be created to optimize resource utilization, and a single EventLoop might be assigned to serve multiple Channels.

Netty's EventLoop operates with a single, immutable Thread, and tasks (Runnable or Callable) can be submitted directly to the EventLoop implementation for immediate or scheduled execution. Depending on the configuration and available cores, multiple EventLoops may be created to optimize resource utilization, and a single EventLoop might be assigned to serve multiple Channels.

Netty's EventLoop extends ScheduledExecutorService and defines only one method: parent(). This method is intended to return a reference to the EventLoopGroup to which the current EventLoop implementation instance belongs, as shown in the following code.

public interface EventGroup extends EventExecutor, EventLoopGroup {
    @Override
    EventLoopGroup parent();
}

In Netty 4, all I/O operations and events are handled by the thread assigned to the EventLoop.

Thread Management #

A key factor in Netty's threading model achieving excellent performance is its ability to check the ID of the currently executing thread, i.e., to verify if the thread is assigned to the current Channel's EventLoop. (The EventLoop processes all events for a single Channel throughout its lifecycle.)

If the calling thread belongs to the EventLoop, the corresponding code block is executed; otherwise, the EventLoop schedules the task for later execution and places it in an internal queue. The EventLoop executes items in the queue when it processes the next relevant event. This mechanism allows threads to interact directly with Channels without synchronizing ChannelHandlers.

Long-running tasks should not be placed in the execution queue; otherwise, other tasks on the same thread will be blocked from executing. If blocking calls or long-running tasks are necessary, it is advisable to use a dedicated EventExecutor.

EventLoop Thread Assignment #

The EventLoop that supports events and I/O for a Channel is contained within an EventLoopGroup. How EventLoops are created and assigned depends on the transport implementation.

• Async transport
  • Asynchronous implementations utilize a small number of EventLoops, which can be shared across multiple Channels in the current model. This allows supporting numerous Channels with a minimal number of threads, without assigning a thread per Channel.
  • Once a Channel is assigned an EventLoop, it uses that assigned EventLoop throughout its lifecycle. This eliminates the need to worry about synchronization and thread safety in ChannelHandler implementations.
  • It's also important to understand the implications of EventLoop assignment for ThreadLocal usage. Generally, since one EventLoop is used by more than one Channel, ThreadLocal values will be the same across all connected Channels. This means it's not suitable for implementing features like state tracking, but it can be useful in stateless environments for sharing large or expensive objects or events across multiple Channels.
• Blocking transport
  • One EventLoop is assigned to each Channel. However, similarly, I/O events for each Channel are processed by a single thread.