Differences between iptables and ipvs

iptables #

iptables is a packet filtering and firewall system that operates on top of the Linux kernel's netfilter framework.

As the name suggests, it's a filter-based firewall that inspects packets and allows/blocks/modifies them according to rules.

• L3/L4-centric, based on IP, port, and protocol
• Capable of state tracking based on connection tracking
• Supports NAT and MASQUERADE
• Intended for single-machine firewall/routing manipulation

ipvs #

ipvs is a kernel module that provides high-performance L4-level load balancing as part of the Linux kernel's LVS (Load Balancing) technology.

In Kubernetes, it appears when kube-proxy uses the ipvs mode instead of iptables.

• High-performance L4-based load balancer
• Hash-based scheduling (provides various algorithms like LC, WRR, SH, MH)
• Faster processing by minimizing the connection table
• Much more efficient than iptables in high-traffic environments

k8s iptables vs ipvs #

The Kubernetes kube-proxy mode can be configured with two options: iptables or ipvs.

iptables #

• Routes Service -> Pod using thousands of iptables rules.
• As the number of rules increases, the cost of packet matching grows.
• Simple, but has performance limitations with large-scale traffic.

ipvs #

• Faster than iptables because it uses kernel hash-based lookups.
• Offers functional advantages such as health checks and connection handling.
• Provides overwhelmingly superior performance when the number of Services / Pods is large.

Therefore, the official documentation recommends ipvs as the preferred mode for large clusters.

Structural issues with iptables and connection tracking #

iptables uses netfilter's conntrack.

When a new packet arrives, it queries the conntrack table and sequentially searches through chained iptables rules. This leads to performance degradation if there are many rules.

The problem is that performance degrades linearly with the number of rules.

In ipvs, the connection structure is different; ipvs does not heavily rely on conntrack for every connection like iptables. Instead, it uses two tables:

1. Service Table
2. Connection Table

Both of these are hash table-based structures.

Here, the conntrack table is necessary because packets don't actually move as single fragments but as units of connection.

For example, TCP involves SYN, SYN-ACK, ACK, DATA, FIN. UDP involves client-request-response relationships. NAT involves client-server mappings.

The OS needs to remember these to enable NAT and firewalls. So the kernel stores the following, which is conntrack:

(client_ip, client_port, protocol) <-> (server_ip, server_port)

You can check this directly in Linux:

/proc/net/nf_conntrack
or
/proc/net/ip_conntrack

IPVS connection cost minimization method #

It stores Services (IP:port) in a hash table using the Service as a key.

• key: VIP (ClusterIP), port, protocol
• value: backend list

When a packet arrives: Service -> Backend matching occurs in O(1) time, without sequentially reading through all rule chains like iptables.

The connection entry creation method is also different: ipvs creates a connection entry only when a specific connection first arrives. Subsequent packets are then looked up instantly in the ipvs connection table in O(1) time, without going through conntrack. This means it only stores necessary connections and doesn't touch the entire system like iptables.

For connectionless packets like UDP, it creates very few entries, and its operation itself supports various modes like DR and TUN, not just NAT.

DR (Direct Routing) sends requests from LB -> Backend, but responses go directly to the client without passing through the LB.

TUN (Tunneling) sends requests from LB -> Backend via IPIP tunneling. This also means responses go directly to the client, similar to DR in structure, but it can be used with servers located far away.

In summary, iptables has a strong dependency on conntrack, and all packets are forced through it. However, ipvs does not require everything to pass through, especially with DR and TUN, which also avoid NAT. Furthermore, its hash-based lookup significantly improves speed, making it superior in many aspects.