How to Configure DNS Failover

Let's explore DNS Failover (DNS-based disaster recovery) as a last resort when even the load balancer in front of the servers fails, or an entire AWS region or IDC goes down.

DNS Failover (Difference from LB) #

Generally, L4/L7 load balancers like AWS ALB and Nginx distribute traffic among servers within a single network.

However, if the load balancer itself dies or the network connection is lost, there's no solution.

DNS Failover is a technology that redirects traffic at the global name resolution (domain to IP) stage.

Smart DNS services (AWS Route 53, Cloudflare) continuously monitor the health of backend servers. If they detect that main server A has failed, they respond to the client with the IP address of backup server B.

Health Check: The DNS server periodically sends pings or HTTP requests (e.g., every 10 seconds) to the server to confirm its liveness.
Routing Policy: Determines the ratio or priority for returning a different IP address when a server fails.
TTL: This is the duration for which clients or ISPs (like KT, SKT) will cache (remember) the IP address. It's the most crucial keyword for DNS Failover.

DNS Failover Routing Policies #

Active-Passive DR - Disaster Recovery Type #

This is the most standard Failover method.

Under normal circumstances, all 100% of traffic is directed to the primary main server.

The secondary backup server remains idle, only receiving data synchronization. If a failure occurs,

meaning the main server's health check fails, the DNS starts returning the IP address of the backup server.

Advantages: Infrastructure management is intuitive, and cache efficiency is good as traffic is concentrated in one place.

Active-Active - Weighted Distribution Type #

Used in global services or large-scale traffic environments, traffic is split by assigning weights (e.g., Seoul region A 50%, Tokyo region B 50%) during normal operation.

In case of a failure, if the Seoul region goes down, DNS removes A from the list and directs all 100% of traffic to the live Tokyo region.

Advantages: Expensive backup servers are utilized even during normal operation for traffic distribution, but management becomes more complex.

DNS Failover TTL Dilemma #

DNS Failover is not real-time, meaning it's not instantaneous.

Even if Route53 replaces the IP of a dead server with a backup server's IP in 1 second, the old, dead IP might still be cached on the user's PC, browser, or ISP DNS server for the duration of the TTL.

If the TTL is set too long, for example, 1 hour, users will send requests to the dead server and encounter errors for an hour during an outage. This causes a delay in disaster recovery. On the other hand, if the TTL is set too short, users will have to query the DNS server every minute, which slightly increases connection latency and incurs DNS query fees.

A compromise here is to set a short TTL, typically 60 or 30 seconds, for critical domains with Failover, so that traffic can switch within 1-2 minutes after an outage.

Example #

Let's look at how to configure and verify an Active-Passive environment using AWS Route53 with IaC.

AWS Route 53 Failover Configuration

Instead of clicking through the console, the following code creates health checks and failover records at once.

# 1. Primary server health check creation
resource "aws_route53_health_check" "primary_hc" {
  fqdn              = "api-primary.mywebsite.com"
  port              = 443
  type              = "HTTPS"
  resource_path     = "/health" # 200 OK at this path indicates it's alive
  failure_threshold = 3         # Considered dead after 3 consecutive failures
  request_interval  = 10        # Checks every 10 seconds
}

# 2. Main (Primary) DNS record (traffic usually goes here)
resource "aws_route53_record" "www_primary" {
  zone_id = aws_route53_zone.main.zone_id
  name    = "www.mywebsite.com"
  type    = "A"
  ttl     = 60 # Short TTL of 60 seconds for quick switchover during failure
  records = ["13.123.45.67"] # Main server IP

  failover_routing_policy {
    type = "PRIMARY"
  }
  set_identifier  = "primary-record"
  health_check_id = aws_route53_health_check.primary_hc.id
}

# 3. Backup (Secondary) DNS record (returns this IP if Primary dies)
resource "aws_route53_record" "www_secondary" {
  zone_id = aws_route53_zone.main.zone_id
  name    = "www.mywebsite.com"
  type    = "A"
  ttl     = 60
  records = ["52.98.76.54"] # DR (backup) server IP

  failover_routing_policy {
    type = "SECONDARY"
  }
  set_identifier = "secondary-record"
}

Terminal verification dig on outage

This is how engineers verify if DNS records are switching correctly from their PC.

# Normal DNS query, using +short for brevity
dig www.mywebsite.com +short

13.123.45.67 # Returns main server IP

# 2. On outage (after Primary server is down, approx. 1-2 minutes elapsed)
# Query Google Public DNS (8.8.8.8) directly, ignoring local DNS cache
dig @8.8.8.8 www.mywebsite.com +short
52.98.76.54 # (Primary Health Check failure detected, automatically switches to Secondary backup server IP)

If traffic has been redirected at the DNS level like this, the DR backup server must have the same data as the original primary server for normal service to be possible.