AWS Route 53 Resolver (Hybrid DNS)

Recap: From Private Links to Private Names

Foundations You'll Need Today

1. Why Hybrid DNS Is a Recurring Exam Problem

Hybrid DNS shows up on the SAP-C02 exam because it is the point where three otherwise independent design decisions collide: network topology, identity of names, and failure domain. A candidate who has memorized that Transit Gateway connects VPCs and that Direct Connect connects on-premises can still fail a question that asks why an application works from one subnet but not another, or why a migration cutover succeeded for IP-based connections and failed for hostname-based ones. The exam uses DNS as a discriminator precisely because it is invisible in a network diagram. Two architectures that look identical on a whiteboard behave completely differently depending on whether name resolution was designed or assumed.

The domain mapping is mostly Domain 1, the organizational complexity and hybrid connectivity domain, but the topic bleeds into Domain 2 as well. Any resilience question that involves a secondary region, a standby data center, or a failover runbook has a DNS dependency, because failover that changes an IP address without changing the name is not failover from the application's perspective. Route 53 Resolver also appears in migration scenarios, where the question is how to keep on-premises clients resolving the same names before, during, and after a cutover without editing application configuration. The recurring shape of the question is a constraint that rules out the obvious answer: you cannot use a public hosted zone because the names must not be publicly resolvable, you cannot edit every client's /etc/hosts, and you cannot put a NAT gateway in the path because the traffic must stay on private connectivity.

What makes the topic tractable is that the AWS-side mechanism is small. There are two endpoint types, one rule type, and a handful of quotas. The complexity is entirely in the direction of the query and in the routing that carries it. Once you can state, for any given query, which resolver asks which resolver and over which path, the exam questions become mechanical. The rest of this lesson is organized around building that directional intuition, because almost every wrong answer on a hybrid DNS question comes from reversing the direction of an endpoint.

2. How Resolver Actually Answers a Query

Every VPC has a Route 53 Resolver, and it is not something you create. It is reachable at the VPC CIDR base plus two — the classic 10.0.0.2 for a 10.0.0.0/16 VPC — and it is the resolver that every EC2 instance uses by default through the DHCP option set. This resolver answers queries for names in private hosted zones associated with the VPC, resolves public names by recursing against the internet, and, critically, is the component that Resolver endpoints extend. The endpoints do not replace the VPC resolver; they give it a way to talk to resolvers outside the VPC.

An inbound endpoint is a set of elastic network interfaces, one per subnet you specify, each with an IP address from that subnet. On-premises DNS servers are configured to forward queries for AWS-hosted names to those IP addresses. The query arrives at the ENI, is handed to the VPC resolver, and is answered from the private hosted zone associated with the VPC. The direction is on-premises to AWS. An outbound endpoint is also a set of ENIs, but it works the other way: the VPC resolver, when it encounters a name that matches a Resolver rule, forwards the query from those ENIs to the target IP addresses named in the rule. The direction is AWS to on-premises. Both endpoint types are regional constructs, both require at least two IP addresses for availability, and both are billed per endpoint-hour plus per query.

The rule type that matters for hybrid DNS is the forwarding rule, which pairs a domain name with a list of target IP addresses and optionally a port. Rules are associated with VPCs, and association is what makes the rule visible to that VPC's resolver. A rule for corp.internal with targets pointing at two on-premises DNS servers means that any query from an associated VPC for a name ending in corp.internal is forwarded out through the outbound endpoint rather than being recursed publicly. This is why the mechanism is described as conditional forwarding: the rule is a condition, and only matching queries take the outbound path. Everything else follows the default behavior.

The path the forwarded query takes is the part candidates most often get wrong. The outbound endpoint's ENIs live in your VPC subnets, so the query leaves the VPC as ordinary IP traffic from those ENI addresses. It reaches the on-premises resolver only if the route tables and the hybrid connectivity — the Transit VIF or the VPN attachment from Day 11 — actually carry traffic from those subnets to the on-premises DNS server addresses. A correctly configured Resolver rule with a broken route is indistinguishable from a rule that does not exist, and the failure looks like a timeout rather than a resolution error.

3. The Core Decision Boundary: Which Direction Is the Query Going?

Nearly every hybrid DNS scenario reduces to a single question: who is asking, and where does the answer live? If the asker is on-premises and the answer lives in a private hosted zone in AWS, you need an inbound endpoint. If the asker is in AWS and the answer lives on-premises, you need an outbound endpoint plus a forwarding rule. The two are not interchangeable, they are not alternatives to each other, and a design that needs both directions needs both endpoints. The exam exploits this by describing a scenario in terms of the application rather than the query direction, so the first move is always to restate the scenario as a query.

The second-order decision is whether the on-premises side is even capable of forwarding. An inbound endpoint only works if the on-premises DNS servers can be configured with a conditional forwarder pointing at the endpoint IPs. In an environment where the on-premises DNS is a managed appliance the team does not control, that may be impossible, and the design has to change — for example, by resolving the AWS names through a different mechanism entirely. The exam rarely states this constraint explicitly, but it appears as a phrase like "the on-premises DNS team will not modify their configuration," which is a signal that the inbound endpoint answer is wrong.

The third decision is scope. Resolver rules are associated with VPCs, and a rule associated with one VPC does not apply to another. In a multi-account landing zone, the standard pattern is to create the outbound endpoint and the rules in a central networking account, then share the rules with workload accounts using AWS RAM — the same sharing mechanism from Day 6. This is why hybrid DNS questions often have a multi-account flavor: the endpoint is expensive and should exist once, but the rules need to be visible everywhere.

4. Endpoint Configuration and What Each Choice Costs

The first configuration decision is how many IP addresses to give an endpoint and in which subnets. AWS requires at least two IP addresses per endpoint for availability, and the guidance is to place them in different Availability Zones. This is not a formality: an endpoint with both addresses in one AZ is a single-AZ dependency for every DNS query in the environment, and DNS failure is unusually total because it breaks new connections rather than degrading existing ones. The cost of the second AZ is one more ENI and one more IP address, which is trivial next to the blast radius of a zonal DNS outage.

The second decision is the protocol and port. Resolver endpoints support DNS over UDP and TCP on port 53, and they also support DNS over HTTPS on port 443 for outbound rules. The DoH option matters in environments where the on-premises resolver is exposed through a service that only accepts HTTPS, or where the security team requires encrypted DNS on the wire. Choosing DoH changes the target port in the rule and the protocol the on-premises side must accept; it does not change the direction or the rule semantics.

The third decision is rule scope and priority. A forwarding rule can be associated with many VPCs, and rules are evaluated by most-specific match, so a rule for corp.internal and a rule for hr.corp.internal can coexist with the more specific one winning. This is how you split a single on-premises namespace across two different DNS servers — for example, sending HR queries to a dedicated resolver while everything else goes to the general one. The tradeoff is operational: every additional rule is another object to keep in sync with the on-premises DNS team's zone delegation, and a stale rule produces timeouts rather than clean failures.

The fourth decision is whether to use a single outbound endpoint for everything or to segment by environment. A single endpoint in a shared networking VPC is cheaper and simpler, but it means production and development DNS traffic share a path and a failure domain. Segmenting gives you isolation at the cost of more endpoints and more rules. The exam generally rewards the shared-endpoint answer unless the scenario explicitly calls for isolation, because the question is usually testing whether you know that endpoints are regional and shareable rather than whether you can build the most elaborate design.

5. Sizing, Limits, and the Numbers Worth Knowing

Resolver endpoints are sized by IP address count, not by a throughput tier you select. Each IP address on an endpoint can handle a certain number of queries per second, and the practical guidance is to add addresses when you approach that ceiling rather than to over-provision up front. The relevant published figures are that each Resolver endpoint IP address supports up to 10,000 queries per second, and that an endpoint can have up to six IP addresses. That gives a single endpoint a theoretical ceiling of 60,000 queries per second, which is far beyond most enterprise DNS loads but is worth knowing because it tells you the scaling lever is address count.

The quota that catches people is the number of Resolver rules per Region, which is 1,000 by default, and the number of VPC associations per rule, which is also 1,000. Neither is likely to bind in a normal environment, but the association quota matters in a large landing zone where a single shared rule is associated with every VPC in the organization. The other quota worth remembering is that you can have up to 600 inbound and outbound endpoints per Region combined, which is effectively unlimited for design purposes but confirms that endpoints are a per-Region resource.

On the pricing side, the model is endpoint-hour plus query volume. Each endpoint IP address is billed per hour whether or not it serves traffic, and each query through an endpoint is billed per million queries. This is the reason the shared-endpoint pattern is the standard recommendation: an endpoint per VPC multiplies the hourly charge by the number of VPCs for no functional benefit, since a single endpoint in a shared VPC can serve rules associated with many VPCs. The exam does not usually ask for exact prices, but it does ask for the design that avoids unnecessary per-VPC endpoints, and the cost model is the justification.

6. Failure Modes and What They Look Like in Production

The most common failure is a forwarding rule whose target IP addresses are unreachable from the outbound endpoint's subnets. The symptom is a query timeout rather than an NXDOMAIN, and the diagnostic move is to check the route table for the subnet containing the outbound endpoint ENIs and confirm there is a route to the on-premises DNS server addresses over the Transit Gateway or VPN. This failure is easy to misdiagnose as an on-premises DNS problem because the query does leave AWS; it simply never arrives. A useful first check is to look at the Resolver query logs, which record the query, the rule that matched, and the response code, making the difference between "no rule matched" and "rule matched but target timed out" immediately visible.

The second failure is a rule that matches more than intended. A rule for internal will capture anything.internal, including names that should have resolved publicly or from a private hosted zone. The symptom is that a name which used to resolve now times out, and the cause is usually a rule added for one purpose that overlaps an existing namespace. Because rules are evaluated by most-specific match, the fix is either to narrow the rule or to add a more specific rule that restores the intended behavior. This is the DNS equivalent of a route table with an overly broad prefix, and it fails the same way: silently, for a subset of names.

The third failure is an endpoint with insufficient IP addresses for the query volume, which manifests as intermittent timeouts under load rather than a consistent failure. Because DNS is usually the first step of every new connection, the application-level symptom is a spike in connection errors that correlates with traffic, not with any single service. The diagnostic move is to look at Resolver endpoint metrics and query logs for throttling, and the fix is to add IP addresses to the endpoint. This failure is worth internalizing because it is the one that looks like an application bug and is actually a capacity problem.

The fourth failure is asymmetric resolution, where AWS can resolve on-premises names but on-premises cannot resolve AWS names, or vice versa. This is almost always a missing endpoint in one direction rather than a misconfiguration of an existing one, and it is the failure mode the exam is most likely to describe, because it tests whether you understand that inbound and outbound endpoints are separate components with separate purposes.

7. The Operational and SRE Angle

DNS is a dependency of nearly every other service, which makes it a poor candidate for best-effort monitoring. The metrics that matter are the Resolver endpoint metrics: query volume, the count of queries that resulted in a timeout or a SERVFAIL, and the health of the endpoint ENIs themselves. A useful alarm is one on the rate of SERVFAIL responses from an outbound endpoint, because that metric rises before applications start failing and gives you a window to act. A second alarm on the endpoint's query volume relative to its IP address count gives you early warning on the capacity failure described above.

Resolver query logging is the operational tool that makes the rest tractable. It can be sent to CloudWatch Logs, S3, or Kinesis Data Firehose, and it records the query name, the query type, the VPC, the rule that matched, and the response code. In an incident, this is the difference between guessing and knowing: you can see whether the query arrived, which rule handled it, and what the target returned. The cost consideration is volume, since every query is logged, so the usual pattern is to log to S3 with a lifecycle policy rather than to CloudWatch Logs for high-volume environments.

From an SLO perspective, hybrid DNS is a shared dependency that should have its own availability target, because it is on the critical path for every service that resolves a name across the boundary. The runbook shape follows from the failure modes: first check whether the query is arriving at the endpoint at all, then whether a rule matched, then whether the target was reachable, then whether the endpoint had capacity. That ordering moves from the most common cause to the least, and it maps cleanly onto the query logs, which is what makes it a runbook rather than a checklist.

One operational detail worth building in early is a synthetic check that resolves a known on-premises name from a known AWS subnet on a schedule. This is the DNS equivalent of the canary from Day 30: it detects a broken path before an application does, and it distinguishes a DNS failure from an application failure during an incident. Without it, the first signal of a DNS problem is usually a cascade of unrelated application alarms.

8. Edge Cases and Exam Gotchas

The single most common mistake is reversing the endpoint direction. The mnemonic that works is to name the endpoint after where the query is going, not where it is coming from: an inbound endpoint is inbound to AWS, and an outbound endpoint is outbound from AWS. If the scenario says on-premises clients need to resolve AWS names, the answer contains "inbound." If it says AWS resources need to resolve on-premises names, the answer contains "outbound." Candidates who reason from the perspective of the client rather than the query get this backwards under time pressure.

The second gotcha is forgetting that the endpoint is regional. A Resolver endpoint in us-east-1 does nothing for a VPC in us-west-2, and a multi-region design needs an endpoint in each Region that has VPCs needing hybrid resolution. This is the same regional-versus-global distinction that appears with Transit Gateway and Direct Connect, and it is a reliable source of wrong answers.

The third gotcha is assuming that a private hosted zone is automatically resolvable from on-premises once an inbound endpoint exists. The zone must be associated with the VPC that the inbound endpoint serves, and the on-premises resolver must be configured to forward to the endpoint IPs. Both halves are required, and the exam will describe a scenario where one half is missing.

The fourth gotcha is the shared-services account pattern. In a multi-account environment, the outbound endpoint and the rules belong in the central networking account, and the rules are shared with workload accounts via AWS RAM. A design that creates an endpoint in every account is functionally correct but is the wrong answer on cost and operational grounds, and the exam will usually include it as a distractor.

The fifth gotcha is the interaction with private hosted zone resolution order. A query for a name that exists both in a private hosted zone and publicly resolves to the private answer from within an associated VPC, which is usually the intent but occasionally is not. If the scenario requires the public answer from inside the VPC, the zone association is the thing to change, not the Resolver rule.

9. Resolver vs. the Alternatives It Gets Confused With

Route 53 Resolver is not the only way to make names resolve across a boundary, and the exam tests whether you can tell the alternatives apart. The most common confusion is with Route 53 private hosted zones themselves. A private hosted zone is the container for the records; Resolver is the mechanism that lets a resolver outside the VPC query those records. Creating a private hosted zone does not make it resolvable from on-premises, and creating an inbound endpoint does not create any records. They are complementary, and a scenario that mentions both is usually testing whether you know which one is missing.

The second confusion is with VPC peering and Transit Gateway. Those provide IP reachability, which is necessary but not sufficient for name resolution. A design that connects two VPCs with peering and expects names to resolve across them will fail unless the private hosted zone is associated with both VPCs or a Resolver rule bridges them. The exam uses this by describing a scenario where connectivity is already established and the problem is still that names do not resolve.

The third confusion is with the public DNS of the on-premises environment. If the on-premises names are publicly resolvable, no Resolver configuration is needed at all, and the correct answer is to do nothing. The exam will sometimes include a scenario where the names are public and the elaborate hybrid DNS answer is a distractor. The signal is a phrase like "the records are already in a public zone."

The practical rule is to ask what is missing: if packets cannot flow, the answer is a connectivity service; if packets flow but names do not resolve, the answer is a Resolver endpoint in the correct direction; if the names should not be private at all, the answer is a public zone. Most hybrid DNS exam questions are one of those three, and identifying which one is the whole task.

Hands-on Lab: Conditional Forwarding to an On-Premises Resolver (45 min)

Scenario Question Drills (20 min)

Peek into Tomorrow

Sources