Trace a request through a VPC

Before adding AWS services, draw the endpoints. Write the source, destination, protocol, and port for every flow that must work. A web request might enter on HTTPS 443 and continue from a load balancer to an application on TCP 8080.

A VPC uses a CIDR block: a range of IP addresses reserved for the network. Divide it into non-overlapping subnet CIDRs with room for growth. Each subnet belongs to one Availability Zone.

A subnet is public only when its associated route table sends internet traffic to an attached internet gateway. For IPv4, the resource also needs the required public address mapping. Calling a subnet “public” changes nothing.

A private application subnet has no direct route to an internet gateway. Its targets can still receive requests from a public load balancer because the load balancer forwards traffic over VPC routing.

A route table answers “where should this packet go next?” It does not answer “is this caller allowed?” AWS selects the most specific matching route in the table associated with the subnet.

An attached internet gateway is useless when the selected route does not point to it. Editing a route table that is not associated with the subnet is also useless. Inspect the effective association and selected prefix.

A security group is a stateful allow-only control attached to a resource. The app security group can allow TCP 8080 from the load balancer security group instead of allowing every address in the VPC.

A network access control list, or NACL, is an ordered stateless control at the subnet boundary. It supports allow and deny rules. Because it is stateless, request and response need matching rules in their own directions.

Debug in order: route first, then permission. Opening a security group cannot repair a missing route, and adding a route cannot override a security-group or NACL denial.

Trace the forward path: client, internet gateway, public-subnet route and NACL, load-balancer security group, listener, target health, target security group, and application port.

Then trace the response. Security groups remember an allowed connection, so return traffic for that connection is allowed. NACLs remember nothing; their outbound rules must allow the response, including the required ephemeral ports.

If the request reaches the app but the response disappears at the subnet edge, inspect the outbound NACL. If the load balancer cannot reach a healthy target, compare its listener and target port, then confirm that the target security group trusts the load balancer security group.

Stop at the first failed gate and change only that gate. Changing several routes and controls together may restore traffic while hiding the cause and widening access.

Two Availability Zones help only when the complete path and enough capacity survive in either zone. Remove one zone, then replay both forward and return paths.