Docs / Use Cases / Public + Private VPC with nginx

Public + Private VPC with nginx, EIP, SSM, NACL

Build the canonical AWS topology in LocalEmu: a VPC with one public subnet (internet-facing) and one private subnet (NAT-only). Launch an EC2 in each, install nginx, hit it from your Mac through an EIP, SSM into the private box, then prove a custom NACL kills its egress.

What the demo does. Creates a 10.100.0.0/24 VPC split into 10.100.0.0/25 (public) and 10.100.0.128/25 (private). Attaches an Internet Gateway, puts a NAT Gateway in the public subnet, wires the two route tables. Creates a Security Group admitting all ports from the loopback CIDR your Mac uses. Launches a t3.small alpine instance in each subnet, associates an EIP to the public one, installs nginx via SSM and configures a vhost that actually serves content, then curls it from the Mac via the auto-published EIP host port. Finally, attaches a custom Network ACL to the private subnet that denies all egress and shows apk add timing out. Requires a running Docker daemon (the LocalEmu EC2 backend defaults to Docker, no env var needed).

🌐

Public + private subnets

IGW for the public subnet, NAT GW for the private one. Two real route tables, two real Docker networks under the hood.

🔗

EIP that actually routes

LocalEmu publishes one loopback host port per port the container listens on (allowed by SG). The mapping shows up as a DescribeAddresses Tag.

💻

Real SSM into private

SSM start-session works without a public IP. The session enters the container directly via docker exec, no SSH key, no jump host.

🛡

Real iptables SG enforcement

Security Group rules are compiled to iptables chains inside each container. Revoke a rule and connections stop within seconds.

🚫

NACL egress block

A custom NACL with a DENY-all egress entry on the private subnet drops every outbound packet. Watch apk add time out.

Step-by-Step Walkthrough

Step 1: Start LocalEmu

$ localemu start

The EC2 backend defaults to Docker, so RunInstances creates real containers out of the box. To opt out (e.g. CI without Docker), set EC2_VM_MANAGER=none and the EC2 API will hold metadata only.

Step 2: Create the VPC and split it into a public + private subnet

$ VPC=$(awsemu ec2 create-vpc \
    --cidr-block 10.100.0.0/24 \
    --query Vpc.VpcId --output text)

$ PUB_SUBNET=$(awsemu ec2 create-subnet \
    --vpc-id $VPC \
    --cidr-block 10.100.0.0/25 \
    --availability-zone us-east-1a \
    --query Subnet.SubnetId --output text)

$ PRIV_SUBNET=$(awsemu ec2 create-subnet \
    --vpc-id $VPC \
    --cidr-block 10.100.0.128/25 \
    --availability-zone us-east-1a \
    --query Subnet.SubnetId --output text)

$ echo "VPC=$VPC PUB=$PUB_SUBNET PRIV=$PRIV_SUBNET"
VPC=vpc-c7e2f944e406ffd34 PUB=subnet-ec781f543ca3a3b08 PRIV=subnet-0840bd9ecd142c3d6

One /24 VPC, split into two /25 subnets. The public one will get an IGW route, the private one a NAT route.

Step 3: Internet Gateway + NAT Gateway

$ IGW=$(awsemu ec2 create-internet-gateway \
    --query InternetGateway.InternetGatewayId --output text)

$ awsemu ec2 attach-internet-gateway \
    --vpc-id $VPC --internet-gateway-id $IGW

$ NAT_EIP=$(awsemu ec2 allocate-address --domain vpc \
    --query AllocationId --output text)

$ NAT=$(awsemu ec2 create-nat-gateway \
    --subnet-id $PUB_SUBNET --allocation-id $NAT_EIP \
    --query NatGateway.NatGatewayId --output text)

IGW=igw-b2c1c2bba55e249b5
NAT=nat-2a9068b98202bd524

The IGW gives the public subnet bidirectional internet. The NAT Gateway lives in the public subnet, eats its own EIP, and lets the private subnet reach out (but never get reached).

Step 4: Route tables wire the two subnets

$ # Public RT: 0.0.0.0/0 -> IGW, associated to public subnet
$ PUB_RT=$(awsemu ec2 create-route-table --vpc-id $VPC \
    --query RouteTable.RouteTableId --output text)
$ awsemu ec2 create-route --route-table-id $PUB_RT \
    --destination-cidr-block 0.0.0.0/0 --gateway-id $IGW
$ awsemu ec2 associate-route-table \
    --route-table-id $PUB_RT --subnet-id $PUB_SUBNET

$ # Private RT: 0.0.0.0/0 -> NAT, associated to private subnet
$ PRIV_RT=$(awsemu ec2 create-route-table --vpc-id $VPC \
    --query RouteTable.RouteTableId --output text)
$ awsemu ec2 create-route --route-table-id $PRIV_RT \
    --destination-cidr-block 0.0.0.0/0 --nat-gateway-id $NAT
$ awsemu ec2 associate-route-table \
    --route-table-id $PRIV_RT --subnet-id $PRIV_SUBNET

One route table per subnet. Public: 0.0.0.0/0 -> IGW. Private: 0.0.0.0/0 -> NAT.

Step 5: Security Group admitting your Mac

$ SG=$(awsemu ec2 create-security-group \
    --group-name le-vpc-demo \
    --description "Allow Mac to reach EC2 on any port" \
    --vpc-id $VPC \
    --query GroupId --output text)

$ # 127.0.0.0/8 is the source LocalEmu sees from your Mac:
$ # the EIP host proxy binds 127.0.0.1, so the accepted
$ # socket's peer IP is loopback. Real AWS would use your
$ # public IPv4 instead.
$ awsemu ec2 authorize-security-group-ingress \
    --group-id $SG \
    --ip-permissions \
       'IpProtocol=tcp,FromPort=0,ToPort=65535,IpRanges=[{CidrIp=127.0.0.0/8}]' \
       'IpProtocol=udp,FromPort=0,ToPort=65535,IpRanges=[{CidrIp=127.0.0.0/8}]'

The SG opens TCP and UDP on every port to 127.0.0.0/8. From the container's point of view this is your Mac: the EIP host proxy in LocalEmu binds 127.0.0.1, so the accepted socket's peer address is loopback. The SG is evaluated against that peer IP on every connection.

Step 6: Public EC2 with an EIP

$ PUB_IID=$(awsemu ec2 run-instances \
    --image-id ami-alpine-3.20 \
    --instance-type t3.small \
    --subnet-id $PUB_SUBNET \
    --security-group-ids $SG \
    --associate-public-ip-address \
    --tag-specifications \
       'ResourceType=instance,Tags=[{Key=Name,Value=le-public}]' \
    --query Instances[0].InstanceId --output text)

$ awsemu ec2 wait instance-running --instance-ids $PUB_IID

$ PUB_EIP_ALLOC=$(awsemu ec2 allocate-address --domain vpc \
    --query AllocationId --output text)

$ PUB_EIP=$(awsemu ec2 describe-addresses \
    --allocation-ids $PUB_EIP_ALLOC \
    --query Addresses[0].PublicIp --output text)

$ awsemu ec2 associate-address \
    --allocation-id $PUB_EIP_ALLOC --instance-id $PUB_IID

$ echo "PUB_IID=$PUB_IID  PUB_EIP=$PUB_EIP"
PUB_IID=i-c8acb37841f1dd9e3  PUB_EIP=198.51.100.2

t3.small on ami-alpine-3.20. The EIP is what your Mac will eventually curl through, but only for ports the SG admits AND the container is actually listening on.

Step 7: Private EC2 (no EIP, no Mac reachability)

$ PRIV_IID=$(awsemu ec2 run-instances \
    --image-id ami-alpine-3.20 \
    --instance-type t3.small \
    --subnet-id $PRIV_SUBNET \
    --security-group-ids $SG \
    --no-associate-public-ip-address \
    --tag-specifications \
       'ResourceType=instance,Tags=[{Key=Name,Value=le-private}]' \
    --query Instances[0].InstanceId --output text)

$ awsemu ec2 wait instance-running --instance-ids $PRIV_IID

$ awsemu ec2 describe-instances --instance-ids $PRIV_IID \
    --query 'Reservations[0].Instances[0].{Priv:PrivateIpAddress,Pub:PublicIpAddress}'
{
    "Priv": "10.100.0.5",
    "Pub": "127.0.0.1"
}

The instance has no EIP and no host port published, so your Mac cannot reach it. LocalEmu reports PublicIpAddress: 127.0.0.1 as a placeholder (the LocalEmu host is always loopback in this emulator), not a routable address. From the AWS-API point of view the instance is private: the only way in is intra-VPC traffic or SSM (Step 11).

Step 8: SSM into the public box and install nginx

$ # SSM into the public box. LocalEmu wires this to
$ # docker exec into the container, no key pair needed.
$ awsemu ssm start-session --target $PUB_IID
/ # apk add --no-cache nginx
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/community/aarch64/APKINDEX.tar.gz
(1/2) Installing pcre (8.45-r3)
(2/2) Installing nginx (1.26.3-r0)
OK: 20 MiB in 24 packages

awsemu ssm start-session drops you straight into the container as root. The apk add works because the public subnet has IGW egress.

Step 9: Configure nginx to actually serve content

/ # # Alpine's nginx package ships a default vhost that returns
/ # # 404 for everything. Replace it with a real one.
/ # cat > /etc/nginx/http.d/default.conf <<'EOF'
server {
  listen 80 default_server;
  root /var/lib/nginx/html;
  index index.html;
}
EOF
/ # nginx
/ # wget -qO- http://127.0.0.1/
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
<h1>Welcome to nginx!</h1>
...
/ # exit

Alpine's nginx package ships a default vhost that literally returns 404 for every URL (the maintainers don't want the package to expose anything by accident). Replace /etc/nginx/http.d/default.conf with a vhost pointing at /var/lib/nginx/html (where the package's welcome page already lives), then nginx starts serving it.

Step 10: Curl nginx from your Mac through the EIP

$ # Back on your Mac. LocalEmu watches the container for newly
$ # listening TCP ports and auto-publishes one host port per port
$ # the SG admits. The mapping shows up as a Tag on the EIP:
$ awsemu ec2 describe-addresses --public-ips $PUB_EIP \
    --query 'Addresses[0].Tags'
[
    {
        "Key": "localemu:HostPort:80",
        "Value": "127.0.0.1:55930"
    }
]

$ curl http://127.0.0.1:55930/
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
<h1>Welcome to nginx!</h1>
...

LocalEmu polls the container every few seconds for newly listening TCP ports, intersects them with the ports the SG admits, and opens one host listener per match on 127.0.0.1. The mapping is exposed as a localemu:HostPort:<cport> Tag on DescribeAddresses, so any script can read it. Curl the host port and you reach nginx through the same path SG and flow-log enforcement use.

Step 11: SSM into the private box to prove NAT egress works

$ # Private box has no public IP, no EIP, no Mac access.
$ # SSM still works (LocalEmu uses docker exec, not the network).
$ awsemu ssm start-session --target $PRIV_IID
/ # # NAT GW gives the private subnet outbound internet:
/ # apk add --no-cache curl
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/main/aarch64/APKINDEX.tar.gz
OK: 13 MiB in 18 packages

The private subnet has no inbound path, but the NAT Gateway gives it outbound internet. apk add downloads packages from the alpine mirrors successfully.

Step 12: Custom NACL on the private subnet, DENY egress

$ # Custom NACL on the private subnet: ALLOW inbound, DENY egress.
$ NACL=$(awsemu ec2 create-network-acl --vpc-id $VPC \
    --query NetworkAcl.NetworkAclId --output text)

$ # --ingress is required so the CLI knows which direction
$ # this rule applies to (no default).
$ awsemu ec2 create-network-acl-entry --network-acl-id $NACL \
    --rule-number 100 --protocol -1 --ingress \
    --cidr-block 0.0.0.0/0 --rule-action allow

$ awsemu ec2 create-network-acl-entry --network-acl-id $NACL \
    --rule-number 100 --protocol -1 --egress \
    --cidr-block 0.0.0.0/0 --rule-action deny

$ ASSOC=$(awsemu ec2 describe-network-acls \
    --filters "Name=association.subnet-id,Values=$PRIV_SUBNET" \
    --query "NetworkAcls[0].Associations[?SubnetId=='$PRIV_SUBNET'].NetworkAclAssociationId | [0]" \
    --output text)

$ awsemu ec2 replace-network-acl-association \
    --association-id $ASSOC --network-acl-id $NACL
{
    "NewAssociationId": "aclassoc-1ae0a00ee80dd05b3"
}

Default NACLs allow everything. Here we attach a new NACL that allows inbound (so existing TCP responses for the SSM session still arrive) but denies all egress. Rule 100 wins because no lower-numbered rules exist.

Step 13: Watch the next apk add time out

$ awsemu ssm start-session --target $PRIV_IID
/ # apk add --no-cache htop
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/main/aarch64/APKINDEX.tar.gz
WARNING: fetching https://dl-cdn.alpinelinux.org/alpine/v3.20/main: temporary error (try again later)
fetch https://dl-cdn.alpinelinux.org/alpine/v3.20/community/aarch64/APKINDEX.tar.gz
WARNING: fetching https://dl-cdn.alpinelinux.org/alpine/v3.20/community: temporary error (try again later)
ERROR: unable to select packages:
  htop (no such package):
    required by: world[htop]
/ # exit
# Every outbound packet from the private subnet is dropped by
# the NACL egress rule, so the apk mirror is unreachable.

With the egress NACL in place, every outbound packet from the private subnet is dropped. DNS resolution fails first, apk reports Temporary failure in name resolution, and the install aborts. Re-associate the original NACL to restore connectivity.

Step 14: Teardown

$ # Teardown order matters. Route-table and NACL associations
$ # must be removed BEFORE the route tables and NACL themselves;
$ # otherwise delete-* fails with DependencyViolation.
$
$ # 1) Instances + EIP + NAT + IGW + SG: independent of subnets.
$ awsemu ec2 terminate-instances --instance-ids $PUB_IID $PRIV_IID
$ awsemu ec2 disassociate-address --public-ip $PUB_EIP
$ awsemu ec2 release-address --allocation-id $PUB_EIP_ALLOC
$ awsemu ec2 delete-nat-gateway --nat-gateway-id $NAT
$ awsemu ec2 release-address --allocation-id $NAT_EIP
$ awsemu ec2 detach-internet-gateway \
    --internet-gateway-id $IGW --vpc-id $VPC
$ awsemu ec2 delete-internet-gateway --internet-gateway-id $IGW
$ awsemu ec2 delete-security-group --group-id $SG
$
$ # 2) Disassociate every route-table -> subnet binding.
$ #    The Main route table is implicit; skip it (Main==true).
$ for RT in $PUB_RT $PRIV_RT; do
    for A in $(awsemu ec2 describe-route-tables \
         --route-table-ids $RT \
         --query 'RouteTables[0].Associations[?Main==`false`].RouteTableAssociationId' \
         --output text); do
      awsemu ec2 disassociate-route-table --association-id $A
    done
  done
$
$ # 3) Detach the custom NACL by replacing every association on it
$ #    back to the VPC's default NACL (the AWS idiomatic way).
$ DEFAULT_NACL=$(awsemu ec2 describe-network-acls \
    --filters "Name=vpc-id,Values=$VPC" "Name=default,Values=true" \
    --query 'NetworkAcls[0].NetworkAclId' --output text)
$ for A in $(awsemu ec2 describe-network-acls \
     --network-acl-ids $NACL \
     --query 'NetworkAcls[0].Associations[].NetworkAclAssociationId' \
     --output text); do
    awsemu ec2 replace-network-acl-association \
      --association-id $A --network-acl-id $DEFAULT_NACL
  done
$
$ # 4) Now safe to delete: subnets -> RTs -> NACL -> VPC.
$ awsemu ec2 delete-subnet --subnet-id $PUB_SUBNET
$ awsemu ec2 delete-subnet --subnet-id $PRIV_SUBNET
$ awsemu ec2 delete-route-table --route-table-id $PUB_RT
$ awsemu ec2 delete-route-table --route-table-id $PRIV_RT
$ awsemu ec2 delete-network-acl --network-acl-id $NACL
$ awsemu ec2 delete-vpc --vpc-id $VPC
$
$ # Shortcut: ``localemu stop`` wipes the in-memory state when
$ # PERSISTENCE=0 (the default), so you can skip the dance above
$ # entirely if you don't need a clean AWS-API teardown.

Three things have to happen before the final delete-* calls succeed: (a) route-table to subnet associations must be removed with disassociate-route-table, (b) the custom NACL must be detached by replacing every association on it with the VPC's default NACL, and (c) only then can subnets, route tables, the custom NACL, and the VPC itself be deleted in that order. If you don't need a clean AWS-API teardown, localemu stop wipes the in-memory state when PERSISTENCE=0 (the default).

How It Works

1. CreateVpc records the VPC in the EC2 metadata store and provisions a real Docker bridge network (initially --internal, so traffic stays inside the VPC).

2. AttachInternetGateway recreates the VPC's bridge without the --internal flag so packets to 0.0.0.0/0 can leave the Docker host.

3. CreateNatGateway spins up a tiny localemu-nat-* container with iptables MASQUERADE configured. Containers on the private subnet get a default route to it.

4. RunInstances starts one container per instance on the VPC bridge, installs iptables on first boot (Alpine ships without it), and compiles the Security Group rules into the container's SG_IN / SG_OUT chains.

5. AssociateAddress registers an EIP route in the host-side asyncio TCP proxy. A port watcher polls the container every few seconds; for each new listening port the SG admits, a loopback host listener is bound and the localemu:HostPort:<cport> Tag is published.

6. When you curl 127.0.0.1:<hp> from your Mac, the proxy accepts the socket, reads the real source IP (loopback), evaluates the SG, emits a flow-log entry, then tunnels the bytes via docker exec -i <ec2> socat - TCP:127.0.0.1:<cport>. No Docker port-publish, no source-IP rewrite.

7. SSM StartSession opens a PTY and docker execs into the container's shell directly, the same way real Systems Manager Session Manager bypasses the network path.

8. Network ACLs are translated to per-subnet iptables chains stamped on every container's SG_IN / SG_OUT jump. A DENY-egress entry drops the packet before it leaves the netns.

← Transit Gateway + Peering IAM Least-Privilege →