Oracle Cloud VM not reachable on port 80 despite public subnet, IGW, routing, and service listening

by

Summary

Key Issue: An OCI Free Tier Ubuntu 22.04 VM runs Nginx listening on port 80, yet HTTP traffic fails to connect. The environment includes a public VCN, an Internet Gateway (IGW), correct routing, and a public IPv4, but tcpdump shows zero inbound traffic on port 80 when curling the public IP from the internet. The root cause is an OCI Always Free instance shape limit on “Gigabit” bandwidth, coupled with the default inbound Instance Network Security Group (NSG) rule blocking port 80. This combination results in traffic being dropped at the Oracle Cloud firewall (SmartNIC) level before reaching the instance OS, or the instance lacking the necessary “Gigabit” bandwidth ticket to support the traffic class.

Root Cause

The issue stems from two distinct Oracle Cloud Infrastructure (OCI) specific conditions:

  1. Ingress Security Rules (Network Security Groups or NSG): By default, OCI instances provisioned via the Console (especially “Always Free” shapes) often belong to a default Instance NSG. This group typically allows SSH (22) and ICMP but explicitly denies all other ingress traffic, including TCP/80. Since the user confirmed an ICMP rule was added to allow ping, this highlights that TCP/80 lacks the necessary Ingress rule.
  2. Free Tier Bandwidth Limit (SmartNIC Offload): OCI Always Free instances (e.g., VM.Standard.E4.Flex) are restricted to 1 Gbps network bandwidth. However, a specific internal OCI ticket or “shape configuration” is required to enable “Gigabit” (1 Gbps) performance. Without this, the instance may default to a lower bandwidth class or suffer packet drops on specific flows. The tcpdump showing no inbound traffic while ICMP works suggests the SmartNIC is dropping the TCP flow due to lack of bandwidth provisioning or state table overflow.

Why This Happens in Real Systems

Oracle Cloud Infrastructure (OCI) differs significantly from AWS or Azure in its networking stack implementation.

  • SmartNIC Offloading: OCI uses an intelligent network interface (SmartNIC) to offload security rules and routing. These rules are enforced before the packet reaches the VM’s hypervisor or OS. If a packet is dropped by the SmartNIC, tcpdump on the VM interface will never see it.
  • Default Security Posture: OCI defaults to a “deny-all” ingress stance for compute instances to prevent open vulnerabilities. Users often assume “Public Subnet” implies public access, but in OCI, “Public” only refers to the route table (0.0.0.0/0 → IGW) and the presence of a Public IP. Security Group rules are a separate layer.
  • Always Free Limitations: Free Tier resources have strict quotas. The “Gigabit” bandwidth is not automatically provisioned for all instances; it requires the instance to be correctly allocated in a specific availability domain or have a support ticket applied to the tenancy to unlock 1 Gbps throughput.

Real-World Impact

  • Service Unavailability: The application (Nginx, potentially serving n8n as hinted by tags) is completely inaccessible from the public internet, rendering the deployment useless for external access.
  • Debugging Difficulty: The tcpdump output is misleading. Seeing no packets suggests a routing or firewall issue outside OCI (e.g., ISP block), while the actual issue is internal to OCI’s hypervisor/SmartNIC.
  • Resource Inefficiency: The user wastes time debugging UFW, iptables, and Nginx configurations (which are correct) rather than looking at the cloud provider’s network layer.
  • Architectural Confusion: It reinforces bad habits if the user bypasses the issue by opening 0.0.0.0/0 to all ports, which compromises security posture.

Example or Code

The following code verifies the listening state and attempts to curl the public IP. The curl command is expected to fail with “No route to host” or timeout, confirming the network block.

# 1. Verify Nginx is listening on all interfaces (User's confirmation)
ss -tulnp | grep :80

# 2. Test internal connectivity (Localhost works)
curl -v http://127.0.0.1:80

# 3. Test external connectivity (Fails)
# This command attempts to connect to the public IP, resulting in failure
curl -v --connect-timeout 5 http://

# 4. Packet capture (User's observation)
# Running this while attempting to curl the public IP shows NO inbound traffic on port 80
sudo tcpdump -i any port 80 -n

How Senior Engineers Fix It

A senior engineer addresses this by validating the OCI network layers sequentially, focusing on the SmartNIC/Security Group interaction.

  1. Verify Ingress Rules:

    • Navigate to the Oracle Cloud Console: Networking > Virtual Cloud Networks > Security Lists or Network Security Groups.
    • Identify the NSG attached to the VM’s VNIC.
    • Add Ingress Rule:
      • Source Type: CIDR
      • Source CIDR: 0.0.0.0/0 (or specific IP range for security).
      • IP Protocol: TCP
      • Source Port Range: All
      • Destination Port Range: 80 (or 443 for HTTPS).
    • Why: This opens the SmartNIC firewall to allow traffic to pass to the instance.

  2. Verify Instance Shape/Bandwidth:

    • Navigate to Compute > Instances > Select Instance.
    • Check Instance Details > Shape.
    • Ensure the shape supports the required bandwidth. For “Always Free” VM.Standard.E4.Flex, 1 Gbps is standard but requires the correct configuration.
    • If bandwidth is suspected to be the issue (e.g., heavy traffic or specific latency issues), submit a support ticket to OCI asking to verify the “Gigabit Network” feature is enabled for the tenancy/instance.

  3. Validate Security List (If not using NSG):

    • If the instance uses a Security List instead of an NSG, ensure the Ingress Rules allow TCP/80 from 0.0.0.0/0.

  4. Check Cloud Init/Route Tables:

    • Ensure the public subnet route table has 0.0.0.0/0 target Internet Gateway.
    • Verify no NAT Gateway or NAT rules are interfering (rare in simple setups).

Why Juniors Miss It

Junior engineers often lack familiarity with the separation of routing (Subnet/IGW) and security (Security Groups/NSGs) in cloud environments, especially Oracle’s implementation.

  • Assumption of “Public Subnet”: Juniors assume that if a VM is in a “Public Subnet” with a Public IP, it is automatically reachable. They miss that OCI requires an explicit Allow rule in the Security Group/NSG.
  • Over-reliance on OS-level tools: tcpdump and ss show the VM’s perspective. If the OCI SmartNIC drops the packet before it hits the VM, these tools will be silent. Juniors often spend hours debugging ufw or iptables on the OS, which are irrelevant if the cloud firewall blocks the packet first.
  • Confusion with “Any” vs. “All”: When configuring rules, they might leave the destination port blank or set it incorrectly, or fail to specify the protocol (TCP) correctly.
  • Free Tier Constraints: Juniors are often unaware of the specific bandwidth or feature limitations of “Always Free” tiers, assuming they behave identically to paid enterprise instances.