AWS Network Troubleshooting – Step-by-Step Approach

So you have created your first VPC – congratulations! However, nothing works when you start an EC2 instance and try to connect. What can be the reason? There is no need to worry. If you’re having network problems with your Amazon Web Services (AWS) resources, this guide will help you troubleshoot and resolve them. We’ll cover the most common scenarios of AWS networking troubleshooting.

To troubleshoot the network in AWS you need to check the following components: Public or Elastic IPs, Security Groups, Network ACLs, Route Tables, Internet Gateways, and NAT Gateways. Also, if you are using SSH – make sure the correct key is being used.

In case you’re just starting your journey into the world of Cloud and DevOps, the following article that covers the networking basics required for a DevOps engineer will be helpful for you. ???? https://yourdevopsmentor.com/blog/networking-for-devops-a-complete-guide/

Check That Public IP Is Assigned.

It may seem obvious; however, this is still important to mention. Your public EC2 instances must be associated with a public or elastic IP address. Instances without a public IP are not routable to the internet for both ingress and egress connections. And, of course, without a public IP address, it’s just not possible to connect to the server. The screenshot below shows an instance without a public IP.

AWS Network Troubleshooting - Check that public IP is assigned

To allocate an Elastic IP and associate it with the instance, do the following:

  • Open AWS Management Console
  • Go to EC2
  • Under the “Network & Security” tab, choose Elastic IPs
  • Press “Allocate New Elastic IP address”
  • Select the allocated Elastic IP and press “Actions”
  • From the drop-down list, choose the “Associate Elastic IP Option”
  • Select your instance and press “Associate”

The screenshot below demonstrates an Instance that has an Elastic IP assigned.

AWS Network Troubleshooting - Check that public IP is assigned

What Error Do You Get?

Your network is fine if an error is “permission denied”, as shown in the picture below. Your ssh request is being received by the server. The server asks you to provide credentials and denies your authentication. It is important to verify that your credentials are correct – check your username and your SSH key.

AWS Network Troubleshooting - permission denied

Check Username

Different AMIs have various user names. Be sure to use the correct one. Here are the usernames for Amazon’s most popular AMIs:

AMI nameUsername
Amazon Linux ec2-user
CentOScentos or ec2-user
Debianadmin
Fedorafedora or ec2-user
Red hatec2-user or root
Suseec2-user or root
Ubuntuubuntu
Oracle Linuxec2-user
Bitnamibitnami

Check Your SSH Key Pair

A key pair is used to connect to an AWS EC2 instance. It consists of a public key that is stored on the instance and a private key that you download locally.

Make sure you have specified the right key when launching an instance.

  • Open AWS Management Console
  • Go to EC2
  • Select your instance
  • Go to the “Details” pane
  • Make sure that the correct key is chosen under the “Key pair name” property

The picture below demonstrates the instance which has the key specified correctly.

AWS Network Troubleshooting - SSH key pair

The picture below demonstrates the instance which does not have the key.

AWS Network Troubleshooting - SSH Key pair

You can also check which key is associated with your instance using AWS CLI; Just run the following command and check its output: 

aws ec2 describe-instances –instance-ids $YOUR_INSTANCE_ID

The picture below demonstrates the CLI representation of an instance which has the key specified correctly.

AWS Network Troubleshooting - EC2 instance

After you’ve verified that your key is assigned to your instance, you can verify that it is being used when you establish an SSH connection from the local machine.

Make sure that you are using the -i flag to pass the filename for the key when starting your SSH session.

AWS Network Troubleshooting - SSH connection

In case you are using Windows, follow the steps below:

  • In putty, choose Connection -> SSH -> Auth
  • Under the Authentication parameters / Private key file for authentication section, press Browse
  • Choose your key file and press open
Putty configuration

Timeout errors mean something is wrong with the network – scroll down below and see how to troubleshoot them. ????????????

AWS Network Troubleshooting - Operation timed out

Security Groups Should Allow Inbound Traffic.

To ssh into an instance, the security group should allow inbound traffic on port 22 from the desired source IPs. These can be your public IP, the IP range of your VPC or 0.0.0.0/0

AWS Network Troubleshooting - Security Groups

To check your security groups, do the following.

  • Open AWS Management Console
  • Go to EC2
  • Select your instance
  • Go to the “Security” pane.
  • Verify security groups assigned to the instance; they should contain any of these:
    • A rule allowing SSH from the 0.0.0.0/0 range – the easiest yet extremely dangerous option. By specifying 0.0.0.0/0 as your source IP range, you are allowing anybody on the internet to connect to your instance. It opens unlimited options for hackers to exploit your system – avoid this option as much as you can and use it only as a last resort.
    • Or a rule allowing SSH from the public IP of your ISP in case you are trying to connect to the public instance. You can find out the public IP of your ISP using the service like https://whatismyipaddress.com/ 
    • Or a CIDR range of your VPC if you are trying to connect to a private instance within your virtual network.
What is my IP

A VPC Should Have an Internet Gateway Attached to It.

An internet gateway is a horizontally scaled, redundant, and highly available VPC component that allows communication between instances in your VPC and the internet. To be routable to the internet, each VPC should have an internet gateway attached to it.

To check the internet gateway attachment, do the following:

  • Open AWS Management Console
  • Go to VPC
  • In the filter by VPC drop-down, choose the ID of your VPC
  • Go to “Internet Gateways”
  • There should be an internet gateway associated with your VPC

The picture below demonstrates a VPC with a properly configured Internet Gateway.

AWS Network Troubleshooting - Internet Gateway

A Route to Internet Gateway Should Exist

A route table is a collection of rules that defines how packets are routed among subnets in an Amazon VPC. You should have a route table that sends all traffic destined for the internet to an Internet gateway.

  • Open AWS Management Console
  • Go to VPC
  • In the filter by VPC drop-down, choose the ID of your VPC
  • Go to “Route Tables”
  • Choose your public route table
  • Make sure there is a route that forwards requests to all addresses 0.0.0.0/0 to your internet gateway.
AWS Network Troubleshooting - Route Tables

Public Route Tables Should Be Attached to Public Subnets

Make sure your public subnets have public route tables attached to them. To check this, do the following:

  • Open AWS Management Console
  • Go to VPC
  • In the filter by VPC drop-down, choose the ID of your VPC
  • Go to “Route Tables”
  • Choose your public route table
  • On the “Subnet associations” tab, make sure that all subnets that are supposed to be public listed.
Route Tables

NAT Gateway Should Reside in Public Subnets

The NAT gateway should be located in a public subnet that has a route table that routes internet traffic towards an internet gateway.

To check this, do the following.

  • Open AWS Management Console
  • Go to VPC
  • In the filter by VPC drop-down, choose the ID of your VPC
  • Under the “Virtual Private Cloud” section, choose “NAT Gateways”
  • Select your NAT Gateway
  • Under the details section, make sure that the public subnet is specified
NAT Gateways

Route to NAT Gateway Should Exist

When troubleshooting your AWS network, make sure that your private route tables are routing internet traffic to the NAT gateway. To check this, do the following:

  • Open AWS Management Console
  • Go to VPC
  • In the filter by VPC drop-down, choose the ID of your VPC
  • Go to “Route Tables”
  • Choose your private route table
  • Make sure there is a route that forwards requests to all addresses 0.0.0.0/0 to your NAT gateway.
AWS Network Troubleshooting - NAT Routes

A Private Route Table Should Be Attached to Subnets

Make sure your private subnets have private route tables attached to them. To check this, do the following:

  • Open AWS Management Console
  • Go to VPC
  • In the filter by VPC drop-down, choose the ID of your VPC
  • Go to “Route Tables”
  • Choose your private route table
  • On the “Subnet associations” tab, make sure that all subnets that are supposed to be private are listed.

Network ACLs Should Allow Traffic

If you are using network ACL – make sure it’s not blocking your desired traffic. It should fall under any of these criteria:

  • A rule allowing inbound traffic should exist.
  • A rule allowing outbound traffic should exist. Since network ACLs are not tracking connection state, an explicit outbound rule is required to send a response.
  • Allow rules should not be overridden by any deny rules that have higher priority.

If this article helped you in troubleshooting your AWS network – please support our efforts by sharing it on your social networks.????


Check our other guides:

You can find other interesting articles on our blog: https://yourdevopsmentor.com/blog/

Apply for individual mentorship here: https://yourdevopsmentor.com/apply/

Originally published Oct 17, 2022 2:21:20 PM (updated October 17 2022 )

Leave a Reply

Your email address will not be published. Required fields are marked *