Uncategorized

Posted on April 15, 2026April 27, 2026 | by Rajesh Kumar

Lab Goal

Install and configure CloudWatch Agent manually inside an Ubuntu EC2 instance using SSH so that CloudWatch can collect:

Memory usage
Disk usage
CPU details
System logs
Application logs, if available

1. Prerequisites

Before logging into Ubuntu, make sure these are ready.

1.1 EC2 Instance

You need:

Ubuntu EC2 instance running
SSH access using key pair
Security group allowing SSH port 22 from your IP

1.2 IAM Role Attached to EC2

Your EC2 instance must have an IAM role with this AWS managed policy:

CloudWatchAgentServerPolicy

Recommended also:

AmazonSSMManagedInstanceCore

Even though this guide uses SSH, SSM permission is useful for future console-based management.

1.3 Check Region

Remember the AWS Region where your EC2 instance is running, for example:

ap-south-1
us-east-1
eu-west-1

You will check CloudWatch in the same Region.

2. SSH into Ubuntu EC2

From your local machine:

ssh -i your-key.pem ubuntu@your-ec2-public-ip

Example:

ssh -i my-lab-key.pem ubuntu@13.201.10.25

After login, update packages:

sudo apt update

3. Check System Architecture

Run:

uname -m

Expected output is usually:

x86_64

or:

aarch64

Use this to choose the right CloudWatch Agent package.

4. Download CloudWatch Agent Package

For Ubuntu x86_64 / amd64

wget https://amazoncloudwatch-agent.s3.amazonaws.com/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb

For Ubuntu ARM64 / aarch64

wget https://amazoncloudwatch-agent.s3.amazonaws.com/ubuntu/arm64/latest/amazon-cloudwatch-agent.deb

Most normal Ubuntu EC2 instances use amd64.

5. Install the CloudWatch Agent

Run:

sudo dpkg -i -E ./amazon-cloudwatch-agent.deb

Verify installation:

/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -h

You can also check package status:

dpkg -l | grep amazon-cloudwatch-agent

6. Create CloudWatch Agent Configuration File

Now create the agent configuration file.

Run:

sudo mkdir -p /opt/aws/amazon-cloudwatch-agent/etc

Open a file using nano:

sudo nano /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json

Paste this configuration:

{
  "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "root"
  },
  "metrics": {
    "namespace": "CWAgent",
    "append_dimensions": {
      "InstanceId": "${aws:InstanceId}",
      "InstanceType": "${aws:InstanceType}",
      "ImageId": "${aws:ImageId}"
    },
    "metrics_collected": {
      "cpu": {
        "measurement": [
          "cpu_usage_idle",
          "cpu_usage_user",
          "cpu_usage_system",
          "cpu_usage_iowait"
        ],
        "metrics_collection_interval": 60,
        "totalcpu": true
      },
      "mem": {
        "measurement": [
          "mem_used_percent",
          "mem_available",
          "mem_total"
        ],
        "metrics_collection_interval": 60
      },
      "disk": {
        "measurement": [
          "used_percent",
          "free",
          "total"
        ],
        "metrics_collection_interval": 60,
        "resources": [
          "/"
        ]
      },
      "diskio": {
        "measurement": [
          "reads",
          "writes",
          "read_bytes",
          "write_bytes"
        ],
        "metrics_collection_interval": 60,
        "resources": [
          "*"
        ]
      },
      "net": {
        "measurement": [
          "bytes_sent",
          "bytes_recv",
          "packets_sent",
          "packets_recv"
        ],
        "metrics_collection_interval": 60,
        "resources": [
          "*"
        ]
      },
      "swap": {
        "measurement": [
          "swap_used_percent"
        ],
        "metrics_collection_interval": 60
      }
    }
  },
  "logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/var/log/syslog",
            "log_group_name": "/aws/ec2/ubuntu/syslog",
            "log_stream_name": "{instance_id}"
          },
          {
            "file_path": "/var/log/auth.log",
            "log_group_name": "/aws/ec2/ubuntu/auth",
            "log_stream_name": "{instance_id}"
          },
          {
            "file_path": "/var/log/cloud-init.log",
            "log_group_name": "/aws/ec2/ubuntu/cloud-init",
            "log_stream_name": "{instance_id}"
          }
        ]
      }
    }
  }
}

Save the file:

CTRL + O
ENTER
CTRL + X

7. Start CloudWatch Agent Manually

Run this command:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
-a fetch-config \
-m ec2 \
-c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json \
-s

Meaning:

Option	Meaning
`-a fetch-config`	Load configuration
`-m ec2`	Running on EC2
`-c file:...`	Use local config file
`-s`	Start the agent

8. Check Agent Status

Run:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a status

Expected output should show:

{
  "status": "running"
}

Also check with systemd:

sudo systemctl status amazon-cloudwatch-agent

Enable it to start after reboot:

sudo systemctl enable amazon-cloudwatch-agent

9. Check Agent Logs on Ubuntu

If something goes wrong, check the CloudWatch Agent local log:

sudo tail -f /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log

Common problems appear here, such as:

IAM permission issue
Invalid JSON config
Wrong log file path
CloudWatch endpoint access problem
Region detection issue

10. Verify Metrics in AWS Console

Now go to AWS Console.

Open CloudWatch.
Go to Metrics.
Click All metrics.
Look for namespace:

CWAgent

Open it.
Look for your EC2 instance ID.
Select metrics such as:

mem_used_percent
disk_used_percent
cpu_usage_user
cpu_usage_system
swap_used_percent

Graph the metrics.

It may take a few minutes for metrics to appear.

11. Verify Logs in AWS Console

Open CloudWatch.
Go to Logs.
Open Log groups or Log Management.
Look for:

/aws/ec2/ubuntu/syslog
/aws/ec2/ubuntu/auth
/aws/ec2/ubuntu/cloud-init

Open a log group.
Open the log stream named with your EC2 instance ID.
Confirm log events are visible.

12. Run Basic Logs Insights Query

Go to:

CloudWatch → Logs → Logs Insights

Select:

/aws/ec2/ubuntu/syslog

Run:

fields @timestamp, @message
| sort @timestamp desc
| limit 20

Search for errors:

fields @timestamp, @message
| filter @message like /error|ERROR|failed|FAILED|Exception/
| sort @timestamp desc
| limit 20

13. Create a Memory Alarm

Now create an alarm from the new CloudWatch Agent metric.

Go to CloudWatch.
Click Alarms.
Click Create alarm.
Click Select metric.
Choose:

CWAgent

Select:

mem_used_percent

Choose your instance.
Click Select metric.
Configure:

Statistic: Average
Period: 5 minutes
Condition: Greater than 80

Name it:

Ubuntu-EC2-High-Memory-Usage

Create the alarm.

14. Create a Disk Usage Alarm

Repeat the same process with:

disk_used_percent

Suggested threshold:

Greater than 80

Alarm name:

Ubuntu-EC2-High-Disk-Usage

15. Create a Basic Dashboard

Go to CloudWatch.
Click Dashboards.
Click Create dashboard.
Name:

Ubuntu-EC2-CloudWatch-Agent-Dashboard

Add a Line widget.
Add these CWAgent metrics:
- mem_used_percent
- disk_used_percent
- cpu_usage_user
- cpu_usage_system
Add an alarm widget.
Select your memory and disk alarms.
Save dashboard.

16. Useful Agent Commands

Start Agent

sudo systemctl start amazon-cloudwatch-agent

Stop Agent

sudo systemctl stop amazon-cloudwatch-agent

Restart Agent

sudo systemctl restart amazon-cloudwatch-agent

Check Status

sudo systemctl status amazon-cloudwatch-agent

Reload Config and Start

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
-a fetch-config \
-m ec2 \
-c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json \
-s

View Agent Logs

sudo tail -f /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log

17. Common Issues and Fixes

Issue 1: Metrics Not Showing in CloudWatch

Check:

sudo systemctl status amazon-cloudwatch-agent

Check logs:

sudo tail -n 100 /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log

Common fixes:

Attach CloudWatchAgentServerPolicy.
Check AWS Region.
Wait 2–5 minutes.
Restart the agent.

Issue 2: Logs Not Showing

Check if file exists:

ls -l /var/log/syslog
ls -l /var/log/auth.log

If the file does not exist, remove it from the config or use a valid log file.

Then restart:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
-a fetch-config \
-m ec2 \
-c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json \
-s

Issue 3: JSON Config Error

Validate JSON:

python3 -m json.tool /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json

If valid, it will print formatted JSON.

If invalid, fix missing commas, brackets, or quotes.

Issue 4: Permission Error

Check IAM role from EC2 console.

The instance role must include:

CloudWatchAgentServerPolicy

18. Final Lab Flow for Students

1. SSH into Ubuntu EC2.
2. Check architecture.
3. Download CloudWatch Agent package.
4. Install the package.
5. Create JSON config file.
6. Start agent with config.
7. Check agent status.
8. Verify CWAgent metrics in CloudWatch.
9. Verify log groups in CloudWatch Logs.
10. Run Logs Insights query.
11. Create memory alarm.
12. Create disk alarm.
13. Create dashboard.

That is the full manual SSH-based installation and configuration process for AWS CloudWatch Agent on Ubuntu EC2.

Manual Installation: AWS CloudWatch Agent on Ubuntu EC2 Using SSH

Lab Goal

1. Prerequisites

1.1 EC2 Instance

1.2 IAM Role Attached to EC2

1.3 Check Region

2. SSH into Ubuntu EC2

3. Check System Architecture

4. Download CloudWatch Agent Package

For Ubuntu x86_64 / amd64

For Ubuntu ARM64 / aarch64

5. Install the CloudWatch Agent

6. Create CloudWatch Agent Configuration File

7. Start CloudWatch Agent Manually

8. Check Agent Status

9. Check Agent Logs on Ubuntu

10. Verify Metrics in AWS Console

11. Verify Logs in AWS Console

12. Run Basic Logs Insights Query

13. Create a Memory Alarm

14. Create a Disk Usage Alarm

15. Create a Basic Dashboard

16. Useful Agent Commands

Start Agent

Stop Agent

Restart Agent

Check Status

Reload Config and Start

View Agent Logs

17. Common Issues and Fixes

Issue 1: Metrics Not Showing in CloudWatch

Issue 2: Logs Not Showing

Issue 3: JSON Config Error

Issue 4: Permission Error

18. Final Lab Flow for Students