Demonstration of using Prometheus & Grafana that are third party tools not in Cloud but are open source.

In this hands-on project, I ventured into demonstrating the simple and easy to use Prometheus & Grafana and how they can be used together for monitoring and visualization to understand the metrics (CPU, memory, disk etc) collected from systems and applications.

Below are few screenshots:

Creating an AWS EC2 instance and installing the docker, add ec2 user, docker compose, stress tool and initialize docker.

EC2 instance created

docker, docker-compose versions

Upload the container

Check resources created

Security group — release ports 3000, 9000 and 9090

EC2 public IP

Access to Portainer

A

Show containers

Dashboard shows the stack, containers, images, volumes and networks

Container list

Images

Network list

Event list

Host overview

Prometheus

Targets

Welcome to Grafana

Grafana Home page

Add new connection

Add Prometheus connection

Successfully queried the Prometheus API

Import Node Exporter dashboard

Create Grafana Stack

Grafana loading

Copy Node Exporter ID

Copy ID and import Prometheus into Grafana

Node Exporter dashboard

Check the current load

top -i to get cpu metrics

Increase disk consumption

CPU Busy — 100%

Added a 4 gb file to do stresss test and root partition is used

RAM usage starts to increase

RAM usage increased as expected

We can see by above demonstration how third-party open source tools like Prometheus and Grafana can be used for Monitoring, Logging and Obervability of Cloud environments same as AWS CloudWatch. Amazing!!!