Platform Engineers

Monitoring and Logging for Containerized Applications: Maintaining Visibility in a Dynamic Environment

Platform Engineers's photo
Platform Engineers
·

4 min read

Containerization has become a popular approach for deploying applications due to its flexibility, portability, and scalability. However, monitoring and logging containerized applications can be challenging due to their dynamic nature. This article discusses the technical aspects of monitoring and logging for containerized applications using popular tools such as Prometheus, Grafana, and ELK Stack.

Monitoring and Logging Challenges for Containerized Applications

Containerized applications are ephemeral and can be scaled up or down quickly, making it difficult to monitor and log them using traditional methods. Additionally, containerized applications can be distributed across multiple hosts, making it challenging to aggregate and analyze monitoring and logging data.

Monitoring and Logging Tools for Containerized Applications

There are several tools available for monitoring and logging containerized applications. Here are some popular options:

  1. Prometheus: Prometheus is an open-source monitoring and alerting tool that can be used to monitor containerized applications. Prometheus provides a pull-based monitoring approach, where it scrapes metrics from containerized applications and stores them in a time-series database.

  2. Grafana: Grafana is an open-source visualization tool that can be used to visualize monitoring data from Prometheus. Grafana provides a user-friendly interface for creating dashboards and alerts.

  3. ELK Stack: ELK Stack is an open-source logging tool that can be used to collect, index, and analyze logs from containerized applications. ELK Stack consists of Elasticsearch for indexing and searching logs, Logstash for processing logs, and Kibana for visualizing logs.

Monitoring and Logging Architecture for Containerized Applications

A typical monitoring and logging architecture for containerized applications consists of the following components:

  1. Metrics exporter: A metrics exporter is a component that exposes metrics from containerized applications in a format that can be scraped by Prometheus. Popular metrics exporters for containerized applications include cAdvisor, Node Exporter, and Kubernetes Metrics Server.

  2. Log forwarder: A log forwarder is a component that forwards logs from containerized applications to a centralized logging system such as ELK Stack. Popular log forwarders for containerized applications include Fluentd and Logstash.

  3. Monitoring and logging platform: A monitoring and logging platform is a set of tools that provides a centralized view of monitoring and logging data for containerized applications. Popular monitoring and logging platforms for containerized applications include Prometheus, Grafana, and ELK Stack.

Example Monitoring and Logging Architecture for Containerized Applications using Prometheus, Grafana, and ELK Stack

Here is an example of a monitoring and logging architecture for containerized applications using Prometheus, Grafana, and ELK Stack:

  1. Metrics exporter: cAdvisor is used to expose metrics from containerized applications running on Kubernetes.

  2. Log forwarder: Fluentd is used to forward logs from containerized applications to Elasticsearch.

  3. Monitoring and logging platform: Prometheus is used to scrape metrics from cAdvisor and store them in a time-series database. Grafana is used to visualize monitoring data from Prometheus. ELK Stack is used to collect, index, and analyze logs from containerized applications.

Example Monitoring and Logging Configuration for Containerized Applications using Prometheus and Grafana

Here is an example of a monitoring and logging configuration for a containerized application using Prometheus and Grafana:

  1. Prometheus configuration:
scrape_configs:
  - job_name: 'cadvisor'
    static_configs:
      - targets: ['cadvisor:8080']
  - job_name: 'my-app'
    static_configs:
      - targets: ['my-app:8080']
  1. Grafana dashboard:
{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": "-- Grafana --",
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 248, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "gnetId": null,
  "graphTooltip": 0,
  "id": 1,
  "links": [],
  "panels": [
    {
      "datasource": {
        "type": "prometheus",
        "uid": "prometheus"
      },
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 0,
        "y": 0
      },
      "id": 2,
      "options": {
        "legend": {
          "calcs": [],
          "displayMode": "list",
          "placement": "bottom"
        },
        "tooltip": {
          "mode": "single",
          "sort": "none"
        }
      },
      "targets": [
        {
          "expr": "rate(container_cpu_usage_seconds_total{container_name!=\"POD\",container_name!=\"\",image!=\"\",namespace=\"my-namespace\",pod_name=~\"my-app-\"}[5m])",
          "legendFormat": "{{pod_name}}",
          "refId": "A"
        }
      ],
      "title": "CPU Usage",
      "type": "graph"
    }
  ],
  "schemaVersion": 26,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": []
  },
  "time": {
    "from": "now-6h",
    "to": "now"
  },
  "timepicker": {
    "refresh_intervals": [
      "5s",
      "10s",
      "30s",
      "1m",
      "5m",
      "15m",
      "30m",
      "1h",
      "2h",
      "1d"
    ],
    "time_options": [
      "5m",
      "15m",
      "30m",
      "1h",
      "2h",
      "1d",
      "1w",
      "1M",
      "1y"
],
"time_range": "6h"
},
"title": "My App Dashboard",
"uid": "my-app-dashboard"
}

Example Monitoring and Logging Configuration for Containerized Applications using ELK Stack

Here is an example of a monitoring and logging configuration for a containerized application using ELK Stack:

1. Fluentd configuration:
```yaml
<source>
  @type forward
  port 24224
</source>

<match **>
  @type elasticsearch
  host elasticsearch
  port 9200
  logstash_format true
  logstash_prefix my-app
</match>
  1. Kibana dashboard:
{
  "title": "My App Logs",
  "panels": [
    {
      "id": "1",
      "type": "lnsLogs",
      "gridPos": {
        "h": 12,
        "w": 12,
        "x": 0,
        "y": 0
      },
      "panelState": {
        "time\_from": "now-6h",
        "time\_to": "now",
        "refresh\_intervals": [
          "5s",
          "10s",
          "30s",
          "1m",
          "5m",
          "15m",
          "30m",
          "1h",
          "2h",
          "1d"
        ],
        "time\_interval": "auto",
        "query": {
          "language": "kuery",
          "query": "*"
        },
        "filters": [],
        "columns": [
          {
            "field": "@timestamp",
            "width": "20%"
          },
          {
            "field": "message",
            "width": "80%"
          }
        ],
        "sort\_column": "@timestamp",
        "sort\_direction": "desc"
      }
    }
  ],
  "layout": {
    "height": "750px",
    "width": "100%"
  }
}

Conclusion

Monitoring and logging containerized applications can be challenging due to their dynamic nature. However, using popular tools such as Prometheus, Grafana, and ELK Stack, platform engineers can maintain visibility into containerized applications and quickly identify and resolve issues. By following best practices for monitoring and logging, platform engineers can ensure that containerized applications are reliable and performant.

Platform Engineering containerizationinfrastructure