Export Confluent Cloud Metrics to Datadog

At one of my previous assignments, we had an environment where we were running our microservices on Azure AKS clusters and were connecting to Kafka clusters in Confluent Cloud. For our monitoring and logging, we were using Datadog. Datadog has an integration with Confluent, but that is not for the Confluent Cloud platform. Since we had all of our dashboards in Datadog we really liked to have the Confluent Cloud metrics in Datadog as well. With this blog post, I will show you how to get this done when you have a similar setup.

After a bit of searching, I found a Prometheus exporter for extracting metrics from the Confluent Cloud Metrics API. All I needed to do was set up this exporter in combination with the Datadog agent, so I could send those metrics to Datadog. As we were running everything on k8s I also wanted to run this exporter on our clusters as well. On my github you can find the manifest which creates a deployment with a ReplicaSet of one and has two containers, namely the ccloud exporter and the datadog agent. Once deployed it will start fetching metrics and exporting it to Datadog.


In Datadog you go to “Dashboard > New Dashboard > New Timeboard“. On your new dashboard, you add a widget by clicking on “Add graph” and you drag the Timeseries widget to the board. By default system.cpu.user is selected as metric, but when you click on it and fill in ccloud then you will see the available metrics. A list of available metrics can also be found in the Confluent Cloud documentation.



If you need some help with creating dashboards you can check the Datadog documentation here. Once you have set up the widget with the metrics you like, you will get something like the dashboard below.

  • Production

If you have any questions then you can leave a comment or contact Bram via GithubLinkedin, or Twitter.

Originally published on Bram’s blog.

Bram is a cloud native engineer with a specialization in Azure, Kubernetes & Container Security. He is people oriented, inquisitively, eager to learn new things and he likes to take on a challenge.
March 17, 2022 | BLOG | 7 MINUTES

Automating DNS with Kubernetes and Synology DSM

After following an internal CKA training, I needed teaching material and flying hours to familiarize myself with Kubernetes to pass the CKA exam (which I …

February 24, 2022 | | 8 MINUTES

Setting up your own K3S home cluster

In 2021 I’ve received the components for my Kubernetes Cluster at home. In this blog, I want to share how I have set up my K3S home cluster because I’ve …

February 10, 2022 | | 9 MINUTES

Why you should split CI from CD!

We have used pipelines for the regular implementation of CI/CD principles for years, but we have come to a point where splitting CI from CD makes a lot of …