Get hands-on with Kafka monitoring setup with Prometheus and Grafana, Kafka operations and Kafka cluster upgrades Setup in AWS.
About This Video
In Detail
This course helps you learn Kafka administration, …
Get hands-on with Kafka monitoring setup with Prometheus and Grafana, Kafka operations and Kafka cluster upgrades Setup in AWS.
About This Video
In Detail
This course helps you learn Kafka administration, Kafka monitoring, Kafka operations, and Kafka upgrades. It covers the following topics:
All the code files are placed at https://github.com/PacktPublishing/Apache-Kafka-Series---Kafka-Monitoring-and-Operations
Downloading the example code for this course: You can download the example code files for all Packt video courses you have purchased from your account at http://www.PacktPub.com. If you purchased this course elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.
Chapter 1 : Course Overview and Introduction
Course Overview 00:07:10
Chapter 2 : Kafka Quick Setup in AWS
What is AWS CloudFormation 00:01:32
Target Architecture and Setup 00:04:19
[Hands-On] Deploy Zookeeper Using CloudFormation 00:08:16
[Hands-On] Deploy Kafka Using CloudFormation 00:07:38
[Hands-On] Overview of our Zookeeper Cluster 00:09:26
[Hands-On] Overview of our Kafka Cluster 00:05:03
[Hands-On] Stopping the Kafka Cluster 00:02:50
Chapter 3 : Kafka Administration Setup
[Hands-On] Overview and Setup of Monitoring Machine 00:08:10
[Hands-On] Setup Zoonavigator 00:07:52
[Hands-On] Setup Kafka Manager 00:08:08
[Hands-On] Setup Kafka Monitor 00:09:28
Chapter 4 : Kafka Monitoring Setup: Grafana + Prometheus
Overview of Monitoring 00:05:06
[Hands-On] Setting up Prometheus + Kafka Broker 1 00:17:31
[Solution] Broker 2 & 3 Prometheus Setup 00:05:21
[Solution] Zookeeper & Prometheus Setup 00:09:40
[Hands-On] Setup Grafana on Administration Server 00:06:49
[Hands-On] Setup Kafka Dashboard on Grafana 00:05:44
Important Metrics to monitor 00:06:05
[Hands-On] Adding Graphs to Grafana 00:10:16
[Solution] Adding more graphs to Grafana 00:03:01
[Hands-On] Observing Grafana when we kill a broker 00:05:05
Considerations to make the setup production-ready 00:02:00
Chapter 5 : Kafka Operations
[Hands-On] Install Jolokia Agent on Broker 1 00:07:54
[[Solution] Install Jolokia on Broker 2 & 3 00:03:38
[Hands-On] Install Rolling Restart Script 00:14:04
[Hands-On] Run Roll Restart Script 00:07:21
[Hands-On] Updating Configurations 1st method (rolling restart) 00:03:45
[Hands-On] Updating Configurations 2nd method (kafka-configs command) 00:11:19
[Hands-On] Rebalancing Partitions using plain commands and JSON 00:12:10
[Hands-On] Rebalancing Partitions using Kafka Manager 00:02:41
[Hands-On] Rebalancing Partitions using LinkedIn tools 00:08:11
[Hands-On] Increasing Replication Factor using plain commands and JSON 00:06:38
[Hands-On] Increasing Replication Factor using LinkedIn tools 00:04:47
[Hands-On] Adding Broker 4 using CloudFormation 00:09:55
[Solution] Moving partitions over to Broker 4 using the CLI tools 00:04:30
[Hands-On] Replacing a Broker keeping EBS Volume 00:07:03
[Hands-On] Replacing a Broker losing an EBS Volume 00:03:41
Removing a Broker Overview 00:01:01
[Solution] Remove Broker 4 00:08:01
Chapter 6 : Kafka Cluster Upgrade
Upgrading a Kafka Cluster Overview 00:02:20
[Advanced] Zero Copy, Up and Down Conversions 00:08:06
[Hands-On] Kafka Broker Upgrade Step 1 00:04:02
[Hands-On] Kafka Broker Upgrade Step 2 00:02:58
[Hands-On] Kafka Broker Upgrade Step 3 00:03:03
[Hands-On] Kafka Broker Upgrade Step 4 00:01:40
[Hands-On] Kafka Broker Upgrade Step 5 00:03:15
Chapter 7 : Next Steps
Cleaning up AWS 00:01:17
Next Steps & Congratulations! 00:02:02