SixthSense agent will remotely monitor Kafka i.e. the agent will run as a separate process and will connect to the JMX port exposed by Kafka for fetching the metrics. Below are the steps to enable Kafka monitoring:
Enabling remote JMX support in Kafka through environment variables/JVM args.
(optional) Enabling authentication on the port exposed by Kafka for JMX remote monitoring.
JMX remote support can be enable through environment variable or JVM args:
a. Need to pass “sun.management.jmxremote” as true in order to enable JMX remote support. Following are the other mandatory JVM arguments for configuring JMX remote. These JVM arguments will be picked by Kafka after restart.
This will be done by Java-provided local JMX authentication method, which stores credentials and controls access using a local file
Create a new folder called jmxremote
> mkdir -vp /etc/jmxremote
Create two files in this directory
password
access
The password file has the credentials that the JMX agent will check during client authentication.
> cat /etc/jmxremote/jmxremote.password
sixthsense R@kuten@1234
info
Here the file content is registering the user "sixthsense" with password "R@kuten@1234"
The access file has authorization information that is who is allowed to do what.
> cat jmxremote/jmxremote.access
sixthsense readwrite
Change the permissions on both files so that only owner can edit and view them.
> chmod -R 600 /etc/jmxremote/jmxremote.*
Add the following
sun.management.jmxremote.authenticate=true,
com.sun.management.jmxremote.access.file=/etc/jmxremote/jmxremote.acces”s
com.sun.management.jmxremote.password.file=/etc/jmxremote/jmxremote.password” in JVM_ARGS.
Navigate to agent.config file i.e. Agent folder/config/agent.config file. The agent can be configured by adding the information in the agent.config file.
Following is a agent.config file section where brokers, producers, consumers and JMX host and port details must be updated:
Broker configuration
brokerConfigs:
- brokerUrl: ${SS_BROKER1_URL:localhost}
name: ${SS_BROKER1_URL:broker1}
brokerPort: ${SS_BROKER1_PORT:1100}
username: ${SS_BROKER1_USERNAME:}
password: ${SS_BROKER1_PASSWORD:}
configType: ${SS_CONFIG_TYPE:kafkabroker}
Producer configuration
- brokerUrl: ${SS_BROKER1_URL:localhost}
name : ${SS_BROKER1_URL:producer1}
brokerPort: ${SS_BROKER1_PORT:1200}
username: ${SS_BROKER1_USERNAME:}
password: ${SS_BROKER1_PASSWORD:}
configType: ${SS_CONFIG_TYPE:kafkaproducer}
Consumer configuration
- brokerUrl: ${SS_BROKER1_URL:localhost}
name : ${SS_BROKER1_URL:consumer1}
brokerPort: ${SS_BROKER1_PORT:1300}
username: ${SS_BROKER1_USERNAME:}
password: ${SS_BROKER1_PASSWORD:}
configType: ${SS_CONFIG_TYPE:kafkaconsumer}
Following is the section of agent.config file where agent general configuration must be updated.