Kafka Platform Operations teams have difficulty troubleshooting / monitoring Kafka environments due to lack of available tools. With SMM, they get intelligent filtering that allows a user to select a producer, broker, topic or consumer. SMM is smart enough to show only those producers that are sending data to the selected topics and show only those consumer groups that are consuming from those topics. The filtering works on the selection of any of the four entities. This enables users to quickly hone in on the root cause when troubleshooting and debugging Kafka issues.
One of the powerful features of SMM is its ability to visualize all the data streams/flows across all your Kafka clusters. You can select any entity such as producers, consumers, brokers or topics and visualize how data flows with respect to the entity selected.
Kafka DevOps teams can gain these benefits
Understand the end-to-end flow of message streams from producers to topics to consumers
Analyze the stream dynamics between producers and consumers using various filters
Track data lineage and governance from edge-to-enterprise
SMM has been fully integrated with Apache Atlas for governance and data lineage, Apache Ranger for security and access control management, Apache Ambari for infrastructure level monitoring and lifecycle actions for the cluster, and Grafana to be able to graph Kafka metrics over time. Click on the Atlas icon within the SMM interface to see the complete data lineage across multiple Kafka hops using SMM’s powerful data flow visualization. Jump into the actual Apache NiFi flow from that visual if NiFi is the consumer of the Kafka message.
Extend monitoring / management capabilities with REST
SMM offers REST endpoints for all of its capabilities. Monitoring / Management REST APIs are delivered through the SMM REST server. This enables developers to integrate SMM with their other enterprise tools such as APM or case/ticketing systems. Typically, Kafka developers have had trouble drilling down to message level for a given topic. But, with SMM, developers can use the visual interface or the REST API to filter and select a topic and open up all the messages within that topic.
SMM is delivered as a DataPlane Platform app. This enables customers to have a single SMM instance manage multiple Kafka clusters within their organization. You can have one single SMM instance manage multiple Kafka clusters across Hortonworks Data Platform (HDP) or Hortonworks DataFlow (HDF). Also, given that SMM is a DataPlane Platform app, this can help support hybrid cloud deployments too.
Apache, Hadoop, Falcon, Atlas, Tez, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie, Phoenix, NiFi, Nifi Registry, HAWQ, Zeppelin, Slider, Mahout, MapReduce, HDFS, YARN, Metron and the Hadoop elephant and Apache project logos are either registered trademarks or trademarks of the Apache Software Foundation in the United States or other countries.