kafka-zk-AdminZkClient.adoc

AdminZkClient

AdminZkClient is created when:

• ConfigCommand is requested to processCommandWithZk

• ReassignPartitionsCommand is requested to verifyAssignment, generateAssignment, executeAssignment

• TopicCommand is requested to createTopic, alterTopic, describeTopic

• Partition is requested to getOrCreateReplica

• DynamicBrokerConfig is requested to initialize

• AdminManager, DynamicConfigManager and KafkaApis are created

Creating AdminZkClient Instance

AdminZkClient takes the following to be created:

• KafkaZkClient

AdminZkClient initializes the internal properties.

Reading Entity Configuration From Zookeeper (Under /config Node) — fetchEntityConfig Method

fetchEntityConfig(
  rootEntityType: String,
  sanitizedEntityName: String): Properties

fetchEntityConfig requests the KafkaZkClient to get the configuration from Zookeeper (under /config node) for the given rootEntityType and sanitizedEntityName.

Note

fetchEntityConfig is used when: ConfigCommand command-line tool is requested to alterConfig and describeConfig ReassignPartitionsCommand command-line tool is requested to removeThrottle, maybeLimit and assignThrottledReplicas TopicCommand command-line tool is requested to alterTopic and describeTopic Partition is requested to look up or create a replica AdminManager is requested to describe the configuration of the specified resources (topics and brokers) and incrementalAlterConfigs DynamicBrokerConfig is requested to initialize ConfigChangedNotificationHandler (of DynamicConfigManager) is requested to processEntityConfigChangeVersion1 and processEntityConfigChangeVersion2 AdminZkClient is requested to getAllTopicConfigs, fetchAllEntityConfigs, and fetchAllChildEntityConfigs

getAllTopicConfigs Method

getAllTopicConfigs(): Map[String, Properties]

getAllTopicConfigs…​FIXME

Note	getAllTopicConfigs is used when…​FIXME

fetchAllEntityConfigs Method

fetchAllEntityConfigs(entityType: String): Map[String, Properties]

fetchAllEntityConfigs…​FIXME

Note	fetchAllEntityConfigs is used when…​FIXME

fetchAllChildEntityConfigs Method

fetchAllChildEntityConfigs(
  rootEntityType: String,
  childEntityType: String): Map[String, Properties]

fetchAllChildEntityConfigs…​FIXME

Note	fetchAllChildEntityConfigs is used when…​FIXME

writeTopicPartitionAssignment Internal Method

writeTopicPartitionAssignment(
  topic: String,
  replicaAssignment: Map[Int, Seq[Int]],
  update: Boolean): Unit

writeTopicPartitionAssignment…​FIXME

Note	writeTopicPartitionAssignment is used exclusively when AdminZkClient is requested to createOrUpdateTopicPartitionAssignmentPathInZK.

createOrUpdateTopicPartitionAssignmentPathInZK Internal Method

createOrUpdateTopicPartitionAssignmentPathInZK(
  topic: String,
  partitionReplicaAssignment: Map[Int, Seq[Int]],
  config: Properties = new Properties,
  update: Boolean = false): Unit

createOrUpdateTopicPartitionAssignmentPathInZK…​FIXME

Note	createOrUpdateTopicPartitionAssignmentPathInZK is used when: TopicCommand is requested to create a topic AdminManager is requested to create topics AdminZkClient is requested to create a topic and addPartitions

Creating Topic — createTopic Method

createTopic(
  topic: String,
  partitions: Int,
  replicationFactor: Int,
  topicConfig: Properties = new Properties,
  rackAwareMode: RackAwareMode = RackAwareMode.Enforced): Unit

createTopic fetches the metadata of the brokers in the cluster (given the RackAwareMode).

createTopic requests AdminUtils helper object to assignReplicasToBrokers (given the broker metadata, partitions and replicationFactor).

In the end, createTopic createOrUpdateTopicPartitionAssignmentPathInZK.

Note	createTopic is used when: TopicCommand is requested to create a topic KafkaApis is requested to create a topic

addPartitions Method

addPartitions(
  topic: String,
  existingAssignment: Map[Int, Seq[Int]],
  allBrokers: Seq[BrokerMetadata],
  numPartitions: Int = 1,
  replicaAssignment: Option[Map[Int, Seq[Int]]] = None,
  validateOnly: Boolean = false): Map[Int, Seq[Int]]

addPartitions…​FIXME

Note	addPartitions is used when…​FIXME

Fetching Metadata of Brokers in Cluster (Broker ID and Rack Information) — getBrokerMetadatas Method

getBrokerMetadatas(
  rackAwareMode: RackAwareMode = RackAwareMode.Enforced,
  brokerList: Option[Seq[Int]] = None): Seq[BrokerMetadata]

getBrokerMetadatas requests the KafkaZkClient for the metadata of the brokers in the cluster.

Note	The broker metadata includes a broker ID, endpoints, and an optional rack information.

getBrokerMetadatas takes the input broker IDs (brokerList) if defined and leaves only those that are available (i.e. among the brokers in the cluster) or falls back to all brokers in the cluster.

getBrokerMetadatas branches off per RackAwareMode as follows:

• For RackAwareMode.Disabled or RackAwareMode.Safe, getBrokerMetadatas returns BrokerMetadata with broker IDs and no rack information

• For RackAwareMode.Enforced, getBrokerMetadatas returns BrokerMetadata with broker IDs and rack information

In the end, getBrokerMetadatas sorts the brokers by broker ID.

getBrokerMetadatas throws an AdminOperationException for RackAwareMode.Enforced mode with some brokers without rack information:

Not all brokers have rack information. Add --disable-rack-aware in command line to make replica assignment without rack information.

Note	getBrokerMetadatas is used when: AdminManager is requested to createPartitions AdminZkClient is requested to create a topic ReassignPartitionsCommand is requested to generateAssignment TopicCommand is requested to alterTopic