kafka-log-LogConfig.adoc

LogConfig

LogConfig is an AbstractConfig for the topic-level configuration properties.

LogConfig uses the keys that are symonyms (aliases) of the keys of TopicConfig. As the TopicConfig itself says: "Eventually this should replace LogConfig.scala."

Table 1. LogConfig’s Topic-Level Configuration Properties

Name	Description
cleanup.policy	Cluster-wide property: log.cleanup.policy Default: delete Used when: GroupCoordinator is requested for the offsetsTopicConfigs TransactionStateManager is requested for the transactionTopicConfigs LogConfig is requested for the compact and delete flags KafkaServer utility is used to create a default log properties for LogManager and AdminManager
compression.type
delete.retention.ms
file.delete.delay.ms	The time (in millis) to wait before deleting a file from the filesystem. Must be at least 0 Property: file.delete.delay.ms Default: 60000
flush.messages	Cluster-wide property: log.flush.interval.messages
flush.ms	Cluster-wide property: log.flush.interval.ms
index.interval.bytes
max.compaction.lag.ms
max.message.bytes
message.downconversion.enable
message.format.version
message.timestamp.difference.max.ms
message.timestamp.type
min.cleanable.dirty.ratio
min.compaction.lag.ms
min.insync.replicas	Cluster-wide property: min.insync.replicas Used when: TransactionStateManager is requested for transactionTopicConfigs KafkaServer utility is used to create a default log properties for LogManager and AdminManager (with the value of KafkaConfig.minInSyncReplicas) Partition is requested to isUnderMinIsr, isAtMinIsr, checkEnoughReplicasReachOffset, appendRecordsToLeader
preallocate
retention.bytes	Used when: KafkaServer utility is requested for default configuration properties of logs Log is requested to deleteRetentionSizeBreachedSegments
retention.ms	Cluster-wide property: log.retention.ms Used exclusively when Log is requested to deleteRetentionMsBreachedSegments
segment.bytes	Cluster-wide property: log.segment.bytes Used when: GroupCoordinator is requested for the offsetsTopicConfigs TransactionStateManager is requested for the transactionTopicConfigs KafkaServer utility is used to create a default log properties for LogManager and AdminManager The current value is available as LogConfig.segmentSize and used when: RollParams utility is requested to create a RollParams Log is requested for the initFileSize and to append records Cleaner is requested to clean a log
segment.index.bytes	Cluster-wide property: log.index.size.max.bytes
segment.jitter.ms
segment.ms	Cluster-wide property: log.roll.ms Default: 604800000 (7 days) Used when: Log is requested to maybeRoll LogConfig is requested for the randomSegmentJitter and maxSegmentMs KafkaServer utility is used to create a default log properties for LogManager and AdminManager
unclean.leader.election.enable	Cluster-wide property: unclean.leader.election.enable Disabled (false) explicity for the __transaction_state internal topic (when TransactionStateManager is requested for the transactionTopicConfigs for KafkaApis) Used when: ZkPartitionStateMachine is requested to collectUncleanLeaderElectionState DynamicLogConfig is requested to reconfigure (and requests the KafkaController to enableDefaultUncleanLeaderElection)

TopicConfigSynonyms

TopicConfigSynonyms are the topic-level configuration properties (with the highest priority) with their corresponding cluster-wide (broker-level) configuration properties.

Note	TopicConfigSynonyms uses the same configuration properties as KafkaServer.copyKafkaConfigToLog utility.

Table 2. TopicConfigSynonyms

Topic-Level Property Name	Cluster-Wide Property Name
cleanup.policy	log.cleanup.policy
compression.type	compression.type
delete.retention.ms	log.cleaner.delete.retention.ms
file.delete.delay.ms	log.segment.delete.delay.ms
flush.messages	log.flush.interval.messages
flush.ms	log.flush.interval.ms
index.interval.bytes	log.index.interval.bytes
max.compaction.lag.ms	log.cleaner.max.compaction.lag.ms
max.message.bytes	message.max.bytes
message.downconversion.enable	log.message.downconversion.enable
message.format.version	log.message.format.version
message.timestamp.difference.max.ms	log.message.timestamp.difference.max.ms
message.timestamp.type	log.message.timestamp.type
min.cleanable.dirty.ratio	log.cleaner.min.cleanable.ratio
min.compaction.lag.ms	log.cleaner.min.compaction.lag.ms
min.insync.replicas	min.insync.replicas
preallocate	log.preallocate
retention.bytes	log.retention.bytes
retention.ms	log.retention.ms
segment.bytes	log.segment.bytes
segment.index.bytes	log.index.size.max.bytes
segment.jitter.ms	log.roll.jitter.ms
segment.ms	log.roll.ms
unclean.leader.election.enable	unclean.leader.election.enable

Note	TopicConfigSynonyms is used when: AdminManager is requested to describe the configuration of a topic and create topics (and createTopicConfigEntry) DynamicLogConfig utility is requested for the ReconfigurableConfigs and the KafkaConfigToLogConfigName

randomSegmentJitter Method

randomSegmentJitter: Long

randomSegmentJitter is one of the following values:

• 0 if segment.jitter.ms is 0

• Random int value modulo the minimum of segment.jitter.ms and segment.ms

Note	randomSegmentJitter is used when LogSegment utility is used to open a log segment (for the rollJitterMs property).

maxSegmentMs Method

maxSegmentMs: Long

maxSegmentMs…​FIXME

Note	maxSegmentMs is used when RollParams utility is used to create a RollParams instance.

Checking Out Whether Compact Cleanup Policy Is Enabled — compact Flag

compact: Boolean

compact…​FIXME

Note	compact is used when…​FIXME

Checking Out Whether Delete Cleanup Policy Is Enabled — delete Flag

delete: Boolean

delete flag is enabled (true) when delete cleanup policy is part of the cleanup.policy configuration property. Otherwise, delete flag is disabled (false).

Note	delete is used when: Partition is requested to deleteRecordsOnLeader Log is requested to deleteOldSegments LogCleanerManager utility is requested for isCompactAndDelete