kafka-log-OffsetIndex.adoc

OffsetIndex — Index Of Offsets Of Log Segment

OffsetIndex is an index of the offsets of a LogSegment.

OffsetIndex uses entrySize of 8 bytes (8-byte offset index entries).

OffsetIndex is created when:

• Lazily for a log segment (via forOffset utility that defers loading the underlying index file until needed)

• DumpLogSegments tool is used (to dump the offset and time indices of a log segment)

Tip	Enable ALL logging level for kafka.log.OffsetIndex logger to see what happens inside. Add the following line to config/log4j.properties: log4j.logger.kafka.log.OffsetIndex=ALL Refer to Logging.

Creating OffsetIndex Instance

OffsetIndex takes the following to be created:

• Underlying File

• Base offset

• maxIndexSize (default: -1)

• writable flag (default: true)

OffsetIndex initializes the internal properties.

While being created, OffsetIndex prints out the following DEBUG message to the logs:

Loaded index file [absolutePath] with maxEntries = [maxEntries], maxIndexSize = [maxIndexSize], entries = [_entries], lastOffset = [_lastOffset], file position = [position]

append Method

append(
  offset: Long,
  position: Int): Unit

append…​FIXME

Note	append is used when LogSegment is requested to append and recover.

lookup Method

lookup(
  targetOffset: Long): OffsetPosition

lookup…​FIXME

Note	lookup is used when…​FIXME

fetchUpperBoundOffset Method

fetchUpperBoundOffset(
  fetchOffset: OffsetPosition,
  fetchSize: Int): Option[OffsetPosition]

fetchUpperBoundOffset…​FIXME

Note	fetchUpperBoundOffset is used when Log is requested to addAbortedTransactions.

truncateTo Method

truncateTo(
  offset: Long): Unit

Note	truncateTo is part of the AbstractIndex Contract to…​FIXME.

truncateTo…​FIXME

lastEntry Internal Method

lastEntry: OffsetPosition

lastEntry…​FIXME

Note	lastEntry is used when…​FIXME

Internal Properties

Name	Description
_lastOffset	The offset of the last OffsetPosition Used when…​FIXME