Feature Request: consumer custom partition assigner

[niamster]

Description

Are there any plans to implement custom consumer assignor strategy (similar to what Sarama implements here)?
Are there any blockers?

How to reproduce

N/A

Checklist

Please provide the following information:

• confluent-kafka-go and librdkafka version (LibraryVersion()): Any
• Apache Kafka broker version: N/A
• Client configuration: N/A
• Operating system: N/A
• Provide client logs (with "debug": ".." as necessary): N/A
• Provide broker log excerpts: N/A
• Critical issue

[edenhill]

There are no immediate such plans, and we typically recommend to use the builtin partitioners as much as possible since they are compatible with partitioners in other languages (e.g., murmur2_random is compatible with the default Apache Kafka java producer partitioner).

[niamster]

Hi @edenhill . Thank you for the prompt reply. In this card I'm talking about the consumer assigner, I'm sorry I was not clear (I will update the card description).
My use case: sometimes, due to the user data submission patters, the partitions are not equally balanced from the consumers point of view, and to achieve better load distribution I would like to have a custom partition assignment strategy based on custom heuristic (for example CPU load).

[edenhill]

Oh sorry, that's my bad.

We don't have any immediate plans for that either, it is a niche use-case and requires a lot of work both in librdkafka and confluent-kafka-go.

[niamster]

Ok, thanks for the confirmation!

[niamster]

Found a sibling confluentinc/librdkafka#2284