Kafka Connect: Unable to trace topics created by source connector in KafkaTopic CR

[viopinto]

Hi,

I have created source connectors using Kafka Connect REST APIs.
However, I am unable to see the topics created by the source connector on listing KafkaTopic CR using following command,

kubectl get kafkatopic -n strimzi20

Whereas, the topic is available in Kafka when listed within the pod using the following command,

bin/kafka-topics.sh --list --bootstrap-server my-cluster-kafka-bootstrap:9092

Could you please advise on this behavior.

[scholzj]

You should probably check the Topic Operator logs then. It is one of the containers in the Entity Operator pod.

[viopinto]

Thanks @scholzj.
The observation is that, the topic name at the CR is appended with an alphanumeric id to that of the topic created at Kafka.

The topic created at Kafka by the source connector is a_ksource, whereas the topic reflecting when listing the CR is a-ksource---27a435dbbe24eee47c915281e4ea73c2a7cb0fc

Is this something to be concerned about? How do we approach in a case where I would require to create a topic via CR before pushing the data through a source connector, given that the alphanumeric id gets appended on auto creation through the connector instance.

[scholzj]

This is expected. k_source is not a valid Kubernetes name. So it cannot be named like this. That is why we generate this kind of valid but unique name: https://strimzi.io/docs/operators/latest/full/using.html#kafka_topic_naming_conventions