feat: add extra_listeners

[marcoppenheimer]

Changes Made

refactor: add extra_listeners config, deprecate certificate_extra_sans

• extra_listeners replaces certificate_extra_sans
• If user provides value with structure foobar-{unit}.baz:30000, they will get listeners like foobar-0.baz:39092 on unit 0
• When the charm requests a certificate, still using certificate_extra_sans, they will get a deprecation warning in the logs
• extra_listeners will be preferred, but default to certificate_extra_sans if unset

refactor: move password actions to actions.py

• Mostly housekeeping, easier to have most 'non-specific' actions in a single place

feat: add get-listeners Action

• Outputs data about the current advertised.listeners for the unit the Action runs against
• Output:

client-sasl-plaintext-scram-sha-512:
  advertised-listener: CLIENT_SASL_PLAINTEXT_SCRAM_SHA_512://10.58.254.50:9092
  auth-mechanism: SCRAM-SHA-512
  name: CLIENT_SASL_PLAINTEXT_SCRAM_SHA_512
  port: "9092"
  protocol: SASL_PLAINTEXT
  scope: CLIENT
extra-sasl-plaintext-scram-sha-512-0:
  advertised-listener: EXTRA_SASL_PLAINTEXT_SCRAM_SHA_512_0://kafka-0.foo-domain.com:39092
  auth-mechanism: SCRAM-SHA-512
  name: EXTRA_SASL_PLAINTEXT_SCRAM_SHA_512_0
  port: "39092"
  protocol: SASL_PLAINTEXT
  scope: EXTRA
extra-sasl-plaintext-scram-sha-512-1:
  advertised-listener: EXTRA_SASL_PLAINTEXT_SCRAM_SHA_512_1://kafka-0.bar-domain.com:49192
  auth-mechanism: SCRAM-SHA-512
  name: EXTRA_SASL_PLAINTEXT_SCRAM_SHA_512_1
  port: "49192"
  protocol: SASL_PLAINTEXT
  scope: EXTRA
internal-sasl-plaintext-scram-sha-512:
  advertised-listener: INTERNAL_SASL_PLAINTEXT_SCRAM_SHA_512://10.58.254.50:19092
  auth-mechanism: SCRAM-SHA-512
  name: INTERNAL_SASL_PLAINTEXT_SCRAM_SHA_512
  port: "19092"
  protocol: SASL_PLAINTEXT
  scope: INTERNAL

Manual Validation

Attempted to verify behavior using the following set-up:

• Host machine
• (on host) LXD VM - 10.155.67.110 (enp5s0)
• (on nested LXD on the VM) Kafka container unit - kafka/0 has IP 10.58.254.246 with hostname juju-ecb145-0

Configured Kafka to expose multiple extra advertised.listeners:

# relevant one is EXTRA_SASL_PLAINTEXT_SCRAM_SHA_512_2://juju-ecb145-0:39292

advertised.listeners=INTERNAL_SASL_PLAINTEXT_SCRAM_SHA_512://10.58.254.247:19092,CLIENT_SASL_PLAINTEXT_SCRAM_SHA_512://10.58.254.247:9092,EXTRA_SASL_PLAINTEXT_SCRAM_SHA_512://kafka-0.foo-domain.com:39092,EXTRA_SASL_PLAINTEXT_SCRAM_SHA_512_1://kafka-0.bar-domain.com:39192,EXTRA_SASL_PLAINTEXT_SCRAM_SHA_512_2://juju-ecb145-0:39292

Edited DNS on the host machine, to point the Kafka unit hostname to point to the Kafka unit IP:

#/etc/hosts
10.58.254.247 juju-ecb145-0 kafka-0
10.58.254.147 juju-ecb145-4 kafka-1

Right now, this isn't resolvable.

Then, manually added an IP route on the host machine, to reach the Kafka unit IP via the enp5s0 LXD VM network:

sudo ip route add 10.58.254.247 via 10.155.67.110
sudo ip route add 10.58.254.147 via 10.155.67.110

Finally, on the host machine, attempted to create a topic on a single Kafka unit, attempting to reach kafka-0:39292 as the bootstrap-server, which succeeded:

# bootstrapping on kafka/0
INFO:kafka.conn:<BrokerConnection node_id=bootstrap-0 host=kafka-0:39292 <connecting> [IPv4 ('10.58.254.247', 39292)]>: connecting to kafka-0:39292 [('10.58.254.247', 39292) IPv4]                               
INFO:kafka.conn:Probing node bootstrap-0 broker version                                                                                                                                                           
WARNING:root:<BrokerConnection node_id=bootstrap-0 host=kafka-0:39292 <authenticating> [IPv4 ('10.58.254.247', 39292)]>: Exchanging credentials in the clear                                                      
INFO:root:<BrokerConnection node_id=bootstrap-0 host=kafka-0:39292 <authenticating> [IPv4 ('10.58.254.247', 39292)]>: Authenticated as admin via SCRAM-SHA-512                                                    
INFO:kafka.conn:<BrokerConnection node_id=bootstrap-0 host=kafka-0:39292 <authenticating> [IPv4 ('10.58.254.247', 39292)]>: Connection complete.                                                                  
INFO:kafka.conn:Broker version identified as 2.6.0                                                                                                                                                                
INFO:kafka.conn:Set configuration api_version=(2, 6, 0) to skip auto check_version requests on startup                                                                                                            
INFO:kafka.conn:Probing node bootstrap-0 broker version                                                                                                                                                           
INFO:kafka.conn:Broker version identified as 2.6.0                                                                                                                                                                
INFO:kafka.conn:Set configuration api_version=(2, 6, 0) to skip auto check_version requests on startup

# successful connection to advertised kafka/1                                                                                                     
INFO:kafka.conn:<BrokerConnection node_id=1 host=juju-ecb145-4:39292 <connecting> [IPv4 ('10.58.254.147', 39292)]>: connecting to juju-ecb145-4:39292 [('10.58.254.147', 39292) IPv4]                             
INFO:kafka.conn:Probing node 1 broker version                                                                                                                                                                     
WARNING:root:<BrokerConnection node_id=1 host=juju-ecb145-4:39292 <authenticating> [IPv4 ('10.58.254.147', 39292)]>: Exchanging credentials in the clear                                                          
INFO:root:<BrokerConnection node_id=1 host=juju-ecb145-4:39292 <authenticating> [IPv4 ('10.58.254.147', 39292)]>: Authenticated as admin via SCRAM-SHA-512                                                        
INFO:kafka.conn:<BrokerConnection node_id=1 host=juju-ecb145-4:39292 <authenticating> [IPv4 ('10.58.254.147', 39292)]>: Connection complete.                                                                      
INFO:kafka.conn:<BrokerConnection node_id=bootstrap-0 host=kafka-0:39292 <connected> [IPv4 ('10.58.254.247', 39292)]>: Closing connection.                                                                        
INFO:kafka.conn:Broker version identified as 2.6.0                                                                                                                                                                
INFO:kafka.conn:Set configuration api_version=(2, 6, 0) to skip auto check_version requests on startup 

[grobbie]

Some comments. The description is ok. I don't have a problem with a single knob that encapsulates a bunch of complex stuff, personally.

[deusebio]

I got a bit confused with the extra_count and a couple of "conflicting" comments, which have probably created some misunderstanding. But I would like to be sure we are on the same page on the UX, and probably simplify the code (skipping the extra_count) which also created a bit of misunderstanding (and I don't think it is clarified in the config option description)

[deusebio]

LGTM! Just some non-blocking suggestions/nipticks attached

[Batalex]

LGTM, a fascinating addition to the charm