Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MQTT Connector Enhancement - Prefix auto generated Client ID to prevent connection drops #49

Open
drc-infinyon opened this issue Jun 7, 2023 · 4 comments
Labels
good first issue Good for newcomers

Comments

@drc-infinyon
Copy link

drc-infinyon commented Jun 7, 2023

We should not use the same client_id for both, because that way mqtt server drops the connection from the older client. When id is not passed, it is randomly generated.
We should prefix auto generated identifier to client_id to prevent connection drop.

MQTT connectors config: only one of them will work, another will be in failed state:

Config Files:

apiVersion: 0.1.0
meta:
  version: 0.2.1
  name: helsinki-bus-filter
  type: mqtt-source
  topic: helsinki
  direction: source
  create-topic: false
mqtt:
  url: "mqtt://mqtt.hsl.fi"
  topic: "/hfp/v2/journey/ongoing/vp/+/+/+/#"
  client_id: "my_mqtt"
  timeout:
    secs: 30
    nanos: 0
  payload_output_type: json
transforms:
  - uses: infinyon/[email protected]
    with:
      spec:
        - operation: shift
          spec:
            payload:
              VP:
                lat: "lat"
                long: "long"
                veh: "vehicle"
                route: "route"
                spd: "speed"
                tst: "tst"

and

apiVersion: 0.1.0
meta:
  version: 0.2.1
  name: helsinki-bus
  type: mqtt-source
  topic: helsinki
  direction: source
  create-topic: true
mqtt:
  url: "mqtt://mqtt.hsl.fi"
  topic: "/hfp/v2/journey/ongoing/vp/+/+/+/#"
  client_id: "my_mqtt"
  timeout:
    secs: 30
    nanos: 0
  payload_output_type: json

When id is not passed, clientId is randomly generated
We should auto generate prefix (probably from connector name) as default unless it is explicit.

@sehz
Copy link
Contributor

sehz commented Jun 7, 2023

this behavior should also be doc in the mqtt connector

@morenol
Copy link
Contributor

morenol commented Jun 23, 2023

Updated documentation to remove client_id occurrences, so user do not use it to repeat configuration when following tutorials. For prefix auto-generated client id, we should create a new parameter

@digikata
Copy link
Contributor

add a fluvio issue to guide oss help

@digikata
Copy link
Contributor

digikata commented Mar 11, 2024

Notes for implementors:

  • In config.rs client_id might need to change to an option type and a new optional [client_id_prefix] should be added.
  • defining both client_id and client_id_prefix should be an error. (we could change client_id to an enum type to express this)

There are several cases to consider:

  • (already implemented) client_id derived from connection url
  • client_id not provided -> autogenerate a client id string for connection to mqtt server (prefixed by connector.meta.name helsinki-bus-filter in top example)
  • client_id provided -> use literal client_id string for connection
  • client_id_prefix provided -> autogenerate id and prepend client_id_prefix string to the id for connection

Edit:
Added note to client_id not provided cases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

4 participants