Need a list of the available downloadable connectors

[Yashaswini1998]

Hi, can someone please direct me to the list of available downloadable list of inbound and outbound connectors?

The hub link only gives the details and not the commands to download the connectors using the cdk command as shown here for the HTTP inbound connector in the quickstart "cdk hub download infinyon/[email protected]"

The config files given in the hub link ask for a complete RUST project to be built from scratch including files like Cargo.toml etc. Can someone please share a sample project I can refer to build my custom connectors as well?

[ajhunyady]

@digikata can you please share the details, and maybe the ‘hello world’ connector as well.

[Yashaswini1998]

@ajhunyady Thank you so much for responding.

Just an update. I got the list of the available connectors by doing "fluvio hub connector list". However, the duckdb-sink connector says 404 error not found.

I'm trying the sql-sink connector now where the given config in the hub link uses json-sql smart module. The json-sql smart module also seems to be missing as this also throws a 404 Not Found error.
$ cdk hub download infinyon/[email protected]

Also, when I try to deploy the connector using cdk deploy start command, it says connector started but after listing it says Stopped under status.

I observed the same with the mqtt-source connector as well. Please find the screenshot below.

So, I think @digikata and @ajhunyady it would be really helpful if you could share that "hello world" connector along with the step-by-step instructions to set it up.

[drc-infinyon]

The sql sink connector download command is

$ cdk hub list

Which gives you

benmanns/[email protected]        public     
day-os/[email protected]        public     
infinyon-labs/[email protected]  public     
infinyon/[email protected]         public     
infinyon/[email protected]          public     
infinyon/[email protected]         public     
infinyon/[email protected]   public     
infinyon/[email protected]  public     
infinyon/[email protected]          public     
infinyon/[email protected]        public     
infinyon/[email protected]         public     
infinyon/[email protected]            public     
qdrant/[email protected]           public     
sdf-test/[email protected]   public

And then the command to download is:

cdk hub download infinyon/[email protected]

Can you share your connector config.yaml for the stopped connectors?

[ajhunyady]

@Yashaswini1998, smartmodules are downloaded on your cluster and they require a different command:

$ fluvio hub smartmodule download infinyon/[email protected]

Unfortunately DuckDb connector was experimental and we did not update it. We'll move that out of the infinyon hub.
However, http-sink should work as expected.

[Yashaswini1998]

@ajhunyady Okay, Thank you so much for your response. Can you please confirm if we can expect duckdb connector to be available anytime soon?

@drc-infinyon Thank you for your response too!
I'm using the config.yaml give in the hub documentation here for the respective connectors.
I'm following the below steps: (using mqtt-source example)

1. I download the connector cdk hub download infinyon/[email protected]
2. Create the sample-config.yaml file with the given script in the link
3. Deploy the connector cdk deploy start --config sample-config.yaml --ipkg infinyon-mqtt-source-0.2.9.ipkg
   Says started connector
4. Immediately when I try to list the deployments it says stopped
   

Am I missing any step?

Note: I have checked that the fluvio cluster is resumed and running
http-source connector works fine that is given in the quickstart

[ajhunyady]

what does cdk deploy log show?

[Yashaswini1998]

timed out searching metadata Partition failed due to timeout: 60000 ms

[ajhunyady]

A few questions:

• What environment are you running?
• Can you produce & consume from the CLI?
• What is the output of fluvio partition list ?