Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use connector for loading data from kafka (custom broker) to PG #28

Open
ctulurus opened this issue Sep 3, 2018 · 2 comments
Open

Use connector for loading data from kafka (custom broker) to PG #28

ctulurus opened this issue Sep 3, 2018 · 2 comments

Comments

@ctulurus
Copy link

ctulurus commented Sep 3, 2018

Hi
I am not very familiar with node js but I want to use this connector for a reading brokers and an insertind data into postgers. I have custom message inside brokers. It looks like {"symbol":"GOOG","close":"973.4850","lag":8}. I have read README.md but I didn't understand how to use connector for my task.
Could you write READE.md more detail about use custom broker and describe your test example.

@krystianity
Copy link
Member

Hi @ctulurus in this case you are looking at the sink connector solution.
The only thing you have to do is write a single JS file to call the runSinkConnector function and pass in the config (to connect to your Kafka cluster/broker as well as PostgresDB) and the second thing you need is an ETL function that you also have to pass to the runSinkConnector function, which defines the process of turning your Kafka message/event into a PostgresDB row for the table your are aiming at.

I might find the time to write a detailed example..

@saranOn
Copy link

saranOn commented Aug 7, 2019

Hi @krystianity
I am calling the runSinkConnector function and runSinkConnector is executed but data is not getting inserted in the table
I have ETL function and it's not being called.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants