How to Use Kafka Connect in ksqlDB

You can use Kafka Connect either by connecting to an existing cluster or letting ksqlDB run it for you.

To set up your ksqlDB cluster and using an existing Kafka Connect cluster, set the property ksql.connect.url to the URL of your Connect cluster. Alternatively, you can also pass the environment variable KSQL_CONNECT_URL when starting your ksqlDB instance.

KSQL_CONNECT_URL='http://my-connect-cluster-host:8083'

ksqlDB has the ability to run Kafka Connect on its own without having to connect to an external Connect cluster.

To use an embedded Kafka Connect, set the ksqlDB property ksql.connect.worker.config to the path of a Kafka Connect property file. ksqlDB will run Connect in Distributed mode, so make sure you set the necessary common and distributed properties.

KSQL_CONNECT_WORKER_CONFIG='/opt/ksqldb/embedded-kafka-connect.properties'