About
The runtime standalone mode of connect when running/starting a worker
Standalone mode is best suited for:
- testing,
- one-off jobs
- or single agent (such as sending logs from webservers to Kafka)
You can still test a distributed mode locally by setting a different rest port. See Kafka Connect - Distributed Worker
Articles Related
Management
Start
To run a worker in standalone:
./bin/connect-standalone [-daemon] worker.properties connector1.properties [connector2.properties connector3.properties ...]
# Example
./bin/connect-standalone ./etc/schema-registry/connect-avro-standalone.properties ./etc/kafka-connect-jdbc/source-quickstart-sqlite.properties
where:
- work.properties is the worker configuration file.
- connectorN.properties are connector configuration files.
Users may optionally provide connector configurations at the command line as only a single worker instance exists and no coordination is required in standalone mode.
Log
When started as a deamon, the log is located to /logs/connectStandalone.out
- Check the log
cat /logs/connectStandalone.out | grep -i "finished"
[2017-10-18 12:43:26,306] INFO Finished creating connector test-source-sqlite-jdbc-autoincrement (org.apache.kafka.connect.runtime.Worker:225)
[2017-10-18 12:43:26,425] INFO Source task WorkerSourceTask{id=test-source-sqlite-jdbc-autoincrement-0} finished initialization and start (org.apache.kafka.connect.runtime.WorkerSourceTask:143)
Configuration
Worker
Kafka Connect - Worker in standalone mode.
See Work Config Reference If you run multiple standalone instances on the same host, there are a couple of settings that must be unique between each instance:
- offset.storage.file.filename - storage for connector offsets, which are stored on the local filesystem in standalone mode; using the same file will lead to offset data being deleted or overwritten with different values
- rest.port - the port the REST interface listens on for HTTP requests
Connector
name=local-file-sink
connector.class=FileStreamSinkConnector
tasks.max=1
file=test.sink.txt
topics=connect-test