Kafka - Rest Proxy

> Data Integration Tool (ETL/ELT) > Kafka (Event Hub)

1 - About

You can read/write data via the REST Proxy.

The REST Proxy depends on the Schema Registry when producing/consuming avro data, so you’ll need to pass in the details for the detached Schema Registry host.

3 - Management

3.1 - Startup

  • Docker
docker run -d \
  --net=host \
  --name=kafka-rest \
  -e KAFKA_REST_ZOOKEEPER_CONNECT=localhost:32181 \
  -e KAFKA_REST_LISTENERS=http://localhost:8082 \
  -e KAFKA_REST_SCHEMA_REGISTRY_URL=http://localhost:8081 \
  -e KAFKA_REST_HOST_NAME=localhost \

3.2 - Port


3.3 - Consume

3.3.1 - Create a consumer instance

curl -X POST -H "Content-Type: application/vnd.kafka.v1+json" \
  --data '{"name": "my_consumer_instance", "format": "avro", "auto.offset.reset": "smallest"}' \

3.3.2 - Retrieve data from a topic

The messages will be decoded, translated to JSON, and included in the response. The schema used for deserialization is retrieved automatically from the Schema Registry service (defined at startup).

  • Example: Retrieve data from the bar topic
curl -X GET -H "Accept: application/vnd.kafka.avro.v1+json" \