Kafka - Schema

> Data Integration Tool (ETL/ELT) > Kafka (Event Hub)

1 - About

You can use an schema, for example, to:

Advertising

3 - Syntax

An Avro schema defines the data structure in a JSON format.

{"namespace": "example.avro",
 "type": "record",
 "name": "user",
 "fields": [
     {"name": "name", "type": "string"},
     {"name": "favorite_number",  "type": "int"}
 ]
}

where:

4 - Management

Schema management is done through the schema registry service

4.1 - List all schemas (subject ?)

Example made with sqlite getting started

curl -X GET http://localhost:8081/subjects
["test-sqlite-jdbc-accounts-value"]

4.2 - List all schema versions

List all schema versions registered under a subject. Example made with sqlite getting started

curl -X GET http://localhost:8081/subjects/subjectName/versions
 
# Example
curl -X GET http://localhost:8081/subjects/test-sqlite-jdbc-accounts-value/versions
[1,2]
Advertising

4.3 - Fetch a schema

Fetch version n of the schema registered under the subject. Example made with sqlite getting started

curl -s -X GET http://localhost:8081/subjects/subjectName/versions/n | jq .
 
# Example
curl -s -X GET http://localhost:8081/subjects/test-sqlite-jdbc-accounts-value/versions/1 | jq .
{                                                                                                                                                                                                          
  "subject": "test-sqlite-jdbc-accounts-value",                                                                                                                                                            
  "version": 1,                                                                                                                                                                                            
  "id": 1,                                                                                                                                                                                                 
  "schema": "{\"type\":\"record\",\"name\":\"accounts\",\"fields\":[{\"name\":\"id\",\"type\":\"long\"},{\"name\":\"name\",\"type\":[\"null\",\"string\"],\"default\":null}],\"connect.name\":\"accounts\"}"                                                                                                                                                                                                          
}

The schema property properly indented is:

{
	"type ": "record",
	"name":"accounts",
	"fields": [{
			"name":"id",
			"type":"long"
		}, {
			"name":"name",
			"type": ["null","string"],
			"default": null
		}
	],
	"connect.name":"accounts"
}

4.4 - Delete

# Deletes all schema versions registered under the subject "Kafka-value"
$ curl -X DELETE http://localhost:8081/subjects/Kafka-value
  [1]
 
# Deletes version 1 of the schema registered under subject "Kafka-value"
$ curl -X DELETE http://localhost:8081/subjects/Kafka-value/versions/1
  1
 
# Deletes the most recently registered schema under subject "Kafka-value"
$ curl -X DELETE http://localhost:8081/subjects/Kafka-value/versions/latest
  1

5 - Documentation / Reference