A service for storing and retrieving versions of Avro schemas.
Schema versions stored by the service are assigned an id. These ids can be embedded in messages published to Kafka avoiding the need to send the full schema with each message.
This application provides the same API as the Confluent Schema Registry.
The service is implemented as a Rails 4.2 application and stores Avro schemas in Postgres. The API is implemented using Grape.
The Confluent Schema Registry has been reimplemented because the original implementation uses Kafka to store schemas. We view the messages that pass through Kafka as more ephemeral and want the flexibility to change how we host Kafka. In the future we may also apply per-subject permissions to the Avro schemas that are stored by the registry.
The application is written using Ruby 2.3.1. Start the service using the following steps:
git clone [email protected]:salsify/avro-schema-registry.git
cd avro-schema-registry
bundle install
bin/rake db:setup
bin/rails sBy default the service runs on port 21000.
Salsify hosts a public instance of this application at avro-schema-registry.salsify.com that anyone can experiment with, just please don't rely on it for production!
There is also a button above to easily deploy your own copy of the application to Heroku.
The service is secured using HTTP Basic authentication and should be used with
SSL. The default password for the service is 'avro' but it can be set via
the environment as SCHEMA_REGISTRY_PASSWORD.
Authentication can be disabled by setting DISABLE_PASSWORD to 'true' in the
environment.
For more details on the REST API see the Confluent documentation.
A client (see AvroTurf) can be used to communicate with the service:
url = 'https://anything:[email protected]'
client = AvroTurf::SchemaRegistry.new(url)
# registering a new schema returns an id
id = client.register('test_subject', avro_json_schema)
# => 99
# attempting to register the same schema for a subject returns the existing id
id = client.register('test_subject', avro_json_schema)
# => 99
# the JSON for an Avro schema can be fetched by id
client.fetch(id)
# => avro_json_schemaSupport for compatibility checking is incomplete. The full Confluent Schema Registry API is supported but compatibility checks are based on Avro::IO::DatumReader.match_schemas from the avro gem.
This is a basic check that ensures the top-level Avro type and name match.
In the future, a complete implementation of compatibility checking may be added.
Tests for the application can be run using:
bundle exec rspec
This code is available as open source under the terms of the MIT License.
Bug reports and pull requests are welcome on GitHub at https://github.com/salsify/avro-schema-registry.