A dependency-free library that defines the types used to describe platform components.
A platform is make up of services components and, optionally, services can be grouped into aggregates components. Each component has a descriptor: a java class that defines metadata about the component. This metadata is used by Creek to provide its functionality and can be used by other components. For example, the output of one component can be used to define the input of another.
There are two broad types of descriptors: component and resource descriptors.
Component descriptors define, among other things, a component's inputs, internals and outputs. There are two types of component: aggregate and service descriptors.
Authors of Creek based services will build implementations of both aggregate and service descriptors.
The inputs of a component define any external resources the component consumes/reads.
All input resources implement the ComponentInput interface.
The internals of a component define any external resources the component uses internally to perform its function.
All internal resources implement the ComponentInternal interface.
The outputs of a component define any external resources the component produces/writes.
All output resources implement the ComponentOutput interface.
Services should provide an implementation of ServiceDescriptor
to define key metadata needed by other components and the Creek system, including things like
inputs, internals and outputs, docker image name and any
environment variables to set during testing.
Each aggregate should provide an implementation of AggregateDescriptor
to define their public inputs and outputs.
Resource descriptors define the resources a component uses in its inputs, internals and outputs.
There are corresponding ComponentInput, ComponentInternal and ComponentOutput interfaces, respectively.
Authors of creek services are not expected to implement the above interfaces. Creek extensions, (such as creek-kafka), expose their own interfaces, which extend these interfaces, and which can be used to define a resource the component uses, e.g. a Kafka Topic.
The reason extensions expose resource interfaces rather than classes is to keep code coupling to a minimum, thereby minimising chances of dependency hell. Extensions provide example code that can be cut & paste for creating the required implementations.
Conceptually, resources, such as a Kafka topic or database, can be owned by a particular service, owned by another service, or shared.
This is encoded into the type of the resource descriptor, e.g. an OwnedKafkaTopicInput vs (unowned)KafkaTopicInput.
Creek supports the following resource descriptor ownership model:
- owned: a descriptor to a resource conceptually owned by the service.
- unowned: a descriptor to a resource conceptually owned by another service.
- shared: a descriptor to a resource not conceptually owned by any service. (This should be a rare thing in a well architected platform).
- unmanaged: a descriptor to a resource which Creek will not manage.
Resources tagged with the OwnedResource interface are conceptually owns by the service.
For example, services generally own their Kafka output topics, as this is the data they are responsible for generating and managing. Therefore, the service's descriptor will define an owned descriptor for this resource, e.g. defining the topic name, key & value types, partition count, and any other parameters required to allow the service to create the topic.
When the service starts, Creek will automatically create the resource when initializing the service.
Future plans are to support a mode where owned resources are created by an initialization tool, prior to deployment. See issue-68.
The owned resource types provided by extensions will define helper methods to obtain an unowned descriptor from an
owned one. For example, the OwnedKafkaTopicOutput has a toInput method to obtain an unowned KafkaTopicInput.
Resources tagged with the UnownedResource interface are conceptually owns by another service.
For example, services generally consume Kafka the owned output topics of upstream services. Therefore, the service's descriptor will define an unowned descriptor for such resource, e.g. defining the topic name, key & value types, partition count, etc.
When the service starts, Creek will not initialize unowned resources.
Unowned resource types provided by extensions should not normally be directly created. Instead, the unowned descriptor
should be created by calling an appropriate helper method on the owned resource. For example, an unowned KafkaTopicInput
is obtained by calling toInput on the associated OwnedKafkaTopicOutput.
Resources tagged with the SharedResource interface are conceptually not owned by any service.
A shared resource's descriptor will normally be defined in a common library and referenced by all components that wish to use it.
Shared resources are initialised via the Init tool before any service that requires them are deployed.
Shared resource types provided by extensions will implement the ComponentInput, ComponentInternal and/or ComponentOutput
interfaces, as appropriate for the resource type. This allows a shared single definition to be used directly as a component's
input, internal or output.
Any resource descriptor that does not implement one of the resource initialization interfaces are deemed not to be initialized by Creek. Such resources must be initialized some other way.
Resources, for example Kafka topics or databases, can be initialized/created at different points in the deployment process. Most resources are created by the service that conceptually owns them when the service starts for the first time. However, some resources may be to be created before a service starts, e.g. shared resources: resources that are not owned by any one single service.
Creek supports initializing resources at different points in the process by defining the following stages of resource initialization:
- init: Initialize an environment with any resources that should be created before services are started. This is generally shared resources, though, in the future Creek will also support pre-initializing owned resources as well.
- service: Initialize any resources that the service conceptually owns and which are managed by Creek.
- test: A special stage used for system testing: for a given subset of components under test, initialize a test environment to ensure all unowned resources are created.
The Init tool can be used to manually initialize resources at these different stages.