02 Cloud infrastructure

eco system - block centered

The two possibilities of hosting environments allow slightly different deployments. In the cloud (the hosted environment) is built as a self replicating system as a multitenant solution with strict separation of tenants.

In a hosted solution, the provider of the multitenant environment is a data processor too.

In a self hosted environment, this is not required, but the service provider package is built multitenant.

The next graphics displays the high level view on the system

The solution shall be able to be build on multiple platforms since it is up to the operator to decide where it runs and individual contracts may be better than usual conditions. But for the sake of brevity, we pick one cloud provider and build the system within that environment, but offering alternatives to other clouds as well. Also important to mention, we describe the functionality in detail, assume some cloud knowledge and it is not a productive solution yet. To implement everything which is required to run it on a public internet facing environment is not part of the work here.

Current state of discussion is, that I reached out to Microsoft and they would be willing to sponsor the creation of the prototype. So we assume now azure as the platform to implement the POC for the data exchange platform.

Cloud setup

In any cloud you need to setup an environment that allows you to run applications within the cloud (a so called azure landing zone). It contains:

• user management (identity and access right) → security
• management of cloud resources → control plane, shared services
• landing zone infrastructure (the zones where applications are running) → auto scaling cluster
• connectivity (networking) → control plane
• and management (incl. billing) → cost management
• last but not least, sandboxes for running prototypes
• and more

The arrows represent a mapping to the more generic picture above.

In azure a reference application looks like this (source):

The above shown example uses uses the superset of functionality, based on the well architected framework. For the sake of being able to host this infrastructure somewhere else, Terraform is a great choice for automated deployments, since those scripts can be extended to create versions for other cloud providers. For the POC only a subset of functionality is required, also to limit spendings a downstripping is useful. 

This kind of segregation makes the handling of the cloud resources easy, limits the access rights of single individuals and is state of the art and proposed by the cloud provider, based on their extensive knowledge of cloud applications. Also it limits the impact of possible attacks. With this architecture, predefined roles and access rights for managing the infrastructure are created and can be used. Therefore we will not discuss all the internals here but refer to it (see link above).

For a productive environment its is not easily to be described generally since existing environments (as f.e. data stores) influence what is be required.

Application Landing zone

In the application landing zone, as the name suggests, are the applications hosted. The number of applications is not clear and need definition over time. But based on current user stories we require:

• a consent store
• a device store with links to its producer
• an audit trail
• a metadata store
• a service store
• search
• and more

All services should follow the bounded context principle to make them as independent as possible. This suggests a microservice architecture because the system will benefit from their scalability, extensibility. Such a microservice follows a pattern as well and they need connections to interact with each other. Additionally for productive workloads also security measures are to apply. But let's start simple too. We will have microservices with or without storage. The services need to follow the shared-nothing principle otherwise the dependencies increase to a not-maintenable state. Therefore if a service requires a storage, it defines its datamodel, protects it's data but offers an API to enable access to the data. The service who owns the data is the only one to write in the data store. There is the possibility to have multiple synchronized, read-only copies of a data store. For more on microservices read Martin Fowlers work on it.

µ-service templates

Such a template defines how to create such a microservice in the eco-system. The figures are using azure components to depict it. As this is a reactive system per nature, we recommend azure functions for compute and managed services for databases, API management. This allows the least possible costs for the price of some latency in the interaction.