Title

Tooling for the generation of database DDL (Data Definition Language) schemas for the storing of VSS data and associated meta-data.

Introduction

There are performance and functionality benefits for defining the schema for how data is stored in a database. For example in SQL using the CREATE statement.

For hacking the creation of database DDL for a small number of VSS nodes can be maintained by hand, but this scales badly for the hundreds of nodes in the VSS reference catalog and for automation. This proposal therefore proposes the creation of tooling to automate the schema creation.

Challenges

1. The output of the generator shall be capable of being used as input to further tooling, e.g. tooling that executes the schema in the database.
2. As the output can be used as input to further tooling, it should allow configuration of output to support major options in the DB schema without manual editing, to support automation of the complete processing chain.
3. The generator shall have an architecture that supports extension for further DBs unless it becomes very clear that different DBs require seperate implementations for recognised architectural reasons.
4. The generator shall separate input of VSS catalog, DB meta-data and VSS overlays. This allows a separation of concerns.
5. VSS overlays should be supported as an input to allow the set of VSS nodes to be processed to be defined. Along with additional meta-data.
6. DB meta-data should be supported to allow the generator to make effective use of DB features such as data-types, encoding, attributes and tags.
7. Development should consider whether it should be implemented as part of the VSS-tools exporter toolset.
8. Development should consider what existing tooling eco-systems exist that it could leverage. For example, mapping VSS vspec to a memory model in VSS-tools, or third-party DB tooling.

Proposal

The diagram below proposes one logical concept as a solution, using Apache IoTDB and the Commercial Vehicle Guidelines as examples.

Development could be phased to allow early functional wins and to get feedback.

Goal

Tooling that supports the generation of DB DDL for the creation of schemas for the VSS data-model and any other model using the vspec notation.

Inputs / Feedback

1) API spec as input (Daniel Alvarez Nov 2024)

Thread with Daniel on Slack summarized here as it will disappear after 90 days. He suggested that if an API can be used as an input it would likely provide hints as to what is needed for the output:

"I thought that if you have the api specification given, one can select the concepts of interest from the specification and generate the schema for the database that matches exactly that. In your case, I’m assuming that the database can be set from scratch up. So, no dependencies or constraints are given. Wouldn’t be easier then to make the database compliant to the api specification?"

Stephen Lawrence : I agree with Daniel's idea that if an API spec for a function + meta-data is available then that should be helpful for generating a schema. That could be an enhancement. We also have the use-case where a DB is used as a data store, with no API available or multiple possible accessors and a straight forward VSS representation is required, e.g. a starting point in the COVESA Central Data Service Playground where VSS data can immediately be stored.