(star) WIP Warning – Work in progress! – WIP Warning (star)


What is it and why?

The VSS Taxonomy defines what data "entities" (Signals and Attributes) we can deal with, and are used in the protocol(s) defined by W3C Automotive Working group, as well as other initiatives inside and outside the vehicle.

But in addition to VSS itself, we need to define the data-exchange formats for measured values of those Signals.  This starts by defining terms, but quickly develops into defining one or several variants of the actual message content format, whether in JSON or other.

Relationship to Protocols

Data formats sometimes overlap protocol definitions in the sense that a protocol, or stack of protocols, must at some point clarify the used data formats, or no understandable exchange can be had.

However, based on the protocols we are discussing, more is required. 
The data exchange protocols we discuss fall into different categories, each requiring some more work on defining value exchange data formats:

  1. Protocol does not (yet) cover all variations of data exchange.
    1.  VISS / W3C Gen2 covers a lot of usage, but supports primarily fetching a single "latest value" measurement, or transferring instant updates according to subscriptions. But the bigger picture should include also exchanging sets of previously measured data between systems.  We can benefit from adding transfer of historically measured data, or derived statistics (e.g. get the average value over time instead of transferring all values), and more.  This work could help protocols like W3C "Gen 2" to cover these bases.
  2. Protocol defines only a "transport"
    1. We often discuss protocols that define some mechanisms of data transfer, such as pub/sub semantics, but they are designed to be generic and assume that any type of information can be transferred by the protocol.  Specifically, they do not strictly define the format of the content of the data container (payload).  This makes the protocols widely applicable, but is ultimately not enough without also defining the payload format.  Examples of some such protocols would be MQTT or WAMP
  3. Protocol defines transport, query semantics, and some rules for data formats, but are still too generic.
    1. GraphQL is a generic technology that is complemented by a schema that defines what types of queries can be made and the datatypes that are expected to be returned.    For that reason it is required to agree on the definition of schemas and types in a particular usage.  This could also be derived from this generic analysis.


Definitions

(proposal, open for discussion)

Signal:

Request:

Job:  

Observation:


Main "Container" – name for this?   Measurement? 

=   A delivery of data sent at a particular time.

(this means the Message that is in response to a Request)

Might need to include some metadata regarding the request:

Discussion: What is the general metadata needed when measuring data?


Record:  (or Measurement, or Observation?)


(question) Why not just use a single record type (superset of all functionality)?  

A: The reason would be to optimize the performance and bandwidth.  In other words, don't transfer what is not needed for a certain case.  If a timestamp is not needed, we should make sure we support transferring data without providing a timestamp, for example.


Record Subtypes:

+ Record types which specify the signal name inside: 


TimeStamp

NOTE:  The exact format of all of these may differ when these concepts are translated to different protocols or languages.
As a starting point however, let's propose to use ISO 8601 standard format, with fractional seconds (e.g. microseconds) and always UTC (Zulu) time zone, if a TimeStamp is given in text format.  For other purposes such as binary formats, more efficient encodings should be considered. 

Real, "Wall clock time"

Relative to a previously predefined time stamp reference:


Bundle

TimeSeries

Snapshot:

Note that values in a Snapshot need a record type that specifies the signal, since different signals are included in the same message.


Stream:



Examples, in JSON


(Plain) Record:

{
   "value" : " 100.54"
}

TimestampedRecord:

{
   "ts" : "2020-01-10T02:59:43.491751Z   # Zulu time, ISO std with microseconds
   "value" : "42 "
}

SpecifiedRecord:

{
   "signal" : "vehicle.Chassis.Axle2.WheelCount"
   "value" : "2"
}

SpecifiedTimestampedRecord:

{
   "signal" : "vehicle.Body.ExteriorMirrors.Heating.Status"
   "ts" : "2020-01-10T02:59:43.491751"
   "value" : "false"
}

TimeSeries:

{ 
    "signal"  :  "vehicle.body.cabin.temperature"
    "count"   : "132"  # Might be redundant information, optional.
    "values" : {
        { 
            "ts" : "2020-01-10T02:59:43.491751"
            "value" : "42 "
        },
        { 
            "ts" : "2020-01-10T02:59:43.491751"
            "value" : "43"
        },
        ... 130 more records
    }
}

Snapshot:

{ 
    "timeperiod" { 
        "start" : "2020-01-10T02:00:00Z",
        "end" : "2020-01-11T01:59:59Z"
    },
    "values" : {
     {
        "signal" : "vehicle.body.cabin.temperature",
        "value" : "22",
        "ts" : "2020-01-10T02:59:43.491751"
     },
     {
        "signal" : "vehicle.drivetrain.engine.rpm.average",
        "value" : "3200",
        "ts" : "2020-01-10T02:59:44.100403"
     }
}