The minutes of the meetings for the Data Architecture and Infrastructure pillar of the Data Expert Group.
Historical note: the minutes for the previous project, the CVII Tech Stack, can be found here: CVII Technology Stack meeting notes
18th October 2023
Agenda:
- Meeting invite: Christian, Sri
- AMM wash-up
- Goal: what went well, could be better, sessions/input we should be aware of
- General including input from related sessions, e.g. Ulf's southbound session
- Data Architecture workshop
- Central Data Service playground proposal
- Goal: move towards project kick-off and work breakdown
- Further input?
- Agree next steps
- Do we have general agreement/understanding on what we are doing regarding the service itself initially (the big picture What) and can move to getting the ball rolling on the details (e.g. checking docker status and choosing requirements of some use cases) of the how?
- How to organise? e.g. Kanban, JIRA. Account for schedules and time zones by doing what we can online.
- AoB
- Adnan: serialisation
- Looking at msg serialisation between vehicle and car. Requirement to keep flexibility in terms of what is sent.
- BMW will have a researcher looking into this. Can something be done in open? Payload only.
- Looking for input.
Minutes:
- Serialisation
- Adnan outlines (Adnan Bekan correct me if my summary is incorrect):
- BMW will have a researcher looking into message serialisation between vehicle and car.
- Scope is the payload only. Not the transport. However some flexibility in terms of what is sent in the payload is required.
- Can something be done in the open in standardising this? Also looking for input on approaches.
- Adnan notes the work done in the past during CVII Tech Stack days using AVRO with the esync alliance, but feels that is a heavyweight solution.
- Various suggestions made about some possible approaches, such as encoding methods that take advantage of sparcity or lack of change.
- Steve: the cloud connection has been cold for a while but only because its been waiting for someone to run with it. This should be of interest a wide group. I would recommend writing an outline or proposal of what problems you want to tackle down, along with some requirements. From that other inputs can be gathered along with people to work on it and a project formed. This should be of interest to the Commercial Vehicle group for example and their telematics requirements.
- AMM wash-up
- Christian: various gateway solutions appearing, e.g. PennyBecker, HIM, Vehicle API etc. Implementations at different levels. Would be good to understand what they look to achieve as some are not directly comparable.
- Arnaldo: difficulty for newcomers to get started. Wiki can be overcomplex to understand.
- Data Arch landing page needs cleanup / simplification.
- Neil: Agree. Some attendees talked to showed interest, but not everyone understands.
- General agreement for the need for more guiding documents like HowTos.
- Steve: I intend to remake the Data Arch landing page as discussed before the AMM to make it easier for newcomers.
- Other sessions - Ulf southbound architecture session
- Steve: Peter Winzell said in the W3C slack that he was working on a southbound feeder to WAII connected to Redis. Is it related to your work.
- Ulf: Peter is using the WAII feeder framework I described in my session. His feeder connects to the RemotiveLabs simulator. As part of his work he's also updated the WAII DockerFile to use the latest code.
- Central Data Service playground proposal
- Only 10 mins remained. Will be main topic next week.
- Steve: Please consider what the near term next steps are, what use cases you want to explore and how we should organise the work.
12th October 2023 - AMM Data Architecture Workshop
- (these are incomplete notes made after the event)
- Workshop delivered as planned, except for compression of time on doc proposal:
- Mongo DB/BMW (9-9:45)
- 10-10:15 break
- Central Data Service Playground (10:15 on)
- Part 1: Readout of the proposal
- Part 2: Workshop the base components and the possibilities for 'spins' using it
- As well as the names below Piotr Krawczyk Neil Puthuff and Halim Ragab actively engaged in the discussion, along with Arnaldo's colleagues. Apologies if I missed anyone from the list - let me know.
- Piotr suggested MQTT as event queue (see whiteboard picture)
- Halim was interested in the project but wanted to know the boundary performance (latency etc) of the Service to be able to understand how it would fit in the overall performance of an end to end touchpoint. He took an action to provide input on what GM requires.
- Outcome: Ford (Ulf Bjorkengren), BMW (Christian Muehlbauer), MongoDB (Arnaldo Vera Humza Akhtar) and Renesas (Stephen Lawrence ) all expressed interest in actively contributing. Enough interest to begin the project.
- Whiteboard Images:
- Group shot after the end of the session:
- Documentation proposal discussion: 30 mins to be fitted in
- As the Playground workshop was going well this was condensed with Steve simply showing some slides on the wider eco-system need for documentation on HowTo's etc.
4th October 2023
- Christian back after a long holiday. Brought him up to speed and reconfirmed MongoDB/BMW session starting the Architecture workshop
- Central Data Service playground proposal
- Steve had got enough of the draft outline written for the intention hopefully be clear. Ran through it with the group.
- Little time left to discuss but Arnaldo, Christian and Ulf are postive about the discussion.
- Once the runthrough is repeated for others in the work at the AMM an early task will be to get feedback from the Architecture members.
27th September 2023
- AMM
- Discussion of what to report in the Data EG readout
- Data Arch
- Thurs AMM workshop
- ..
- Ulf Southbound session
- Data Arch workshop
- Arnaldo confirmed the MongoDB/BMW session.
- Steve has written some of the outline for the Central Data Service playground proposal and went through it. Target will be to get more of it done for next week.
20th September 2023
- AMM planning
- Workshop
- Mongo DB/BMW input
- Christian on holiday. Arnaldo not present.
- Steve: Anything discussed whilst I was away?
- Paul B: Arnaldo does not want to run a PoC specific workshop in Detroit. MongoDB session currently moved out in schedule to accomodate possible workshop. Can move back if it makes sense.
- After discussion the running order from 6th Sept minutes still makes most sense. Basically take MongoDB/BMW sessions and Ulf Bjorkengren session on VISS southbound as input to move onto workshop the Central Data Service Playground propose.
- Discussion of the Ford data model presentation prior Tues and the Blackberry IVY showcase webinar.
13th September 2023
- Paul Boyes facilitated meeting whilst Stephen was on holiday.
6th September 2023
- AMM planning
- AMM Planning October 10-12 2023 - Technical Session and Workshop Proposals
- Thursday (9-12:15)
- Mongo DB/BMW (9-9:30)
- Christian says specifics depends on readiness of the PoC.
- Plan A is to present evolution of the Tiered Sync they have been working on.
- Recommends contacting Mongo DB to check what plan B is.
- Central Data Service Playground (9:30 on)
- Part 1: Readout of the proposal (9:30-9:45/50)
- 10-10:15 Break
- Part 2: Workshop the base components and the possibilities for 'spins' using it, e.g:
- Data models: knowledge layer, data models, VSSo
- Interface pillar: IFEX, service definition, Vehicle API/Autosar
- Data Architecture: Sync, reasoning at the edge etc.
- Documentation proposal discussion: 30 mins to be fitted in
- Knowledge Layer proposal
- Christian will send slides for presentation he made last week
- Paul reminds about the Ford/VSSo presentation next week. See community calendar and slack for details
- Holidays:
- Christian: on holiday from 11th Sept, back 4th Oct. May check email/slack time to time.
- Steve: holiday Tues 12th-Tues 19th.
- Paul will host meeting next week.
30th August 2023
- Debrief of Christian's presentation of his Knowledge Layer proposal Tues to VSSo group
- Has synergy with the Central Data Service playground
23rd August 2023
10th August 2023
- With summer holidays a sparsely attended meeting this week. Started with 3, became 5 in 2nd half. As such more informal discussion.
- Some discussion of Tim Welsch's meeting yesterday on Service Registry.
- Some discussion of Christian's Knowledge Layer proposal and what he is considering in certain areas.
2nd August 2023
- Knowledge Layer proposal
- Christian Muehlbauer has posted his first outline
- Steve: what would you like to achieve and how best to take this forward?
- Details in the proposal but Christian would like to explore what would be needed to have a technology agnostic seperation between knowledge and the underlying areas. How would they interact and stay in sync etc.
- After discussion about way forward, Christian happy for people to promote its existance. Paul/Steve will mention it to other groups. Christian will think how he would present the work in a future 1 hr type dedicated meeting to the topic.
- Document and diagram storage
- Steve was planning on creating data-arch-doc.git rep to get us started.
- He brought topic up in Thursday call. Erik mentioned the existing (little used currently) Data EG git rep. That is probably a good place for top level diagrams like the scope diagram.
- AMM planning
- Steve has added a planning page here 2023 Autumn AMM (Data Architecture and Infrastructure)
- Steve: any feedback from request last week to consider what we want to discuss?
- Christian Muehlbauer discussing internally about what we might be able to show. Workshop could discuss knowledge layer proposal.
- Arnaldo Vera Some possibilities we are discussing: Tooling for conversion to JSON-schema. Realm for OTA. Connected car sync.
- Steve mentions Adnan Bekan PR for a VSS vspec to json-schema tool. Arnaldo will take a look.
- Stephen Lawrence Store/Server playground, Design doc publishing.
- Ulf Bjorkengren plans to present update on HIM including new additions.
- Paul Boyes interesting to see conceptual design/requirements for connecting signals to non-signal info/knowledge in-vehicle and on to cloud.
- Discussion of how this relates to other areas. Some topics likely best as a wider group workshop, e.g. combining data models
- Paul will be posting table for Thursday tracks for planning purposes
- Data Pattern proposal
- Steve has restarted completing the text. Currently working on the publishing section
- Data Arch front page redo is open AI.
26th July 2023
- Data Pattern proposal
- No change due to the collab meetings last week. Meetings confirm the need - see below.
- Steve will return to completing the draft.
- Knowledge layer proposal
- Christian catching up after holiday. Still intends to upload the proposal for feedback.
- Topics arising from collaboration meetings (EU Horizon, SDV Summit, Covesa Board F2F) last week
- The Covesa Scope diagram was used in EU Horizon SDV group to discuss SDV. It was also used in SDV Summit.
- This both shows Covesa materials being used externally and their value. Basically further proof of the need being addressed in the Data Pattern proposal
- This flags again some recent topics: where to store them and enhancing discoverability for external people
- Christian had been considering a diagram showing how Data Arch relates to the other topics. He'll have a go at drafting it.
- Git rep / where to put diagrams
- We have had some discussion previously but now with the external use we need to move to a solution to encourage re-use.
- Steve: I still am not in favor of a 'big bucket' repository containing all source and docs. Source should be organised by product for reuse. I would suggest starting with a git rep dedicated to documentation, e.g. data-arch-doc.git
- Discoverability
- Feedback from last week shows discoverability could be improved.
- Christian: when I first came to Covesa/VSS work I found it easiest to orientate myself from the VSS (Hugo) document website.
- We know this and had been prioritising getting the tech work going. However now need to address it.
- Three main touchpoints immediately come to mind:
- data-arch-doc.git
- Holding doc, diagram source
- Hugo data arch documentation website
- Serving rendered documents as per the document proposal
- Confluence Data Arch landing page
- Reforming group landing page
- Steve: I'll try and find the time to bring the Data Arch Landing Page up to date, e.g. show the Scope diagram there. At least a draft for review by the group.
- AMM
- Steve: we need to start considering what we want to do at the AMM.
- I would expect a Data Arch pillar intro as usual, possibly as part of Data EG intro.
- I would also expect somekind of combination workshop + presentations for the Thursday as we have done in the past, but need your input.
- AI: Consider what you want to discuss/present
- Discussion on process of creating content for Hugo documentation website
- Christian: People may think it is a single persons viewpoint if its all markdown in a single git post
- Steve: Yes collaboration on the text will be needed in many cases. I would suggest for such documents the draft text be worked on somewhere collab is easier. Confluence has inline comments for example. Of course direct to markdown or something else is also possible. Point is to make it easier to comment and discuss.
- Steps could be:
- Creation stage:
- Collab on text (Confluence, whatever)
- Convert to markdown(github)
- Sign-off
- Publishing stage:
- Move markdown to Hugo tree
- CI Runner generates Hugo site and publishes to production
- <Possible to have a parallel CI for WIP if needed>
19th July 2023
- No meeting due to collaboration meetings F2F in Cologne
12th July 2023
Agenda:
- Data Pattern / Arch proposal
- Discussion of changes since last week
- Knowledge layer
- No discussion as Christian on holiday
- Next meetings / holiday planning
- Ulf: 2 weeks after F2F
- Arnaldo: last two weeks
- Richard: 18-21st Aug.
- Meeting cancelled Weds 19th due to meetings in Cologne
- News
- VISS WAII reference implementation
- gRPC added to MQTT, WS and HTTP as transport protocol. Payload is compatiable with the others.
- Discussion pending in W3C VISS standards group to add gRPC to VISSv2 standard.
5th July 2023
- Knowledge layer
- Christian has made further progress on his document.
- Discussion of some of the points.
- It is complete enough that Christian would soon like to get feedback and input and asks about how best to do that.
- In terms of publicity Steve says Paul can help, but he would suggest other Data EG meetings and the slack.
- In terms of publishing Steve says he was meaning to add a "Proposals" sub-page to the Data Arch Confluence to organsie proposals under including his own Doc proposal
- Data Pattern / Data Architecture proposal
- Resolved new comment from Christian
- The Current Situation section draft is now complete.
- Started work on the proposal section. Added some bullets to the summary outline.
- Next few meetings
- Steve: I may have a funeral to attend next Weds. There is also the Board F2F the following week and I will not be able to run the meeting. Do you still want to meet?
- Holidays:
- Christian: July 6→16th,
- Ulf: not confirmed but expect End of July→mid Aug
- Steve: considering a week at the end of July but would probably be around to run the meeting.
- As only three people on the call Steve will send an email to the group to see what people want to do.
28th June 2023
- Knowledge Layer
- Christian Muehlbauer showed the state of his work (currently in Word doc) on his Knowledge Layer ideas.
- Positive feedback for him to continue.
- Some discussion of the implications.
- Arnaldo raises point about sharing knowledge decision is analogous to conflict resolution in data centric arch sync. Who decides the knowledge, communicates it and receives an update. Only want to store state when it changes.
- Steve suggested a top down approach to the description to seperate the most generic ideas from the more specific ones. Basically to avoid a 'must adopt it all to be relevant' issue.
- Steve asked Christian to let us know when he wanted to get a wider view on the work.
- Data Pattern / Data Architecture proposal
- Steve has made some good progress since last week.
- Majority of the text for the current state of play is now done.
- Will now move onto next steps.
- Socialised the initial proposal and the idea to use Hugo for publishing to Technical Steering Committee and Data Expert Group with no push back.
- Git rep naming
- Steve gave some thought to this after last weeks discussion.
- Still reluctant to have a single 'big bucket' git rep for the usability and maintainability reasons that is not normally done in engineering practise.
- However likely don't want 20 reps either.
- Steve: How about as a starting point to have some level of classification to narrow its scope, e.g. data-arch-doc.git or data-arch-doc-arch-patterns.git?
- No clear answer. Team asked to think on it as we will need to make a decision fairly soon.
- News
- Ulf Bjorkengren has published documentation in his HIM proposal
- In VSSo call there was a heads up that Daniel Wilms had a proposal for using GraphQL schema to create VSS graphs. Intention was to present to the VSS group in a couple of weeks.
- Adnan explains that Data Centric Architectures has been listed as a need in the topics to be tackled within the EU Horizon SDV project.
21st June 2023
- Data Arch rep naming discussion
- Short initial discussion of how to organise git repositories for the data arch pillar.
- Single 'big bucket' vs project specific?
- Ulf suggests big bucket
- Steve concerned about maintainability of big bucket.
- Steve: please consider what you want and we will discuss again
- Refining Steve's Data Pattern / Data Architecture proposal
- Steve outlined his idea to use online publishing paradigm already used in Covesa for VSS doc. Brings all the docs together and is useful to link to for non-engineers.
- Feedback was positive.
- Christian added comment about sync which is now resolved
- In meeting added Knowledge layer as a topic to the content list
- Christian working on outline based on idea he explained recently
- Wants to know where he would put it?
- Steve: need to resolve the question of data arch rep naming. In meantime depending on your format you can push to the google drive, create personal git rep, or I can create you one in my area.
14th June 2023
- Knowledge layer (45 min)
- Christian Muehlbauer has noted that in the Covesa organisation slides showing the Data EG pillars that the knowledge layer is listed under the Data Models / Ontologies. Should it be in Data Architecture instead/as well?
- Discussion of the question. Christian has a draft diagram showing Machine Learning and Semantic Reasoning (requires ontologies like VSSo) acting on VSS data as a means for deriving high level knowledge. This is an example of how the Knowledge Layer is also applicable to Data Architecture. Ulf likes the diagram and mentions he intends to explore some of this in his HIM project.
- One area of possible interest is standardised knowledge representations. You derive some knowledge from standard data. Is there a standard way to represent the knowledge? This has benefits in loosly coupling the knowledge, from possible consumers.
- Conclusion is everyone agrees that it also applies to Data Architecture in such a context and could be considered part of the scope. Open question as to how to include it in such a summary slide. For sure it could be part of any Data Architecture pillar description that was not so space constrained.
- Data Pattern / Data Architecture proposal (15 mins)
- As discussed last week Steve has started work on a proposal for Design Pattern and Data Architecture documentation which can be found here https://wiki.covesa.global/x/boA8B
- Principal aim is to use it facilitate what to create.
- Short on time so quick run through. It's still a WIP towards a draft. Please edit or add in-line comments.
7th June 2023
- Data Architecture Terminology.
- Following last weeks discussion Steve has moved the Data Sync description from Data Store into its own section.
- Next steps in the high level work..
- S: There has been some useful conversations at the AMM and after that built understanding. I would like to focus a discussion on advancing the high level shared work in the group.
- Revisiting needs
- Top down
- We know there is a need for a set of shared high level views. Both internally in the group and externally as newcomers try to understand VSS and it's eco-system.
- As we have discussed where sensible these should use logical components to avoid getting trapped down in the weeds in system architecture, product selection etc.
- For illustration some examples:
- Terminology
- Common patterns
- Building blocks both descriptive and PoC (trial, playgrounds etc.)
- User stories and deployment scenarios illustrating the combination of the above
- A few of us have been illustrating some examples previously and most recently.
- e.g. Adnan last week showed some early high level diagrams in the field of VSS+methods / services / vehicle API
- Scenario 1: HVAC service <=> gateway (vehicle API - VSS Data model) | Autosar / RT / RTOS (southbound low level parts)
- RT area may contain proprietary information and ASIL that is contained.
- Gateway provides abstraction API to e.g. get/set/sub VSS data
- HVAC service provides service API to rest of system. Acting both as an abstraction and handler for the complexities of the southbound system. e.g. something like HVAC is often CAN based which brings complexity when trying to set values.
- Felix illustrated two data store variants last week
- Whilst thinking about it Steve was reminded of similar high level descriptions in http://github.com/slawr/vss-otaku
- Various earlier ones: Miro, Ulf's work, AMM etc.
- S: I propose to combine high level description with these kind of light weight illustrative examples in diagram+text form as a way forward.
- I think I will follow the approach Daniel Alvarez has taken to his VSSo proposal of a top down logical approach, avoiding solutions and write up a proposal for some documents that can be commented and the beginning of the next phase of shared work.
- Brain storming some classifications:
- Patterns / building blocks:
- Data Server, Data Store
- Variants: Mobile App, in-vehicle app, Domain Data Service
- Integration
- Variants: data flow bi/uni-directional, conflict resolution, latency, volume, frequency
- Scenarios
- Snapshots focusing on specific areas: Vehicle API, Service + gateway (as in Adnan example above), etc.
- Data EG Deployment scenarios: IVI, Smart device, Car2Cloud, Cloud2Cloud,
- Data EG Touchpoints
31st May 2023
- Data Architecture Terminology.
- Steve presented updated terminology that has a first pass approach to adding Data Store sync
- Felix suggested moving Sync into its own component description. That matches one of the options previously discussed. Steve happy to try it that way also.
- Wide ranging discussion of the needs and approaches to getting at a flexible, reusable set of high level component diagrams/descriptions
- Felix diagrammed some options re the Data Server and Data Store variants. Discussion included differences between a 'fat' Data Store that might be connected to a Data Gateway, versus an in-vehicle application (e.g. following MVC design pattern) that has an API and local 'thin' Data Store dedicated to it.
24th May 2023
- Data Architecture Terminology.
- Steve has reworked the VSS-Feeder component text to be VSS Connector as discussed last week.
- Discussion of handling Data Sync function as its own component so it can be a part of both Data Server and Data Store. Agreement to try it that way. Arnaldo will take a look at the description based on their experience with Realm Sync.
- PoC
- Ulf asks about the goals of the PoC. After summary from Steve (see 10th May minutes for summary) he thinks it could be useful and hopes to find some time to participate.
17th May 2023
- Minutes made a week later and so may be less complete
- Data Architecture Terminology.
- Continued review discussion
- Sync
- Christian provided input offline about including Sync as he has discussed in his presentations.
- Open question was how best to include it and preserve the fact that the functional descriptions can be combined in a product. So whilst it might be most likely in the State Storage as he has been using it, it could be elsewhere. Perhaps describe it as component itself, which can then be used by either State Storage or Server?
- Data Feeders
- Ulf feels Feeder suggests a uni-directional flow Northbound when it could be in both directions. Can we rename to something else? He suggests Bridge but is open to alternatives.
- Steve also suggests Connector.
- Steve will try rewriting the description using the suggested names.
- Diagrams
- Steve has started recreating the Deployment Scenario diagrams from the 2023 Q1 Workshop in Miro https://miro.com/app/board/uXjVPoh2zow=/?share_link_id=262317958896 in DrawIO so they can be easily shared, and edited.
- As a working area he has created the sub-folder Deployment_scenarios in the Data Arch scratch area of the Data Expert Group Google Drive.
- First diagram to be converted was the top level Logical Architecture showing the Data Expert Group scope which can be found here Logical Architecture Overview.drawio.svg
- This was a first pass conversion. Not yet looked to see if it needs resizing to fit into common use cases in A4 document or slide
- Happy to get help in recreating the others..
10th May 2023
- Data Architecture Terminology.
- Steve did first pass review and made some edits.
- Got feedback from Felix.
- Intention is that along with the diagrams this becomes the basis for 'logical component' discussion in the Data Expert Group
- Diagrams
- Christian: We should show that the data server/state storage combination in-vehicle can be reused in the cloud and mobile device as well, i.e. data centric architecture reuse of arch.
- PoC
- Christian: Is this intended to be public?
- Steve: Yes
- Illustrations to show possibilities:
- Covesa github all public
- x86 docker
- Major components of the basic 'lego':
- VISS data server, e.g. WAII
- State Storage, e.g. add Realm, Apache IoTDB
- Step 1: DBs via WAII (CCS) State Storage component
- Step M..N trials using the basic lego to prove/investigate various topics:
- High frequency
- Knowledge pyramid, AI at the edge
- Multiple data models
- Automotive H/W
- Generic code could be implemented on Automative H/W for some appropriate topics, e.g Latency Southbound
- Feedback:
- Piotr: Could have a PoC variant that shows support for different OSs via vsock for example to show possibility of southbound data from various sources in the car.
3rd May 2023
- Welcome to Arnaldo Vera from MongoDB. Short overview of the work of the Data Architecture pillar.
- People report successful AMM
- Combining VSS with other data models
- Logistically neither Ulf or Christian get to each others presentations on this subject
- Positive discussion to look at the proposals and find commonalities for way forward
- Logical Components / Terminology
- Discussion of the usefulness of logical component diagrams as found in the CVII Tech Stack/CCS diagrams and the Terminology Dictionary.
- Avoids the competitive aspects of talking about products, falling into specific system architectures and gives a common approach for discussion.
- Useful for design patterns, deployment scenarios and architecture documents.
- How to move towards a common view?
- Review terminology list.
- Christian will see how the components fit the i7 showcase demo.
- Collectively iterate both the terminology and diagrams.
- Data Server / State Storage PoC basic building block
- Steve summarised the discussion Thursday at the AMM to create a basic building block PoC consisting of a Data Server and State Storage.
- The aim is a reusable component that could be configured to extend it for use in various deployment scenarios, e.g. connecting Vehicle API gateway.
- At the same time the basic block could also be used to investigate connections between server and storage, e.g. low latency data.
26th April 2023
19th April 2023
- AMM planning
- Steve added an AMM sub-folder to the group scratch Google Drive:
- Tues Intro
- Steve has merged the prior outline into the template and uploaded a new working draft to the above folder.
- Needs completing by Steve/Christian
- Thur Data Arch Workshop
- Arrivals
- Christian arriving Sunday, but working on car Monday. Steve Sunday. Felix Monday.
12th April 2023
- AMM planning
- Tues Intro
- Steve/Christian discussed the outline.
- Steve will redo the draft outline in the AMM template towards end of the week.
- Christian will work on his Data Centric slides in parallel.
- Aim to merge and finalise next week
- Wed Deployment Scenarios
- Thur Data Arch Workshop
- No outlines yet from Christian or Felix (see last week) for their proposals, or Felix's workshop.
- Christian: I am meeting with Felix tomorrow in Munich and should be able to finalise my session afterwords.
- Discussion of Deployment Scenarios proposal. See workshop planning page for notes.
- Piotr: Might be idea to put Deployment first in the running order as somewhat of an introduction.
- Piotr: One topic for discussion could be the management of version compatibility between components
- Piotr describes some examples. Such as Vehicle API supporting different VSS versions
- Steve: Interesting questions. Can you create something like a summary of the problem scope and what needs to be tackled? That could then be used in various ways at the AMM, e.g. Vehicle API workshop Friday and afterwords.
- Piotr: yes.
5th April 2023
- AMM planning
- Intro
- Steve: As the Intro contains slides from Christian on his views on Data Centric Arch would he like to present that part?
- Christian: OK
- Deployment Scenarios
- Steve expresses concern that we (Data EG) advance the Deployment Scenarios whilst at the AMM. 1.5h session is not enough..
- Data Arch Workshop
- Share updates and proposals:
- Christian thinks he can do a Different Data Domains (approx 20-30 mins)
- Presentation on BMW experience.
- Starting from using personal data as example of different data domain and moving to data middleware.
- Steve: m/w may be handy bridge towards follow on discussion of persistence
- Felix: Persistence
- Presentation on classifying different persistence options
- Steve: Persistence
- Steve: My usual topics:
- Data Server to Data Store connection, including high volume data
- How Data Store can support Data Centric Arch, e.g. Data analytics using TS DB features
- etc.
- AI: Add outlines for each proposed session to the planning page so ppl have an idea of what is to be discussed and what the hoped for outcome is.
- Thurs PM/Friday
- Return of experience: Felix proposed a workshop using tech from the work between BMW and Mongo.
- AI (Felix): add outline to the planning page as above.
- From discussion this could be scheduled Thurs PM
22nd March 2023
Agenda:
- Roadmap creation
- Any updates on internal discussions / Action Items
- AMM Sessions: Intro, Deployment Scenarios, Workshop
- Roadmap creation
- Updates on internal discussions / Action Items?
- Christian summarised the result of BMW discussions about the AMM. It's not finalised by some update on results from the internal PoC work should be possible. Unclear at the moment if its a longer session, video of the PoC, or some slides.
- Organisation
- AMM
- Steve summarised the three main Data Arch sessions:
- Tues PM: "Data Arch Intro (currently called "Data Centric Arch" in schedule)" (Steve/Adnan)
- Weds PM : "Deployment Scenarios" diagrams (Adnan/Erik/Steve)
- Thurs AM: 9-11am Data Arch Workshop (all)
- In addition there is a Data Expert Group general update that will mention the Data Arch pillar (Steve) and Friday an all day Data EG Workshop which of course will include Data Arch topics
- Data Arch Intro
- Steve showed a rough outline of the ppt discussed previously
- Group agrees with the approach
- Workshop
- Steve: I would like to have a topic skeleton for us to follow to keep things on track and avoid too much unproductive tak
- Felix suggests 3-4 topics if we have two hours
- Felix and Steve both interested in Data Storage/Persistance.
- Christian suggests requirements of "Different Data Domains (which means also models)"
- Discussion of possibilities
- Felix offers to do 10 mins on OSS version (minus sync) of Realm
- Steve shows the diagram hacked on last week which facilitates discussion of relationship between storage, server and feeder (Felix away last week). Interested in possibilities that functional storage brings, e.g. alternator health example. Felix makes good suggestion to have cleaned-up version which includes high level topics to tackle down (saves jumping around slides in discussion)
- Group agrees this is a way forward. Needs further work.
- AI (Steve): Provide a placeholder abstract for the Thursday workshop. Website currently shows the one from last year.
- Timings
- Steve: Thursday PM is not packed with sessions so we may be able to expand if needed.
- Felix: I do not leave until Sunday.
- Steve: I am the same.
- Steve: Friday is more a general Data Expert Group workshop but we can of course bring Data Arch topics.
15th March 2023
- Welcomed new participant from Aicas.
- AMM
- Friday workshop
- Paul: who can make it?
- Christian: BMW (Christian/Adnan/Andre) has an appointment Friday. Need to have internal discussion about what we can do that day
- Piotr: Flying later, should be available in the morning.
- Ulf: Need to leave in the afternoon for my flight
- Steve: available all day
- Steve: That's useful as it shows that the Thursday morning workshop 9-11am will be important moment when we all there. Need to plan to use that time and Friday AM wisely and have an outline of what we are discussing.
- In-vehicle Data Stores
- Steve reports that he's restarted work adding further storage options as backend to WAII VISS server. Shows rough diagram to illustrate.
- Initial discussion of some possibilities that could enable
- Christian: the diagram shows data store holding VSS data. Wouldn't it be useful for other data models also to be stored?
- Steve: Completely agree. The box is a hold over from the Covesa Tech Stack in-vehicle diagram and should be amended. This is definitely about the arch supporting the various data models required.
- Piotr brings up some possibilities using a message queue as a data feeder northbound and southbound of data server
8th March 2023
Agenda:
- Roadmap creation
- AMM organisers need to finalise schedule
Minutes:
- AMM schedule
- Weds
- Paul: There will me a mgt level VSSo session (as opposed to down in the weeds of VSSo syntax) to be delivered by VSSo team
- Explain use case and ask for more..
- Thurs technical track
- Currently Data Architecture scheduled 9:00-11:00
- Technical intro
- ... <tbd>
- Steve: We need to decide an abstract for this session ASAP
- Friday workshop
- Discussion / updates
- Christian summarized some internal discussion. Defining other data domains. Personal data one example. Lessons learned from a few years ago (explained in earlier AMM presentation) starting with using VSS which moved to need for VSSo. He will be talking to Daniel Alverez. BMW may present something at AMM but not yet clear.
- Felix: Interest in data persistence in-vehicle and data models. They are interested in performant needs for query of automotive data, e.g. GPS data as an area filter.
- Steve: we have just released v2 of our OSS Whitebox SDK for our R-Car S4 SoC. Included is a Yocto recipe for the WAII VISS Server. The OSS nature of the SDK means it will be a natural integration point for me for our work.
1st March 2023
- Ulf outlines his plan to present forest of data model idea at AMM
- Christian: Internally we have been looking at Ontologies and generation of data models. So interested to discuss Ulf's forest of data model ideas.
- Piotr: Interested in interfacing southbound native data via messaging northbound to VSS.
- Moving forward..
- Steve: Need to keep eye on mid-term and the roadmap. What we do together. If there is not an appetite to document an architecture as a first step what are we doing instead? Work on viewpoints/touchpoints to find commonalities? Perhaps document some fragments or illustrations - I feel we there are some interesting discussions but we are not capturing.
- Ulf will document his "VISS" viewpoint over next couple of weeks.
- Christian will sketch some areas he is interested in. He will also discuss internally?
- AMM
- Intro presentation
- What
- Scope diagram
- Goals/motivations and some questions
- How
- Documenting Viewpoints/touchpoints
- Call to action on input/participation
- What sessions for Thursday? Want to encourage broard input but also need to keep eye on the roadmap
22nd February 2023
Agenda:
- Workshop debrief
- Do we have agreement on some high level points?
- Acceptance of the zone arch diagram from Miro
- Creation of Data Arch / Pattern
- Outline:
Document Goals, Motivations, Questions - Stephen, Andre, Christian, Adnan
Illustrate Design Patterns - Stephen, Andre, Christian, Adnan
POC Definition [ongoing] - Stephen, Andre, Christian, Adnan
- Arch / Pattern outline Qs
Minutes:
- Debrief
- General agreement around the high level arch diagram and results from the workshop.
- Ulf: Detail - Revisit arched line from computing to HU in top level diagram
- Ulf: More than one arch may be possible, e.g. data centric, interface centric (e.g. CCS centricity around VISS)
- Roadmap
- Steve outlined some possible approaches to describing a data architecture and associated patterns electronically.
- As discussed in the workshop moving from the general (Goals, Motivations, Questions) to specifics such as Design Patterns and Touchpoint illustrations.
- Andre will have internal discussion about this
- Goal remains to build out a work plan
15th February 2023
8th February 2023
Two themes:
- The general data centric data architecture pattern
- Motivations/goals
- Key questions to be answered
- Data domains:
- Autosar, Telematic, IVI, etc
- Data characterisation:
- More descriptive/prescriptive orientations of the pattern for specific use cases or touchpoints
- e.g. Android touch-point:
- Methods of signal abstraction via VHAL or VSS Data Server
- App connection via cloud and in-vehicle
- Workshop
- Agreement building
- Opening
- Motivations/goals
- Key questions to be answered
- Data domains:
- Roadmap
1st February 2023
- Useful long discussion on finding common ground. Overran by 30 mins.
- Recognition that a workshop soon would be useful to facilitate longer discussion.
- Working towards this 'central text' Design Pattern
- Data model and technology agnostic
- Capable of supporting multiple touch points from same arch (robust, flexible etc)
- Capable of multiple data domains sharing same data m/w
- Explain requirements but avoid deep heavy specification setting
- Vehicle is not a single IT blob. Need to engage with reality of requirements from different domains in outlining use cases and touch points: Hard RT vs needs of Analytics, signal read vs storage for processing.
25th January 2023
- Steve: Let's continue discussion of BMW presentation with goal of moving towards a roadmap
- Christian is waiting for permission to make presentation public
- Christian: Requirements are definitely an interesting collaboration point for us
- Felix: Alternative approaches to meet the requirements
- Ulf: I've forwarded the presentation within Ford waiting for feedback
- Christian: Slide 7
- Important point is the architecture supporting other data models and personal data (arch not be VSS specific)
- VSSo not just for data integration but enabling knowledge transfer
- Steve: to my mind we seem to be moving towards a central design pattern from which others and implementations may be spun off
- Proposal sketch: Design Pattern: Data Centric Arch including requirements
- Patterns for Touchpoints
- Implementations/PoCs proving such
- e.g Realm showing VSS sync via middleware
- General agreement this makes sense.
- Ulf: The implementation section matches what I am working on in Ford
- Next steps:
- Start to plan the central Design Pattern
18th January 2023
Agenda:
- BMW presenting input for the roadmap
- Adnan/Christian Muehlbauer from BMW presented their input into the early 2023 roadmap
- Presentation: 2023_01_18_COVESA_Data_Middleware_V2.0.pptx
- Initial discussion during the remaining time shows overlap in interests between participants. Needs expanding upon. Participants commit to returning next week to continue discussions.
- Shows in part the usefulness of the proposed longer workshops which might take some of the questions and requirements raised and provide the time for longer discussion.
11th January 2023
(Paul's notes from the meeting)
- Adnan - will bring in arch and use cases for next meeting including use cases
- Goal is to settle on piece that is useful and focus on a piece of the architecture
- Setting up direction for data architecture
- Setup high level arch
- Requirements - like objectbox.io
- List use cases
- In workshop provide hands on experience
- What is the initial purpose? What are we trying to
get?- Exploring solutions to data architecture
- Learning how things are built and Thought
leadership! - What solutions look like when provided requirements
- Learn how to shape with requirements then look at what solutions
- Community requirements
- Let vendors show their solutions
- Christian will provide first proposal
- Ted
- Be able to receive data from OEM - $0.60 a month - Curve logging
- Want to work on TCU’s directly
- Willing to work with us
- Adnan - How to improve data handling between car and cloud.
- Stephen - use 80% vss but need to be able to handle other data in addition
4th January 2023
Agenda:
- Continue roadmap discussion including possibility of virtual workshop
- Thoughts over xmas?
- Ulf has been considering his thin API proposal over xmas and will update in the next Autosar sync call. Has proposal to extend beyond data into procedure calls for services.
- Steve repeated his intention to resume work on extending state storage to support time series DBs including Apache IoTDB
- Discussion of connection between state storage, VSS servers and the proposed vehicle API
7th December 2022
- EG project announcements
- Update on Autosar Collaboration
- Xmas break
- Steve can only remotely phone in next week, which would be the last meeting before Xmas. Nic can't make it.
- Decision taken to break for Xmas early. This is the last meeting of the year. Thanks to all who contributed.
- Request: think about the Design Pattern goals over xmas
- Data Arch Theme discussion
- Proposal to hold a virtual workshop for multiple hours early next year to accelerate the design pattern discussion.
30th November 2022
- Project status
- Majority of VSS call taken up with VSS struct support discussion. Best practices good/bad one of the focus points next.
- Ted summarised VISS/VSSo updates
- Data Arch Theme discussion
- GraphQL
- Andre: Nice for query. Problematic to have generalised set method. Have to do heavy development in schema for setter. Complicates cross vehicle model support.
- GraphQL still of interest in the tech landscape but VISS is the more obvious first priority
- Discussion of existing high level in-vehicle data architecture diagram. What it was created for and its limitations.
- Ulf showed how his V2C proposal could address data needs
- (Data) Design Patterns
- Discussion of in-vehicle architecture leads back to the existing topic of local data reduction at the edge/within function and smart exchange between functions
- Local processing generating higher logic data possibly at lower frequency, e.g. engine management does local monitoring and then externally reports decisions
- Reduces in-vehicle network traffic
- Reduces costs associated with data transmission to the cloud
- This is an obvious candidate for a design pattern: benefits, how it may be done, VSS/VISS role in that.
23rd November 2022
- Project status
- VSS struct support heads up
- VSSo meetings have restarted
- Status of the collaboration between Covesa and Autosar and work on Autosar Gateway/Vehicle API
- Data Arch Themes
- Given prior input on peoples interest Steve outlined an early roadmap for work that could create both PoCs and data patterns and which is easily adopted into related Covesa projects such as the Autosar Gateway and Android.
- VSS Data Server
- GraphQL (Covesa C++)
- VISS Data Server
- Aim: demonstrating two possibilities
- VSS Data Store
- Connection to Data Server
- Embedded app DB (e.g. Realm), 'full' DB (e.g. IoTDB)
- Aim: demonstrating two possibilities
- Target: x86 (e.g container) => embedded h/w
- Discussion:
- Piotr: suggests adding VM to x86 targets as it allows investigation of concepts like VirtIO that may be harder in containers
- Ulf: It makes sense to follow a pluggable architecture
- Piotr makes the good point that the data architecture diagrams will need to evolve to cover higher function needs such as the proposed Vehicle API as well as the current coverage for data.
- AoB
- Ulf presented his proposal for a thin API approach to the Vehicle API requirements for the Autosar collaboration.
- This is not yet public. He will work on enabling that with goal to present it to the Autosar working group once it gets going.
16th November 2022
- Brief recap of Covesa project status
- Continuing discussion of future work
- Paul: what would people like to see?
- Felix: interested in GraphQL and connection of databases as data back-end. Along with what benefits data store may enable, e.g. processing at the edge.
- Nick: ability for apps devs to discover what signals are available and on what terms, e.g. frequency.
- Answered Felix's question about what work had gone on before.
- Future is very much open to be set by the group. Cloud needs picking up again - with VSS/VISS now established the original work on CCS could move to other aspects. In-vehicle the component landscape is sketched but much work to be done selecting patterns based on data needs.
- Steve showed a draft high level data arch diagram showing the proposed Autosar gateway communicating with QM data architecture.
- Agreement that the interest around data architecture presented and discussed at the AMM is a useful starting point both for PoC investigation and likely creation of data patterns. Let's continue the discussions with a view to creating a strategy.
9th November 2022
- Introductions as we have newcomers Felix Reichenbach (MongoDB) and Francois Ozog (Shokubai.tech)
- Project updates
- Steve summarises VSS, VSC, VSSo status and EG Leadership discussions around governance and project reporting
- Roadmap
- Discussion of possible data patterns. What are people interested in?
- Felix: One area is GraphQL data server shown in the architecture. Would be interesting to have embedded implementation backending to data store.
- Steve outlines some of the work to date on that including the GraphQL libraries in the Covesa github.
- Francois outlines his interests around function abstractions.
- Steve summarises work in that area in Covesa and suggests background reading.