The minutes of the meetings for the Data Architecture and Infrastructure pillar of the Data Expert Group.

Historical note: the minutes for the previous project, the CVII Tech Stack, can be found here: CVII Technology Stack meeting notes


5th March 2025

Agenda:


Minutes:

26th February 2025

Meeting cancelled

19th February 2025

Agenda:

Minutes:

Hi @Christian Mühlbauer re the discussion in last week's call about docker modularity you might find these links useful in the Docker doc:


Rough example of one possible approach using extend:
We currently have docker/docker-compose-cdsp.yml as a reference compose for a single logical instance of CDSP-core. Assume the Knowledge and Information Layer services have been added. The question is how to use/re-use it in more expansive use-case examples?

Lets say we want to create a use-case blueprint in the examples folder that illustrates a larger touchpoint involving in-vehicle, mobile and cloud. For this example we can create a compose that uses extend to create services in those three domains using the base service defined in docker-compose-cdsp.yml. It might be a simple duplicate and rename, or it might in addition modify the service.

To illustrate let's take the first service in CDSP-core iotdb-service and create three instances in our new blueprint compose example/vehicle-cloud-mobile/docker-compose-cdsp-vehicle-cloud.yml:

name: cdsp-vehicle-cloud-mobile

services:

# In-vehicle HPC Apache IoTDB
  vehicle-iotdb-service:
    extends:
      file: docker-compose-cdsp.yml
      service: iotdb-service

# Cloud Apache IoTDB
  cloud-iotdb-service:
    extends:
      file: docker-compose-cdsp.yml
      service: iotdb-service
    build:
      # Use cluster environment

# Mobile Apache IoTDB
  mobile-iotdb-service:
    extends:
      file: docker-compose-cdsp.yml
      service: iotdb-service


Now for our blueprint we have three services vehicle-iotdb-service, cloud-iotdb-service and mobile-iotdb-service all based on the service definition in docker-compose-cdsp.yml.

We can optimise further by configuring each service appropriately for their domain. Using cluster in the cloud for example.

I don't think there is one pattern that will fit all use cases. It might make sense to use merge or include rather than extend in some cases. We should also be wary of making things so complex that a high level of docker knowledge is required. Ideally these should be useable to a wide range of end users not just infrastructure experts. Some good compromises as to how to modularise should emerge as we go.

Anyway hope that helps.

19th February 2025

Agenda:

Minutes:

Hi @Christian Mühlbauer re the discussion in last week's call about docker modularity you might find these links useful in the Docker doc:


Rough example of one possible approach using extend:
We currently have docker/docker-compose-cdsp.yml as a reference compose for a single logical instance of CDSP-core. Assume the Knowledge and Information Layer services have been added. The question is how to use/re-use it in more expansive use-case examples?

Lets say we want to create a use-case blueprint in the examples folder that illustrates a larger touchpoint involving in-vehicle, mobile and cloud. For this example we can create a compose that uses extend to create services in those three domains using the base service defined in docker-compose-cdsp.yml. It might be a simple duplicate and rename, or it might in addition modify the service.

To illustrate let's take the first service in CDSP-core iotdb-service and create three instances in our new blueprint compose example/vehicle-cloud-mobile/docker-compose-cdsp-vehicle-cloud.yml:

name: cdsp-vehicle-cloud-mobile

services:

# In-vehicle HPC Apache IoTDB
  vehicle-iotdb-service:
    extends:
      file: docker-compose-cdsp.yml
      service: iotdb-service

# Cloud Apache IoTDB
  cloud-iotdb-service:
    extends:
      file: docker-compose-cdsp.yml
      service: iotdb-service
    build:
      # Use cluster environment

# Mobile Apache IoTDB
  mobile-iotdb-service:
    extends:
      file: docker-compose-cdsp.yml
      service: iotdb-service


Now for our blueprint we have three services vehicle-iotdb-service, cloud-iotdb-service and mobile-iotdb-service all based on the service definition in docker-compose-cdsp.yml.

We can optimise further by configuring each service appropriately for their domain. Using cluster in the cloud for example.

I don't think there is one pattern that will fit all use cases. It might make sense to use merge or include rather than extend in some cases. We should also be wary of making things so complex that a high level of docker knowledge is required. Ideally these should be useable to a wide range of end users not just infrastructure experts. Some good compromises as to how to modularise should emerge as we go.

Anyway hope that helps.

12th February 2025

Agenda:

Minutes:

5th February 2025

Agenda:

Minutes:

29th January 2025

Agenda:

Minutes:

22nd January 2025

Agenda:

Minutes:

15th January 2025

Agenda:

Minutes:

8th January 2025

Minutes:

11th December 2024

Agenda:

Minutes:

4th December 2024

Agenda:

Minutes:


27th November 2024

Agenda:

Minutes:

20th November 2024

Agenda:


Minutes:

13th November 2024

Agenda:

Minutes:

6th November 2024

Minutes:

30th October 2024

Agenda:

Minutes:

23rd October 2024

Meeting cancelled due to holidays

16th October 2024

Meeting cancelled due to holidays

9th October 2024

Agenda:

Minutes:

Minutes:

2nd October 2024

Agenda:

Minutes:

25th September 2024

18th September 2024

Agenda:

Minutes:

11th September 2024

Agenda:


Minutes:

4th September 2024

Agenda:

Minutes:


28th August 2024

Agenda:


Minutes:

21st August 2024

No meeting due to holidays.

14th August 2024

Agenda:

Minutes:


7th August 2024

Agenda:

Minutes:

31st July 2024

Agenda:


Minutes:

24th July 2024

Agenda:


Minutes:

17th July 2024

Agenda:


Minutes:

10th July 2024

Agenda:


Minutes:


3rd July 2024

Agenda:

Minutes:

26th June 2024

Agenda:


Minutes:

19th June 2024

Agenda:


Minutes:

12th June 2024

Agenda:

5th June 2024

Agenda:

Minutes:

29th May 2024

Agenda:


Minutes:

21st May 2024

Agenda:

Minutes:

15th May 2024

Agenda:

8th May 2024

Agenda:


Minutes:

1st May 2024

Minutes:

24th April 2024

Agenda:


Minutes:

17th April 2024

No meeting due to AMM

10th April 2024

Agenda:


Minutes:

3rd April 2024

Agenda:


Minutes:

27th March 2024

Agenda:

Minutes (retrospective summary):

20th March 2024

Agenda:

Minutes:

13th March 2024

Agenda:

Minutes:

6th March 2024

Agenda:


Minutes:

28th February 2024

Agenda:

Minutes:

21st February 2024

Agenda:

  1. Spring AMM sessions
  2. Central Data Service Playground
    1. Status reports
    2. Upstreaming Apache IoTDB support into WAII VISS Data Server
    3. Continue discussion of Knowledge Layer Connector. One related aspect is the topic of schemas on the data layer side.
    4. Online documentation
    5. Roadmap/backlog/work package definition. What does AMM release look like?
    6. Feeders incl WAII, Volvo/Remotive
  3. AoB


Minutes:

14th February 2024

Agenda:

  1. Spring AMM sessions
  2. Central Data Service Playground
    1. Status
    2. Knowledge layer connector
  3. AoB


Minutes:


7th February 2024

Agenda:

  1. Spring AMM sessions
  2. Central Data Service Playground
    1. Based on discussion with Christian Muehlbauer kick off discussion of Connector in the Playground between the data and knowledge layers
    2. Ulf has sent a PR to add the Hugo online doc: Any discussion needed related to that, e.g. workflow, review
    3. Database connectors: status, handling WAII changes
    4. Feeders (time allowing)
  3. AoB

Minutes:

31st January 2024

Agenda:

  1. WAII update
  2. Spring AMM sessions
  3. Central Data Service Playground

Minutes:

24th January 2024

Agenda:

  1. Spring AMM sessions
  2. Central Data Service Playground
  3. AoB
    1. (Ulf): WAII update


Minutes:


17th January 2024

Minutes:

10th January 2024

Agenda:

Minutes:

3rd January 2024

13th December 2023

Agenda:

Minutes:

6th December 2023

Agenda:

Minutes:

29th November 2023

Agenda:


Minutes:

22nd November 2023

Agenda:


Minutes:

15th November 2023

Agenda:


Minutes:

8th November 2023

Minutes:

1st November 2023

Minutes:

25th October 2023

Agenda:

Minutes:

18th October 2023

Agenda:

Minutes:

12th October 2023 - AMM Data Architecture Workshop

4th October 2023

27th September 2023

20th September 2023

13th September 2023

6th September 2023

30th August 2023

23rd August 2023

10th August 2023

2nd August 2023

26th July 2023

19th July 2023

12th July 2023

Agenda:

5th July 2023

28th June 2023

21st June 2023

14th June 2023

7th June 2023

31st May 2023          

24th May 2023

17th May 2023

10th May 2023

3rd May 2023

26th April 2023

19th April 2023

12th April 2023

5th April 2023

22nd March 2023

Agenda:


15th March 2023

8th March 2023

Agenda:


Minutes:

1st March 2023

22nd February 2023

Agenda:

Minutes:

15th February 2023

8th February 2023

1st February 2023

25th January 2023

18th January 2023

Agenda:


11th January 2023

(Paul's notes from the meeting)

4th January 2023

Agenda:


7th December 2022

30th November 2022

23rd November 2022

16th November 2022

9th November 2022