Page History

Time

Day 2

TODO

09:15

Start

09:15-10:00

Summary of expectations

• Sachin: Which design to use (develop), or at least prioritize ?
• Alex: which directions should we go ?
  
  • external data server as a service,
  • SomeIP connection to framework layer and apps

Sachin: it looks promising in particular the integration of VSS, we need to select one approach and projectize it

Recap of yesterday's discussion

• approach 1: (data server app) is sitting somewhere, on a Linux partition or a different ECU, we have an App installed on Android, this App needs a web token to get the data
• approach 2: (architecture proposal via custom HAL), the app does not have any direct communication to the data service
• approach 3: SomeIP approach there is a SomeIP service somewhere and there is a generic SomeIP service in the framework
• Nadym: where is the bottleneck in approach 2 ?
• Alex: this has to do with the number of requests for data
• Sachin: we need to be compliant with the Android Auto architecture (not from a CTS standpoint, only from the code structuring standpoint), do we follow the principles of code organization set up by Google ?
• Alex: GAS App do not have any service inside the framework
• Alex: let us skip the SomeIP approach because we know how to do it with model transformation and code generation
• Alex: let us focus on the vehicle data server
• Gunnar: we all agree

AASIG Statement Of Work - DECISION
Vehicle Data Access Architectural Design & Implementation

• AA SIG will develop a PoC implementation of the External Data Server (priority one).
• AA SIG will develop a PoC implementation of the Data Server inside the Framework (so-called Internal Data Server) (priority two)
• AA SIG will develop a PoC implementation of the SomeIP stack inside the Framework (priority three)
• AASIG will develop a PoC implementation of the Google VHAL + OEM Extensions inside (priority four)

10:00-10:10

Break - meeting split - Vehicle HAL and Audio HAL tracks

10:10-12:15

Vehicle HAL / Vehicle Data Access Track

Work Breakdown Structure for the External Data Server Proof-Of-Concept

• The objective of this session is to develop an initial WBS for implementing the External Data Server.
• The TODOs listed below were added to the EA block-diagram, look at ExternalDataServerPoC-WBS.pptx
• After review and addition of a description of work and definition of done to each TODO, these TODOs will be entered into Jira.
  Vehicle HAL split meeting participants elaborated the WBS below
• TODO Develop an initial WBS for implementing an InternalDataServer (Priority 2) Proof-Of-Concept)

VSS feeder component

• todo finalize permissions layer concept (independent work item)
• todo create a layer concept for the Franca to VSS leaf mapping (model transformation)
• todo design and implement franca service (SomeIP)
• todo implement feeder as PoC
• todo check signal to service translation in Adaptive Autosar
• todo agree on PoC use cases for the implementation
• todo create PoC Someip simulation component to playback agreed use cases
  
  • note: we could consider using an Adaptive Autosar node like what we did for the FARACON demo
  • note: VSS data base could be merged into the VSS feeder
• todo select and implement VSS data storage (e.g. VISS, Geotab W3C PoC implementation)
  
  • look at W3C wiki page on VISS
  • https://at.projects.genivi.org/wiki/pages/viewpage.action?pageId=40403466 
  • look at melco: https://github.com/MEAE-GOT

VSS data server component

• todo APP implement authentication to access the data server (eg JWT), e.g. json Web token
  
  • relates to Vehicle Signal Authentication
• todo APP implement access for in-vehicle data, (e.g. App manifest permissions layer concept)
• todo APP implement request/response serialization
  
  • Gunnar: this relates to a communication protocol (so-called binary protocol)
• todo resolve requested data for the APP from the VSS data structure
• todo change/write data values for requested data leaves
  
  • look at https://github.com/w3c/automotive/issues/322
• todo handle the subscription for APPs
• todo write a generator which will handle permissions in the data server
  
  • look at on-going discussion in W3C on the dynamic registry https://www.w3.org/2019/11/W3C_Gen2_dynamic_registry.pdf (Geotab proposal)

Framework Layer / Authentication Service

• todo request APP permissions from package manager
• todo generate access token for the APP including the APP permissions

Application Layer

• todo Implement the APP permissions based on permissions defined/proposed in VSS layers
• todo implement access token request
• todo APP implement request/response serialization for the client
• todo implement the selected use cases

13:00-13:45

Lunch break

13:45-16:00

Vehicle HAL / Vehicle Data Access Track

Roadmap for the PoC(s)
Philippe : suggests the following roadmap for the PoC(s) implementation

• stage 1 Spring AMM (12-14 May)
• stage 2 Go to Google readiness check (early Q3 - July ?)
• stage 3 Fall tech summit (Q4 – October-November ?)
• stage 4 CES 2021 (January 2021)

Target platform for the PoCs

• VSS data server on laptop/linux with vsomeip component
• AOSP+application layer on Renesas R-Car H3 or NXP boards with AOSP P

External Data Server Concept – BOM (technology selection)

• Platform: Renesas R-Car H3 or NXP boards with AOSP P
  
  • 1- Android App in Application Layer
    
    • Android Apollo Plugin
  • 2- Authentication Service in Framework layer
    
    • Java Implementation
• Platform: Notebook with Linux / Docker / SomeIP (vsomeip)
  
  • 1- NodeJS
  • 2- Apollo GraphQL Server (Data Server)
    
    • Schema generated out of VSS
    • Permissions generated out of VSS
    • JWT authentification.
    • implement Resolvers
    • implement Mutations
    • implement Subscriptions
  • 3- Feeder

Use Cases for the PoCs

1. Battery status (high voltage)
2. Tire pressure
3. Air Conditioning

Work Breakdown Structure for the Internal Data Server Proof-Of-Concept

• rework of WBS for the External Data Server in order to adapt it to the internal data server architecture
• look at EA diagrams (TODO TBD to rework the EA diagrams, then add link to presentation) AASIG-F2F-4-5Feb-InternalDataServerPoC-WBS.pptx
  

Identify the deliverables / outcome

Deliverable #1 - Vehicle Data Access Architecture
type: technical brief
content:

• External Data Server concept, pros and cons, why this is the baseline concept
• Internal Data Server concept, pros and cons
• SomeIP inside the framework, pros & cons

Deliverable #2 - Vehicle Data Access Permission Management
type: technical brief
content:

• External Data Server concept, permission management baseline
• Internal Data Server concept: permission management as a variant of previous one
• SomeIP inside the framework concept: permission management (TBC)

Deliverable #3 - Vehicle Data Access Proof-Of-Concept Bills of Materials
type: technical brief
content:

• External Data Server concept - component & technology selection
• Internal Data Server concept - component & technology selection
• SomeIP inside the framework concept - component & technology selection

• Question: is there a component for which it would be worth writing formalized specifications ?

16:00-16:05

Vehicle HAL – recap for the Audio HAL participants

• Alex introduces the work breakdown structure for priority 1 PoC (external data server)

Vehicle HAL Track ends

10:10-12:15

Audio HAL Track

Agenda for this track follows the List of prioritarized topics for the Audio HAL

Introduction to Android Automotive Audio

• Bartosz (Tieto) introduces current audio system, including some various issues using this AA Audio-Source Management.pptx
• There must be a primary audio HAL this is typically used for car speakers other HALs exist (USB audio etc)
• !!! The default is that every output is mixed and heard.
• For automotive, added: Audio contexts. Provide additional info to HAL about the source,
• so that HAL impl can modify mixing rules etc.
• In automotive, it is not really expected to handle volume in the source side but rather in the HAL, or possibly external amplifier etc.
• Audio HAL can mix internal and external sources easily...
• Google interfaces exist, are defined. Vendors often provide some extensions, because there were not.
• Sachin: Native Android AOSP provides some implementation of the HAL though, right?
• Some parts of the implementation thus exist...
• Piotr: Yes, a kind of reference implementation, e.g. for emulator
• Android provides a HAL abstraction.
• Android can be updated without changing the HAL.
• Proper ALSA configuration should make it work.
• Reference implementation works with emulator, but does not appear to be a lot of work into making it production quality and flexible enough for production, etc.
• TinyALSA is optimized, unnecessary things removed...
• System Player - the visible player is a kind of skin/UI on the system player.
  
  • Including the codec support.

(Point 11) Source Management

• External source (like external Tuner chip) needs to be input to AHAL
• HwAudioSource configures some "triggers" .e.g to set up the correct mixing inside AudioFlinger
• The audio stream enters the HAL however.
• There are plans to improve volume control for external sources. These are TODO comments in code.

Audio Capture

• Discussing different ways (before and now) for listening for HotWord ("Hey Google", "Siri", "Alexa, ")
• Pre-Android 10
  
  • One active capture client at one time created limits.
  • HW SoundTrigger used.
• Android 10
  
  • AHAL now required to allow simultaneous activation of input streams
  • but still two audio apps cannot record at the same time (background apps are still connected but receive silence)
• Some privileged applications can always get the audio stream (e.g. for hotword recognition)
• Vendors can create privileged applications if they want.
• We think it is intended behavior to not allow applications to record (i.e. to make sure they receive silence)
• Possibly an app records the phone call in background.
• Piotr: Documentation usually mentions the new feature/change, but to understand the logic you have to read the code.

Android 10: Multi-Zone audio

• Application can be played in a zone
• A zone contains audio devices
• A zone has separate volume
• A zone can be requested by app
• Not yet any automatic mapping of app to zone (based on which display was used) (maybe it was fixed in updates of 10, or in later release)
• HW volume keys controls primary zone only
• Primary Zone is typically used for driver screen
• Secondary Zone typically rear-seat
• Volume to gain mapping curves
  
  • This exists in Android (phone) but not used in Automotive
  • The mapping is a problem because it only has min, max, and step but realistically you want non-linear gain.
  • Is it OK for Audio HAL to apply the non-linear system ?

Pre-android 10 AudioFocus

• not enforced (applications must respect the setup)
• only phone calls prioritized but not several levels of phone call types (for example)

Android 10: CarAudioFocus

• Internal interaction matrix (currently fixed )
• Support multi-zone audio (maintains focus per zone)
• Interaction Matrix – Reject, Exclusive, Concurrent
• Can the Caraudiofocus interaction matrix be modified ?
  
  • (Yes) Only by modifying the AOSP (system partition) – part of car service
  • System update would overwrite the change.
• Now: Will the matrix be customized for OEM production projects?
• Later: Will it become possible to configure for the vendor/OEM?
• Discussion: This changes behavior of the system, therefore it might be in Google's interest to make it as fixed as possible (according to the non-fragmentation / identical behavior direction Google prefers for Android).
• Configurability of the matrix is desired! Also, potentially more categories than currently.
• Radio support is relatively poor currently.
• There is a concept of global effects. No real way to control them.
• Control panel can be used to modify the effects (but this is a user feature).
• It is lacking system calibration features.
• TODO What is the shortlist of issues that must be solved? (Gap analysis)

(Point 7) Global Effect HAL

• Assuming that the other APIs are stable, and sources are connected to ALSA and working then we could define global effects.
• We should have (create) a HAL API to modify the global effects.
• See Architecture picture in "Global Effect HAL" AudioEffects.pptx and look at the following components

1. Global Effect HAL
2. Global Effect Service

• Both of these are proposed to be "GENIVI implemented" blue components (the meaning of this is only that there is a potential for a common implementation across many systems, and thus could be done once, as open source).
• Gunnar: if these can be implemented once for all systems, why are they part of HAL from Google perspective? (In other words, if it could be common, why would it not be part of System Partition / AOSP, a.k.a. Green code?)
  
  • Piotr: Yes, there are slight differences (e.g. DSP choice) but presumably some of the implementation could be common, and an abstraction to the system/DSP specific parts could be provided as part of this component.

More questions

• Question: Let's say you install Spotify, how can you integrate the desired behavior?
• Piotr: Simply, Spotify uses the primary audio device today.
• Driver can route output to a display or audio zone (e.g. to a particular headphone jack)
• TV use cases (video/audio mix). i.e. lipsync feature.
• Audio setup directly influences the user experience.

12:45-13:00

Audio HAL recap

• Sachin: very complex topic, too many open points
• Sachin: from the HAL perpective, we will get this from the vendor
• Sachin: Tieto made a proposal for standardization between audio HAL and audio services
• TV - Gunnar: we open up the topic but need to further discuss it
• audio is strongly related to user experience and therefore an important topic for OEMs

13:00-13:45

Lunch break

Multi-source management: Multi-source multi-sink

(Point 1) Networked Audio

• What's the concrete question here?
• Idea: Answer this question: "Is AVB supported well in Android?"
• investigate AVB for the purpose of learning how a network audio system would be designed
  
  • (because MOST is not so popular).
• A2B based networks are an option.
• What other technologies are similar and would drive us towards better understanding ?
• What is meant by device in the question asked to the F2F ? Device = speaker, or device inside Linux/Android system, a sound card or another ECU
• Hypothesis: ALSA interface is appropriate (enough*) abstraction for both the control question and the stream. If this is not true, someone provide counter-examples. Concrete example needed of either the architecture we want to solve, or the use case.
  
  • *However, things like the added latency of the ALSA driver in question might be required to know...

(Point 4) Audio data transfer

• Bandwidth issues?
• Piotr: You normally would push this over a socket of some sort. There is generally no bandwidth problem even pushing multiple PCM streams across a socket.
• (Summarized discussion)
• Shared memory? There should not be an issue to just use shared memory mechanism the Linux kernel provides to transfer data between processes
• However, if we are speaking of transfer between different hardware, or a virtualized system, it is not obvious but at the same time, solutions exist.
• To what level is this intended to be answered? Some aspects are clearly hardware / platform specific. Others ought to be "known" (i.e. the Linux kernel API and what is theoretically possible to do). To go deeper would require a concrete shared open-source implementation. Is this what the group expects?

(Point 5) Equalization

• Can be part of the bigger topic global effects (Point 7). Concrete proposal from Piotr to define the API of a HAL for global effects.
• "There is a need for a generic interface for controlling audio effects at HAL level, global effects are designed for input streams but control over them is limited by the available interface."

1. The user controlling the effects (setting surround sound etc.)
2. Globally applied system calibration is different (done by OEM/supplier at integration phase). This too needs more support. It must be very configurable.
   
   • (A system calibration file could be inserted in HAL level without crossing the HAL boundary (not needed in the system)).
   • For development (maybe even in production ?), the calibration data might need to be tweaked in real-time, access limited and not available to user.
   • It would be easier if it is part of HAL API for that reason.

• Both of these aspects should be considered when designing the system.
• All in all, the system needs a much more generic interface with more capabilities.
• Next step: Just do it....

(Point 9) Multiple audio channels

• "The challenge is to adapt the AA framework to HW I/O, e.g. there are 4 audio channels that need to be presented as 2 audio channels to AA"
• We don't know what exact issue this is referring to?
• Henric: It might have been an isolated thing we had to do in a previous project.
• There are 8 channels in Android.

(Point 15)

• (Virtualization, and splitting audio function between Android and "safety OS") may lead to scheduling challenges.
• Yes, this is known, but it is hard to say how to solve without a concrete case.
• Are there any features that might help, in general ?
  
  • Process Priority.
  • Kernel tweaks.
  • Fixing/rewriting kernel drivers.
  • Hypervisor specific issues...

(Point 14) Noted phone call priority above in the discussion we already had.

(Point 16) This is just a description of how things are. We can't really change it. (In some other points we mentioned places where limitations require changes in the framework anyway, even if you are not supposed to)

(Point 13) Combination of BT stream with other streams?

• We are not sure what the problem is exactly. Probably it is the general question about audio routing.
• Connect microphone and speaker to the BT input and output. Handsfree function.
• General problem with not good documentation for the Audio HAL (and lacking features). Only after the bus devices were introduced, then this was possible to solve. We set up device types using call_rx/tx but those come from the cellphone call audio, which is really a different use case.
• With dynamic routing enabled this is still a problem.
• Wassim: In addition there is audio processing involved including echo cancel, noise reduction.
• Bartosz: There are interfaces prepared for this.
• Some improvements in Android 10, e.g. a reference echo stream is handled.
• We suspect really advanced setups might not fit into this. (More than one microphone, more than one speaker stream that needs to be affect cancellation in different ways).

(Point 12) "Audio focus can be lost at any time."

• A bit improved in Android 10 with the interaction matrix
• Already covered before.

(Point 8) Latencies

• Agreed. Android has quite an inefficient audio stack and latencies are an issue.
• AAudio - https://developer.android.com/ndk/guides/audio/aaudio/aaudio
  
  • this is made to rewrite and improve performance. A kind of direct API towards ALSA (for system programming, but could be used by Applications through NDK) "Designed specially for applications that require low latency"

Concrete development plan

Global effects

1. Implement a simple proof of concept
2. Make a list of all possible effects
3. Design the Global Effects API using experience from 1/2. (code first)

• A first simple effect is filter/equalizer
• Proves ability to manipulate the sound in real time.
• It exists but no simple way to apply it globally. Effects are provided but each application sets their own effect settings.

Multi-zone connected to displays

• Not yet connected displays
• Implement the possibility to change the audio zone system-wide
  
  • (e.g. force an application's sound to a particular zone)
    1. Implement a simple proof of concept
    2. Need to track carefully the changes Android will make here because there is a plan to connect zones and displays...

16:00-16:15

recap (Gunnar)

• 2 ideas of concrete projects came up
  
  • General Audio Effects service interface
  • rerouting audio streams

Audio HAL Track ends

End of Day 2