List of prioritatized topics for the Audio HAL

F2F meeting organization (participants, logistics, agenda)

Next Meeting - Wednesday 5 February - 8:30 CET - F2F Meeting Day 2 - Audio HAL track

Zoom dial-in (possibly updated for the F2F meeting) telco details will be sent on the aasig mailing list

Zoom dial-in

https://zoom.us/j/470113570

Dial-in by your location

Meeting ID: 470113570

Agenda

Agenda items (to be prioritarized at the beginning of the call)

• review of the agenda of upcoming F2F Audio HAL part
• AOB

23 January 2020

Participants

• Andrey, Piotr, Wassim, Henric, Philippe

Minutes

upcoming F2F meeting

• agenda is here, Audio HAL part is scheduled on Day 2 morning
• review of high priority topics selected for discussion at the F2F: Common HAL & Source Management
  • assignment of preparation work done (although one is TBC)

AOB

• CES debriefing: GENIVI showcase at CES went out well, GENIVI hosted 1400 people, feedback about the event was vry positive
• AMM: the Spring AMM will be scheduled in May likely, dates and location should be announced soon

19 December 2019

Participants

• Andrey, Suhasini, Piotr, Stephen Lawrence, Gunnar, Philippe

Minutes

upcoming F2F meeting for the Vehicle HAL project

• decision on the dates was made => 4-5 February, at BMW, Munich, Germany
• Philippe: we have an opportunity to coschedule (and colocate) a meeting for the Audio HAL project
• /TODO/ all fill in the participation table and indicate whether you would be available and interested by joining a meeting either F2F or via telco

Review of the list of prioritarized topics (continuation)

• 13- BT handsfree

  • Suhasini: it is rather a question of multisource management

  • Andrey: would agree it is related to multisource management

  • Gunnar: on the mobile phone, the BT role is the server role, the car is more the handsfree role

  • Andrey: we need to clarify the automotive use cases

  • the group should work on use cases but there are more fundamental questions to be solved on multi source management

• 14- Android does not implement all features required by customer.

  • Andrey: this is about multisource management, support of external devices, limited number of priorities, etc. the various bullet points can be distributed over the table

  • /TODO/ Andrey distribute the bullet points among the table

  • Piotr: what are the priorities you are talking about ?

  • Andrey: these are the priorities related to phone calls, I will clarify this when I do the split (phone calls vs. not phone calls)

• 15- Android Audio subsystem is developed only for infotainment purposes. Safety-related features need to be implemented in another RTOS

  • Andrey: thinks the safety-related audio features are not in the scope of this group

  • Piotr: we need to keep in mind that there is a higher priority audio functionnality in the system

  • Gunnar: it might be worth discussing how AA can use external components that manage the higher priority audio functionnality, these are system-level considerations

  • /TODO/ Gunnar add HERE what was discussed in this call about the system level architecture Thanks

  • Gunnar: do we look for solutions on AA side first or do we look also at the general system design for audio ? my recommendation is to start something concrete at the design level

• 16- Android way of extending its functionality is developing vendor extensions without modification of the framework

  • Piotr: we need to avoid changing the framework because there is a lot of manual work necessary to track the changes that need to be taken into account

  • Andrey: agreed

  • Andrey: we have a mechanism in Harman to follow up the changes in AOSP

  • Gunnar: we need to start the design work asap and as a general principle we do not change the framework

• 17- Google is still very “hand wavy” about solution

  • Gunnar: WR has a lot of work to implement the audio HAL by themselves

  • Piotr: people look at the code of android directly, I would like to get some tooling to cope with the fact the doc will not be provided by Google anyway

  • Piotr: tool can take care of the configuration setting, helping with work we do currently manually

  • Andrey: we need to adapt to the Google way, we cannot change this

  • Piotr: we can translate HIDL to Franca and provide a more flexible way of implementing things

  • Andrey: we are considering similar ideas within Harman

  • Gunnar: agreed, it is part of the discussion on how to use a common interface in multiple systems (with the remote interfaces, not only on AA), something like CommonAPI, there is a generic discussion to have there

• 18- Genivi ?

  • Gunnar: in my opinion this is a question about the role of GENIVI, we need to think about what could be the outcome of the group, as I said before, the question is rather to find solutions, it can be anything from a formal specification, to guidelines and even code

  • category changed to "Deliverables"
  • Philippe: IMHO this is also a question about the recycling of earlier work done by GENIVI, e.g. the audiio manager, we said we would organize a presentation of the GENIVI audio manager and the automotive use cases it supports
  • /TODO/ Gunnar plan a presentation of the GENIVI audio manager in Q1, 2020

list of prioritarized topics

• /TODO/ all review the minutes of 12 & 19 December together with the list of prioritarized topics and identify any gap / missing points in the list

Next call

• Thursday 23 January, 15:00 CET / 20:30 India / 6am US Pacific
• agenda
  • review comments for the list of prioritarized topics

12 December 2019

Participants

• Andrey, Suhasini, Piotr, Jimhyuk Jung (mobis), Patrick Carlisle (mobis), Stephen Lawrence, Giovanni, Gunnar, Philippe
• apologies: Wassim

Minutes

Roundtable

• everyone delivers a quick personal introduction
• Jimhyuk works for mobis technical center Europe (MTCE), he is an IVI expert
• Patrick works for mobis technical center America (MTCA)
• Piotr indicates he would like to have a less hardware aware Android

Review of the list of prioritarized topics

• list upated online, each contributor explains the rationale for setting the priority he chose
• 1- Networked Audio Devices: Support for the configuration of networked audio devices. Currently Android can view it only as a single device/sound card. An audio network in car might have multiple devices that needs to be controlled by the Android OS
• 2- Overall configuration management
  • Piotr: there is no way to automatically validate the configuration
  • Andrey: IMHO there is more important things to solve than the configuration
• 3- Common Audio HAL

  • Piotr: audio HAL is not so HW dependent because of tinyalsa, the porting the common audion HAL should be limited to tinyalsa

  • Andrey: I do not think whether we can make a common audio HAL, it would be great to have such a common HAL for using eAVB ?

  • Piotr: some hw vendors are working on it, must be feasible

  • Gunnar: this is about finding the common part, this is about avoiding duplicated effort to provide the same functionalities, this about reducing the amount of effort for each vendor to implement the HAL

• 4- Audio Data Transfer

  • Suhasini (not captured)

  • Piotr; utilizing a DSP is very hw specific, this is why I put it a low priority, however it is important, it should be high priority

  • Gunnar: this is about finding the right abstraction

  • Piotr: we need to talk about interfaces / abstraction to create audio effects

• 5- Equalization
  • all agree it is high priority to have a way to control the equalization
• 6- Audio calibration
  • Suhasini; audio calibration sounds to me like controlling global effects, not as big a problem as others
  • Piotr: agreed, could be covered by the same interfaces, there is a need for the a generic interface
• 7- Controlling audio effects
  • the above 3 items could be covered by the same interfaces

• 8- Latencies

  • Suhasini: for Analog Devices, latencies are important because we have a lot of time sensitive applications, currently we are doing these on the DSP, we would be interested in moving some of them on the Android system

  • Piotr: IMHO the latencies question is pretty hw specific, it cannot be solved in a general way, this is rather Google responsibility with their framework

  • Piotr: another problem is shared audio on a virtualized platform, not sure we can solve this problem in a joint effort

  • Gunnar: in the GENIVI Hypervisors (HV) project, Opensynergy is behind the proposal for a set of standard interfaces for the audio platform, we could dig into it if there is an interest in the Audio HAL group

  • /TODO/ Gunnar organize a presentation of the work done by the HV project on the standard interfaces for the audio platform

  • Andrey: we encountered a problem with latencies when we switched to hypervisord, actually we had problems with latencies AND scheduling

  • Gunnar: it is a difficult challenge, agreed

• 9- Multiple audio channels
  • Bosch indicated their challenge is to adapt the AA framework to their HW I/O, e.g. they have 4 audio channels while they need to present them as 2 audio channels to AA

  • Piotr: it is hard for me to judge because it is very specific, my claim is that it can be solved with AA

  • Andrey: I do not know it was feasible

  • Philippe: this needs further investigation

• 10- Early audio for RVC

  • Andrey: early audio RVC will be rather handled by the HV or RTOS, because there are safety requirements which cannot be achieved by AA

  • Suhasini: what is the audio application for RVC ?

  • Andrey: (not captured)

  • Stephen: there might be one opportunity for standardization, some people manage the early audio on RTOS and then switch over to AA framework. Can the methods on Android side by standardised? This also applies to other 'early' features such as video for reversing camera.

  • Piotr: one possibility is to have 2 audio HALs, one "simple" audio HAL for early audio  and one full feature audio HAL

  • Philippe: iy would be good to look at a standardized way of switching modes

  • Andrey: we still have the safety requirements for this

• 11- Source Management
  • Multi‐source multi‐sink, etc.
  • Piotr: this is in the framework, it maybe not be worth looking at it as first priority, this is why I put a low
• 12- Audio focus
• Andrey: it would be great to get applications on AA to request audio focus (not optional audio focus), usually automotive use cases have a limited

  number of sound applications

• Patrick: at Mobis we have our own sound manager and pass it to AA, adding a little man between the sound applications and AA works well

• Patrick : I do not know much effort we put in this tool though

• Gunnar:  IMHO the concepts supported by the GENIVI audio manager component  might be relevant for this issue

• /TODO/ Gunnar contact the "right" person (i.e. who knows the GENIVI audio manager component) and organize a presentation of the concepts supported

• review of other topics will continue next week

F2F opportunity in early Q1/2020

• Philippe: the Vehicle HAL project intends to have a F2F meeting in early Q1/2020, BMW will host the meeting in Munich, Germany
  • look at Vehicle HAL F2F - 4-5 February 2020 - Organization
• Philippe: it might be an opportunity for the Audio HAL project to meet F2F at the same date & location
• /TODO/ all check about your availabilities and the clearance of travel expenses

Next call

• it will be scheduled on Thu 19 December at 11:30am CET (usual time)

4 December 2019

Participants

• Andrey, Suhasini, Wassim, Stefan, Maria, Pontus, Gunnar, Philippe

Minutes

Introduction

• Philippe reminds all participants to identify additional extensions to Audio HAL that would not change too many things and identify a part of the Audio HAL that could common for all SoC vendors
  • this analysis needs more work and consensus reaching within each participant's organization
• Suhasini asks whether the list of topics is available in the wiki
• /TODO/ Philippe merge all inputs from minutes and slide decks into a single wiki page DONE
  • page is List of prioritarized topics for the Audio HAL
  • tentative categories have been added: configuration, common audio HAL, performance, multi-source management, multi-OS management, extensions. All categories are To Be Confirmed (TBC) and the mapping of categories to topics are all TBC.
  • /TODO/ all participants review the list of topics, categories, mapping and add priority (high, medium, low)

Configuration & Common Audio HAL

• Piotr shows the slide deck he presented at the tech summit in order to exemplify what our workplan should consist of

Next call

• Thursday 12 December at 16:00 CET / 7am PST (unusual time)

F2F meeting opportunity

• Vehicle HAL project will meet F2F at the end of January / beginning of February, this is an opportunity to colocate a F2F meeting for the Audio HAL project
• please look at Vehicle HAL F2F - 4-5 February 2020 - Organization and fill it in (you may have to ask for credentials)

21 November 2019 - Project Scoping  - Second Call

Participants

• Andrey (Harman) Sw architect
• Suhasini (Analog Devices) SW engineer for the car infotainment
• Wassim (BMW) Audio team SW architect, works on cross-ECUs function, audio manager, specific implementation, needs to clarify his position w.r.t. the maintenance role for GENIVI audio manager
• Stefan (Bosch) component maintainer for Audio at Bosch Car Multimedia, Hisldesheim, Germany
• H. Carlsson (Bosch) Bosch Car Multimedia, Lund, Sweden
• Bartosz + Piotr (Tieto)
• Gunnar
• Philippe

Minutes

Introduction

• Philippe welcomes participants and details the agenda

• Philippe recommends the review of the presentations delivered by Tieto and Windriver at the tech summit in Detroit where they presented their return of experience on Android Automotive, in particular on Audio HAL

  • please look here, search for "return of experience"

Introduction to Audio HAL

• Bartosz presents this slide deck
• We have more than 1 Audio HAL module in the system. There is always primary audio HAL that handles primary output (car speakers) and primary input (mic, aux, tuner, etc.) and in Android Automotive usually we also have USB audio HAL (dedicated to USB audio) and A2DP audio HAL (dedicated to Bluetooth A2DP). Different Audio HAL modules don't talk directly to each other, but the routing/bridging between them is handled via AudioFlinger/AudioPolicy based on audio policy configuration (where routing topology is defined).
• slide 6: grey boxes are external to AA
• Wassim: question on slide 14 about configuration
• Bartosz: configuration files are in the system partition or in the vendor partition, we lack a good concept for configuration, and for the persistence of configuration as well, support tools would greatly help

Gathering of list of topics for the Audio HAL project

• Bosch: we have issues with the BT audio stream / BT handsfree but I am just starting digging into it
• Gunnar: I am surprised that the BT handsfree is not part of the Google AA code
• Bosch: there is some support in the framework to support handsfree
• Gunnar: is the basic hansfree profile implementation part of the BT support in AA ?
• Bosch: pairing the phone with the device works but there is a need to connect the audio
• Andrey: shows a list of questions / problems, look at the list below
  
  • Android does not implement all features required by customer.
    • Audio focus can be lost at any time
    • Limited number of sources
    • Limited number of priorities -2
    • No easy way to support external amplifiers
    • No priorities of phone call types.
  • Android Audio subsystem is developed only for infotainment purposes. Safety-related features need to be implemented in another RTOS
    • Need to share the same hardware between 2 OSes
    • Running Android as a virtual machine inside RTOS leads to problems with scheduling of audio processes
  • Android way of extending its functionality is developing vendor extensions without modification of the framework.
    • But in some cases you need to modify framework. There is a process for this, but slow. And changes needed to one customer / developer is not needed for another. Even if you do this, you have to wait until Tier 1 merges this change to its drop
    • You have to retain compatibility for third-party apps
• Gunnar: what are the most important topics to tackle on ? maybe there is a particular problem to look at first
• Wassim: there is some topics that are low-hanging fruit
• Stefan: IMHO some of the topics are not related to Audio HAL but rather system-level architectural issues
• Gunnar: Audio HAL is much more complex than the Vehicle HAL
• Gunnar: in the Vehicle HAL we are finding solutions that can sit aside from AA, we do not intend to make a fork which would provoke a lot of
  integration problems
• Stefan: priority of sources varies among OEMs, we would need a better configuration capability from Google to handle this better
• Bartorz: extending the framework is fine provided you do not change the APIs, the HAL is actually something where we can put additional stuff "for free"
• Bartoz: this is a much bigger challenge to work out a common Audio Hal and port it to different SoCs but it is interesting to do
• Gunnar: agreed
• /TODO/ all participants identify additional extensions to Audio HAL that would not change too many things
• /TODO/ all participants identify a part of the Audio HAL that could common for all SoC vendors
• /TODO/ Gunnar and Philippe create the wiki page to gather the priorities of topics of interest listed so far  (from minutes of first meeting and presentations delivered by Tieto and Windriver at the tech summit)

Adjourned: 18:20 CET

7 November 2019 - Project Scoping  - First Call

Participants

• Andrey (Harman) principal architect, Android expert, based in Nijni-Novgorod, Russia
• Denis (Harman) manager of Android COE team (Center Of Excellence)
• Alexander (Harman) workis in the COE for low level Android
• Suhasini (Analog Devices), automotive infotainment software engineer, based in Bangalore, India
• Piotr + 1 (Tieto) mobile devices, automotive HW layer of Android, based in Poland
• Maria + 2 (Bosch Car Multimedia) manager and two SW engineers
• Gunnar (GENIVI)
• Philippe (GENIVI)

Minutes

• Philippe reminds the objective of the call (or of the first series of calls rather) which is to define the scope of a Audio HAL project
• Suhasini: we have not much experience with Android Automotive (AA), we are interested in AA because AA is now in the head unit
  • we have a portfolio of audio processors and an automotive audio bus to distribute audio within the car, we want to configure our proprietary audio bus SW stack to use it with AA as we did for Linux
  • we want to determine whether we have a problem of bandwith when using our proprietary network with AA, one approach is to use shared memory transfer
    in the audio HAL
• Piotr: we identified the following list of problems / tasks with AA when integrating the HAL for silicon vendors and OEMs
  • configuration of Automotive overlay, there is not much tool to configure the AA system
  • there is no way to control the equalization, i.e. no simple way to control global effects for output streams (by default Android application controls its own tracks)
  • there is no Audio calibration interface
  • there is a need for a generic interface for controlling audio effects on HAL level, global effects are designed for input streams but control over them is limited by interface
  • early audio for RVC (Rear View Camera) or other services
  • configuration management component for TinyALSA
  • Audio Focus doesn't forbid to interrupt Audio, Android 10 provides additional interface for Automotive to solve this problem
• Suhasini: we would like the following topics to be investigated
  • Audio data transfer / streaming to a co-processor for post processing of audio from the HAL/other Android layers
  • Support for the configuration of networked audio devices. Currently Android can view it only as a single device/sound card. An audio network in car might have multiple devices that needs to be controlled by the Android OS.
• Bosch engineer: we are working on two things, with the audio HAL we received from our silicon vendor
  • 1- patches for handsfree management, combination of BT stream with other streams, how to enable the correct audio routing, documentation on the Audio HALis missing, it is difficult to find examples on line
  • 2- I/O for our Bosch HW, the challenge is to adapt the AA framework to our HW, which has for instance 4 audio channels while we need to present them as 2 audio channels to AA
• Gunnar: how many of you are familiar with the Audio Manager of GENIVI ? I would recommend you to look at the features supported by GENIVI Audio Manager because it has been designed for automotive audio specifically
• Gunnar: introduces shortly the Audio Manager daemon and plug-ins
• Harman engineer: using Audio Manager am implementation designed for linux would raise an issue because of the Google CTS
• Andrey: safety related audio features can likely not be implementted in AA because AA is not a real-time OS

Next steps

• Gunnar: @Harman - would you consider making a list similar to what Tieto presented ?
• Suhasini: having an overview of the AA audio HAL would help ? can someone give such a presentation ?
• /TODO/ Harman & Bosch provide a list of topics that could be worked on jointly
• /TODO/ Philippe create a wiki page DONE
• /TODO/ Piotr prepare a short presentation on the AA audio HAL so that we share the same understanding of Audio HAL features

Next call

• we will have calls every other week, on Thursdays at 11:30 CET
• next call is scheduled on Thursday 21 November, a calendar invite has been sent