List of prioritatized topics for the Audio HAL

Next Meeting - Thursday 19 December - 11:30 CET

Zoom dial-in

https://zoom.us/j/470113570

Dial-in by your location

Meeting ID: 470113570

Agenda

Agenda items (to be prioritarized at the beginning of the call)

• review of the priorities of topics of interest for the Audio HAL project (continuation)
• discussion on additional extensions to Audio HAL that "would not change too many things"
• discussion on the part of the Audio HAL that could common for all SoC vendors
• AOB

11 December 2019

Participants

• Andrey, Suhasini, Piotr, Jimhyuk Jung (mobis), Patrick Carlisle (mobis), Stephen Lawrence, Giovanni, Gunnar, Philippe
• apologies: Wassim

Minutes

Roundtable

• everyone delivers a quick personal introduction
• Jimhyuk works for mobis technical center Europe (MTCE), he is an IVI expert
• Patrick works for mobis technical center America (MTCA)
• Piotr indicates he would like to have a less hardware aware Android

Review of the list of prioritarized topics

• list upated online, each contributor explains the rationale for setting the priority he chose
• 1- Networked Audio Devices: Support for the configuration of networked audio devices. Currently Android can view it only as a single device/sound card. An audio network in car might have multiple devices that needs to be controlled by the Android OS
• 2- Overall configuration management
  • Piotr: there is no way to automatically validate the configuration
  • Andrey: IMHO there is more important things to solve than the configuration
• 3- Common Audio HAL

  • Piotr: audio HAL is not so HW dependent because of tinyalsa, the porting the common audion HAL should be limited to tinyalsa

  • Andrey: I do not think whether we can make a common audio HAL, it would be great to have such a common HAL for using eAVB ?

  • Piotr: some hw vendors are working on it, must be feasible

  • Gunnar: this is about finding the common part, this is about avoiding duplicated effort to provide the same functionalities, this about reducing the amount of effort for each vendor to implement the HAL

• 4- Audio Data Transfer

  • Suhasini (not captured)

  • Piotr; utilizing a DSP is very hw specific, this is why I put it a low priority, however it is important, it should be high priority

  • Gunnar: this is about finding the right abstraction

  • Piotr: we need to talk about interfaces / abstraction to create audio effects

• 5- Equalization
  • all agree it is high priority to have a way to control the equalization
• 6- Audio calibration
  • Suhasini; audio calibration sounds to me like controlling global effects, not as big a problem as others
  • Piotr: agreed, could be covered by the same interfaces, there is a need for the a generic interface
• 7- Controlling audio effects
  • the above 3 items could be covered by the same interfaces

• 8- Latencies

  • Suhasini: for Analog Devices, latencies are important because we have a lot of time sensitive applications, currently we are doing these on the DSP, we would be interested in moving some of them on the Android system

  • Piotr: IMHO the latencies question is pretty hw specific, it cannot be solved in a general way, this is rather Google responsibility with their framework

  • Piotr: another problem is shared audio on a virtualized platform, not sure we can solve this problem in a joint effort

  • Gunnar: in the GENIVI Hypervisors (HV) project, Opensynergy is behind the proposal for a set of standard interfaces for the audio platform, we could dig into it if there is an interest in the Audio HAL group

  • /TODO/ Gunnar organize a presentation of the work done by the HV project on the standard interfaces for the audio platform

  • Andrey: we encountered a problem with latencies when we switched to hypervisord, actually we had problems with latencies AND scheduling

  • Gunnar: it is a difficult challenge, agreed

• 9- Multiple audio channels
  • Bosch indicated their challenge is to adapt the AA framework to their HW I/O, e.g. they have 4 audio channels while they need to present them as 2 audio channels to AA

  • Piotr: it is hard for me to judge because it is very specific, my claim is that it can be solved with AA

  • Andrey: I do not know it was feasible

  • Philippe: this needs further investigation

• 10- Early audio for RVC

  • Andrey: early audio RVC will be rather handled by the HV or RTOS, because there are safety requirements which cannot be achieved by AA

  • Suhasini: what is the audio application for RVC ?

  • Andrey: (not captured)

  • Stephen: there might be one opportunity for standardization, some people manage the early audio on RTOS and then switch over to AA framework

  • Piotr: one possibility is to have 2 audio HALs, one "simple" audio HAL for early audio  and one full feature audio HAL

  • Philippe: iy would be good to look at a standardized way of switching modes

  • Andrey: we still have the safety requirements for this

• 11- Source Management
  • Multi‐source multi‐sink, etc.
  • Piotr: this is in the framework, it maybe not be worth looking at it as first priority, this is why I put a low
• 12- Audio focus
• Andrey: it would be great to get applications on AA to request audio focus (not optional audio focus), usually automotive use cases have a limited

  number of sound applications

• Patrick: at Mobis we have our own sound manager and pass it to AA, adding a little man between the sound applications and AA works well

• Patrick : I do not know much effort we put in this tool though

• Gunnar:  IMHO the concepts supported by the GENIVI audio manager component  might be relevant for this issue

• /TODO/ Gunnar contact the "right" person (i.e. who knows the GENIVI audio manager component) and organize a presentation of the concepts supported

• review of other topics will continue next week

F2F opportunity in early Q1/2020

• Philippe: the Vehicle HAL project intends to have a F2F meeting in early Q1/2020, BMW will host the meeting in Munich, Germany
  • look at Vehicle HAL F2F Planning - January-February 2020
• Philippe: it might be an opportunity for the Audio HAL project to meet F2F at the same date & location
• /TODO/ all check about your availabilities and the clearance of travel expenses

Next call

• it will be scheduled on Thu 19 December at 11:30am CET (usual time)

4 December 2019

Participants

• Andrey, Suhasini, Wassim, Stefan, Maria, Pontus, Gunnar, Philippe

Minutes

Introduction

• Philippe reminds all participants to identify additional extensions to Audio HAL that would not change too many things and identify a part of the Audio HAL that could common for all SoC vendors
  • this analysis needs more work and consensus reaching within each participant's organization
• Suhasini asks whether the list of topics is available in the wiki
• /TODO/ Philippe merge all inputs from minutes and slide decks into a single wiki page DONE
  • page is List of prioritarized topics for the Audio HAL
  • tentative categories have been added: configuration, common audio HAL, performance, multi-source management, multi-OS management, extensions. All categories are To Be Confirmed (TBC) and the mapping of categories to topics are all TBC.
  • /TODO/ all participants review the list of topics, categories, mapping and add priority (high, medium, low)

Configuration & Common Audio HAL

• Piotr shows the slide deck he presented at the tech summit in order to exemplify what our workplan should consist of

Next call

• Thursday 12 December at 16:00 CET / 7am PST (unusual time)

F2F meeting opportunity

• Vehicle HAL project will meet F2F at the end of January / beginning of February, this is an opportunity to colocate a F2F meeting for the Audio HAL project
• please look at Vehicle HAL F2F Planning - January-February 2020 and fill it in (you may have to ask for credentials)

21 November 2019 - Project Scoping  - Second Call

Participants

• Andrey (Harman) Sw architect
• Suhasini (Analog Devices) SW engineer for the car infotainment
• Wassim (BMW) Audio team SW architect, works on cross-ECUs function, audio manager, specific implementation, needs to clarify his position w.r.t. the maintenance role for GENIVI audio manager
• Stefan (Bosch) component maintainer for Audio at Bosch Car Multimedia, Hisldesheim, Germany
• H. Carlsson (Bosch) Bosch Car Multimedia, Lund, Sweden
• Bartosz + Piotr (Tieto)
• Gunnar
• Philippe

Minutes

Introduction

• Philippe welcomes participants and details the agenda

• Philippe recommends the review of the presentations delivered by Tieto and Windriver at the tech summit in Detroit where they presented their return of experience on Android Automotive, in particular on Audio HAL

  • please look here, search for "return of experience"

Introduction to Audio HAL

• Bartosz presents this slide deck
• We have more than 1 Audio HAL module in the system. There is always primary audio HAL that handles primary output (car speakers) and primary input (mic, aux, tuner, etc.) and in Android Automotive usually we also have USB audio HAL (dedicated to USB audio) and A2DP audio HAL (dedicated to Bluetooth A2DP). Different Audio HAL modules don't talk directly to each other, but the routing/bridging between them is handled via AudioFlinger/AudioPolicy based on audio policy configuration (where routing topology is defined).
• slide 6: grey boxes are external to AA
• Wassim: question on slide 14 about configuration
• Bartosz: configuration files are in the system partition or in the vendor partition, we lack a good concept for configuration, and for the persistence of configuration as well, support tools would greatly help

Gathering of list of topics for the Audio HAL project

• Bosch: we have issues with the BT audio stream / BT handsfree but I am just starting digging into it
• Gunnar: I am surprised that the BT handsfree is not part of the Google AA code
• Bosch: there is some support in the framework to support handsfree
• Gunnar: is the basic hansfree profile implementation part of the BT support in AA ?
• Bosch: pairing the phone with the device works but there is a need to connect the audio
• Andrey: shows a list of questions / problems, look at the list below
  
  • Android does not implement all features required by customer.
    • Audio focus can be lost at any time
    • Limited number of sources
    • Limited number of priorities -2
    • No easy way to support external amplifiers
    • No priorities of phone call types.
  • Android Audio subsystem is developed only for infotainment purposes. Safety-related features need to be implemented in another RTOS
    • Need to share the same hardware between 2 OSes
    • Running Android as a virtual machine inside RTOS leads to problems with scheduling of audio processes
  • Android way of extending its functionality is developing vendor extensions without modification of the framework.
    • But in some cases you need to modify framework. There is a process for this, but slow. And changes needed to one customer / developer is not needed for another. Even if you do this, you have to wait until Tier 1 merges this change to its drop
    • You have to retain compatibility for third-party apps
• Gunnar: what are the most important topics to tackle on ? maybe there is a particular problem to look at first
• Wassim: there is some topics that are low-hanging fruit
• Stefan: IMHO some of the topics are not related to Audio HAL but rather system-level architectural issues
• Gunnar: Audio HAL is much more complex than the Vehicle HAL
• Gunnar: in the Vehicle HAL we are finding solutions that can sit aside from AA, we do not intend to make a fork which would provoke a lot of
  integration problems
• Stefan: priority of sources varies among OEMs, we would need a better configuration capability from Google to handle this better
• Bartorz: extending the framework is fine provided you do not change the APIs, the HAL is actually something where we can put additional stuff "for free"
• Bartoz: this is a much bigger challenge to work out a common Audio Hal and port it to different SoCs but it is interesting to do
• Gunnar: agreed
• /TODO/ all participants identify additional extensions to Audio HAL that would not change too many things
• /TODO/ all participants identify a part of the Audio HAL that could common for all SoC vendors
• /TODO/ Gunnar and Philippe create the wiki page to gather the priorities of topics of interest listed so far  (from minutes of first meeting and presentations delivered by Tieto and Windriver at the tech summit)

Adjourned: 18:20 CET

7 November 2019 - Project Scoping  - First Call

Participants

• Andrey (Harman) principal architect, Android expert, based in Nijni-Novgorod, Russia
• Denis (Harman) manager of Android COE team (Center Of Excellence)
• Alexander (Harman) workis in the COE for low level Android
• Suhasini (Analog Devices), automotive infotainment software engineer, based in Bangalore, India
• Piotr + 1 (Tieto) mobile devices, automotive HW layer of Android, based in Poland
• Maria + 2 (Bosch Car Multimedia) manager and two SW engineers
• Gunnar (GENIVI)
• Philippe (GENIVI)

Minutes

• Philippe reminds the objective of the call (or of the first series of calls rather) which is to define the scope of a Audio HAL project
• Suhasini: we have not much experience with Android Automotive (AA), we are interested in AA because AA is now in the head unit
  • we have a portfolio of audio processors and an automotive audio bus to distribute audio within the car, we want to configure our proprietary audio bus SW stack to use it with AA as we did for Linux
  • we want to determine whether we have a problem of bandwith when using our proprietary network with AA, one approach is to use shared memory transfer
    in the audio HAL
• Piotr: we identified the following list of problems / tasks with AA when integrating the HAL for silicon vendors and OEMs
  • configuration of Automotive overlay, there is not much tool to configure the AA system
  • there is no way to control the equalization, i.e. no simple way to control global effects for output streams (by default Android application controls its own tracks)
  • there is no Audio calibration interface
  • there is a need for a generic interface for controlling audio effects on HAL level, global effects are designed for input streams but control over them is limited by interface
  • early audio for RVC (Rear View Camera) or other services
  • configuration management component for TinyALSA
  • Audio Focus doesn't forbid to interrupt Audio, Android 10 provides additional interface for Automotive to solve this problem
• Suhasini: we would like the following topics to be investigated
  • Audio data transfer / streaming to a co-processor for post processing of audio from the HAL/other Android layers
  • Support for the configuration of networked audio devices. Currently Android can view it only as a single device/sound card. An audio network in car might have multiple devices that needs to be controlled by the Android OS.
• Bosch engineer: we are working on two things, with the audio HAL we received from our silicon vendor
  • 1- patches for handsfree management, combination of BT stream with other streams, how to enable the correct audio routing, documentation on the Audio HALis missing, it is difficult to find examples on line
  • 2- I/O for our Bosch HW, the challenge is to adapt the AA framework to our HW, which has for instance 4 audio channels while we need to present them as 2 audio channels to AA
• Gunnar: how many of you are familiar with the Audio Manager of GENIVI ? I would recommend you to look at the features supported by GENIVI Audio Manager because it has been designed for automotive audio specifically
• Gunnar: introduces shortly the Audio Manager daemon and plug-ins
• Harman engineer: using Audio Manager am implementation designed for linux would raise an issue because of the Google CTS
• Andrey: safety related audio features can likely not be implementted in AA because AA is not a real-time OS

Next steps

• Gunnar: @Harman - would you consider making a list similar to what Tieto presented ?
• Suhasini: having an overview of the AA audio HAL would help ? can someone give such a presentation ?
• /TODO/ Harman & Bosch provide a list of topics that could be worked on jointly
• /TODO/ Philippe create a wiki page DONE
• /TODO/ Piotr prepare a short presentation on the AA audio HAL so that we share the same understanding of Audio HAL features

Next call

• we will have calls every other week, on Thursdays at 11:30 CET
• next call is scheduled on Thursday 21 November, a calendar invite has been sent