List of prioritized topics for the Audio HAL
Multiple-zone audio management - System Level Audio
Next Meeting - Thursday 23 April - 11:30am CET
Zoom dial-in
Dial-in by your location
Meeting ID: 470113570
Agenda
Agenda items
- list of investigation topics and definitions for the work on multi-zone audio management
- AOB
Thursday 16 April - 11:30am CET (AUDIO_HAL_CW16)
Participants
Philippe Robin Gunnar Andersson Piotr Krawczyk Unknown User (bartoszbialek) Suhasini Raghuram Ruslan Murtazin Unknown User (niskandar) Unknown User (wassim.filali) Andrii Chepurnyi @Henric
Agenda
- Discuss the gathered topics, who can work on which topics and which strategy
- Status update on the "access raw streams"
Minutes
Changes done in this page Android and System Level Audio:
- Gunnar: a bit of restructuring was done in order to better assess the project workflow:
- we need to first investigate the topics and choose what topics needs investigation
- Then we can apply analysis of both solutions to the topic
- definition were also added in order for all to agree on the terms that we use
Raw Streams discussion started
- Bartosz: we can get raw streams (unmixed streams) by bipassing the Android framework mixer.
- Applications can request for this, but by defaults all internal streams are going through the framework mixer.
- There is also the Android Audio API which is used for low latency streams that allows us to get raw streams.
- Gunnar:Unknown User (bartoszbialek) please add your comments in the page Android and System Level Audio
- Suhasini: what about the Network audio devices? can we use the device outbus for direct access of raw streams?
- Piotr: in summary, this API is used to handle addressable ports not for use of raw streams
- Bartosz: this is used for Android to capture external streams and represent them as internal streams but the audio path is longer (more latency)
- Raw Streams are both input and output streams
- Gunnar: when we talk about Raw streams do we want to take into consideration compressed streams
- Wassim: this is a link for the data format https://source.android.com/devices/audio/data_formats
- Piotr: the PCM format is mostly used and since we are starting with a generic solution, might as well not extend it beyond this scope
- Gunnar: what about the encoded and compressed streams?
- Piotr: multimedia framework is doing this and it's not part of the Audio HAL
- Bartosz: Android controls whether we want to take the stream and decoded using the CPU or not (this is called Offloading)
- Wassim: each stream has properties: depth, container size, compression, etc.
- here is a page of more attribute
- gunnar: Unknown User (wassim.filali) : please write down the decision about supporing the comrpession in our scope in the page Android and System Level Audio
- Wassim : As described in the Android data_formats page, Compression is a Property of each audio stream. Although audio streams are most of the time uncompressed (PCM) due to avoidance of compression decompression cpu time loss, it might happens that a source stream is provided compressed by default. In that case the common formats are FLAC (lossless) and MP3, AAC (lossy). Further details about compression would be case specific.
Input streams
- Gunnar: input streams no one yet investigated
- Gunnar: Unknown User (niskandar): investigate input steams as well from the page Android and System Level Audio
Presentation on the status of the POC from Piotr (presentation is attached and will be updated)
- Piotr Krawczyk : please update the presentation attached
- Piotr:Stage 1 of the POC should be finished next week
- Gunnar Andersson: provide a place to add the code of the POC
DONE → 2020-04-20: android-external-audio-mixing repository
- Gunnar Andersson: provide a place to add the code of the POC
- Piotr: the concept is that we can either
- Create a separate framework parallel to the android framework that takes care of accessing devices. Any application that is aware of such a framwork can use it in order to better control the streams.
- Or use the framework to create many devices and connect them as needed.
- Gunnar: this is also nice for systems with different OS
- Piotr: there is still one point of investigation
- the Android Audio API, the one created to allow low latency streams. we have to think how we would handle this.
Gathered Todo's:
- Unknown User (bartoszbialek) please add your comments in the page Android and System Level Audio
- Unknown User (wassim.filali) : please write down the decision about supporing the comrpession
- Unknown User (niskandar): investigate input steams as well from the page Android and System Level Audio
- Piotr Krawczyk : please update the presentation attached
- Gunnar Andersson: provide a place to add the code of the POC → android-external-audio-mixing repository
- @All: please check the topics of investigation to see if you would like to add more topics to the page Android and System Level Audio
- @All check your emails to see if you have received the invitation about the summit
Thursday 09 April - 11:30am CET (AUDIO_HAL_CW15)
Participants
Philippe Robin Gunnar Andersson Piotr Krawczyk Unknown User (bartoszbialek)Andrii Chepurnyi@Henric Suhasini Raghuram Stephen Lawrence
apologies: Wassim, Nadim
Agenda
- from last week's minutes
- Discuss the gathered topics, who can work on which topics and which strategy
- Status update on the "access raw streams"
- AOB
- abstract of the GENIVI virtual tech summit AASIG Audio HAL workshop
Minutes
list of gathered topics on multi-zone audio management
Philippe: asked whether the participants started to think about the gathering of detailed topics for multi zone audio management analysis
- Philippe: reminds about the project timeline until the GENIVI virtual tech summit workshop scheduled on Thursday 14 My 9:00-12:00am CET
Gunnar: I started thinking about the definition of what a raw stream is
Gunnar: I would like the team to write down the topics or questions or features first and then check whether those fit in one strategy or the other or both
Suhasini: there are some termonilogy questions and concepts to align on
Gunnar: agreed, raw stream is one of them, are there others ?
Gunnar the list of prioritized topics is too high level, raw stream is at a lower level of details
Gunnar: I will create new headings on investigation topics and definitions in the wiki page created by Wassim
investigation topics added: raw streams from Android, input streams to Android
definitions added : raw stream, raw stream metadata
- /TODO/ all participants to populate the above sections with their proposals
hint: reading through the List of prioritized topics for the Audio HAL again and copy/paste whatever seems to you relevant for the work on multizone management
abstract of the GENIVI virtual tech summit AASIG Audio HAL workshop
"AASIG AAL abstract Audio management is one important aspect and constraint for the utilization of Android Automotive for the IVI unit. During Q1, 2020, the AASIG AUDIO HAL project team has finalized the list and priorities of audio management related topics. The team has decided to investigate the software architectural design for audio multi-zone management. The following design options are investigated, (a) all audio management is inside Android Automotive, and AA manages all sinks & sources and (b) all audio management is outside Android Automotive. The team is implementing proof-of-concepts to experiment the design options. The AASIG AUDIO HAL project team will present a status report of the proof-of-concept implementation and explain the design choices made. The following next steps will be then comprehensively debated:
- definition of proof-of-concept roadmap, versioning of proof-of-concept content w.r.t. roadmap milestones
- early assessment of implementation TRL (Technical Readiness Level) and discussion on when and how to reaching out Google
- identification of other proof-of-concepts implementation based of the list of prioritarized topics for audio management"
Thursday 02 April - 11:30am CET (AUDIO_HAL_CW14)
Participants
Philippe Robin Gunnar Andersson Piotr Krawczyk Unknown User (bartoszbialek) Unknown User (wassim.filali) Andrii Chepurnyi Unknown User (niskandar) @Henric Suhasini Raghuram
Agenda
- Slides from Wassim about the two models inside and outside of AA
- discussion of the cases/features that we are trying to solve
- checking the Audio Manager to decide whether or not it needs an update
Minutes
- 2 strategies or options:
- android provides sources and sinks
- Android controls the complete systems
- We will not discuss the need to have such options
- Basically some functions cannot be integrated to AA
- Audio sources coming from android or from external (android not aware of them)
- Each strategy has limitations and can be criticized
- Idea is to get each model and apply to it questions/cases
- Questions like safety sources, raw streams, etc.
Piotr: what do you mean by vendor partition
- Wassim: I didn't want to specify the actual parition because it can be vendor specific
- Gunnar: android concept "vendor partition" is booked, maybe we can find another name like virtual machines?
Piotr: audio sink with headphones, what did you mean?
- Wassim: a sink, it's meant as an Android sink, a sink that is connected to Android
- Gunnar: But is it that straightforward?
- Piotr: it's not, the BT for example is a special case, we would add it to the open questions
Piotr: this is actually something we can work on (refering to the BT case)
- Wassim: I'll add it to the wiki and make it possible for people to add topics
Wassim: we can also think about it that: what questions should be done where
- The output of this project would be a recommendation: we as GENIVI, recommend to use this feature outside of AA and this feature inside of AA, etc.
- Gunnar: this is a nice idea actually to make a hybrid solution because we cannot recommend one solution as GENIVI
- Gunnar: we learn from both approaches and use it to build the hybrid solution
- Piotr: it's also not sure if we can achieve one of these extreme cases completly
Discussion about the page followed to define the new page in the wiki
- Page will describe each topic and we will add comments if we can do it in Android or outside of Android and what areadvantages/disadvantages
- The page will help us reach our recommendation
Piotr: Major problem for common HAL is how to we get raw streams
- Henrik: how to get raw streams, what do you mean by this?
- Piotr: current solution, audio data -> tiny ALSA -> hardware
- To get more flexibility -> not directly to tiny ALSA, direct it to whatever we want
- If we can manipulate this behavior we can manipulate the audio stream and direct it to output
- Gunnar: raw stream is the stream that the source is creating, if we have this stream without volume change, without mixing, without effects, etc.
- Raw Stream: as little change as possible to the stream
- Piotr: I would start by extracting these stream from Android, see if this is feasible
- Gunnar: before implementing or starting we would just need to analyze the current system
- Wassim: we should know if we can extract raw stream from Android or not. Without this point we cannot start the second extreme
- Wassim: hybrid also needs the raw streams
Nadim: What about the point Audio manager update?
- Gunnar: There are updates going on but we are not yet concerned about them
- Gunnar: If we start with outside of android strategy we will come to this subject and check if we need to add to it or update it etc.
Decisions
To proceed in our project of Android and System Level Audio we need several steps:
- Gather topics or questions or features that we would like to check whether they fit in the two strategies
- @all Gather topics in the wiki page created by Wassim
- Analyze the topics in both strategies to see if they are feasible, if they are easy to implement, etc.
- POC and code for each topic in the different strategies would be needed
- Conclude GENIVI recommendation from the POCs made: form a hybrid solution.
- Which topic do we recommend to be implemented in AA and which outside of AA
Starting point would be to check if we can access the raw streams from AA
- Piotr Krawczyk: Tieto suggested to take responsivbility of this task, expect 2-3 weeks for a POC
Next Agenda (for AUDIO_HAL_CW15)
- Discusss the gathered topics, who can work on which topics and which strategy
- Status update on the "access raw streams"
Thursday 26 March - 11:30am CET
Participants
- nadim, henric, bartosz, piotr, lukasz, suhasini, wassim, philippe, gunnar
Agenda
- how to control multizone from the system service
- reference system design
- AOB
Minutes of AUDIO_HAL_CW13
how to control multizone from the system service
- tietoEVRY shows slide deck presenting the results of the PoC experimentation they made
- under the hood slide: this a long away to go to achieve a simple functionality
- useful links for understanding audio zone management in Android Automotive
- discussion on the slides follows
- Nadim: you cannot change zone w/o stopping the playback, that's a negative or missing point
- the use case discussed in the previous call (push a button and change the audio focus) is discussed again
- Piotr: the approach was not to change the AA framework to implement this PoC
- Piotr: I would recommend to build a new functionality based on existing functionalities, i.e. like the development of a get UID of active tracks, create such a service would allow the driver to manipulate the UI for redirecting applications to a given audio zone
- Wassim: are these zones like "play in the front or in the back", or "play on the main speakers or on the headphones
- Piotr: for each zone, the context is defined
- Wassim: how does the balance function interact with the multizone management ?
- Suhasini: are the zones more logical than physical ?
- Piotr: Google says a zone is a collection of devices (look at the definition at the beginning of the slide deck)
- Wassim: the device can have a mapping of the different speakers, this is different from a setting like the balance
- Wassim; do we have a limit of the number of apps that can be mixed ?
- Piotr: there is no limitation AFAIK
- Nadim: there is a table (as Piotr said it's static) that determines what audio can be mixed https://source.android.com/devices/automotive/audio/multi_zone/audio-focus
- Wassim: let us assume the actual hw have a limitation on the number of audio channels that can be mixed at the same time
- Bartozs: when the limit is encountered, the audio HAL should report an error which is communicated back to AA
- Wassim; the limitation does not come from AA, but the error can be reported (general error handling in AA)
- Piotr: the audio focus is separate for each audio zone
- Nadim: what is the A2B in the presentation?
- Piotr: AA is missing support for A2B
- Suhasini: yes we know about this.
- random thoughts
- Gunnar: let's think of a multizone concept outside of AA because:
- from history many systems (like MOST based for example) are outside of the Infotainment Head Unit, getting the raw data streams out and they would rely on local DSP or amplifier to tune or do an effect
- (alternatively, if you wish) think of it as just a thought experiment to see what problems we might encounter to understand better the concept
- → is there any limitation in the AA model? let's discover it
- (Question: But why not just assume AA to be in charge of all the audio management?)
- Gunnar: Another reason is the complexity of real car development. Consider car models with different (Infotainment) variants where the one variant might not have AA but are based on a different OS? (You may then be forced to use solutions that work across all those variations of systems)
- Henric: in Audio HAL there are options to access input/output streams, this can be useful
- Gunnar: if we are under the Audio HAL we have the possibility to do whatever we want with the streams
- Wassim: so this gets us back to the main topic of thinking about managing the audio with AA in control and with AA not in control
- Wassim: we need to think more about which model is better suited for our cases
- Nadim: I think it's better to follow the AA in control model because it might be cheaper in the long run and because it includes a lot flexibility
- Wassim: how much flexibility though? does it cover all of our cases? what are our cases actually? maybe we can gather the cases for next week
- Piotr: what direction should we continue with the POC?
- Wassim: I think we should determine what model is better: AA in or not in control
- Piotr: so the AA not in control would be some sort of a Framework (FW) service that is injecting audio data to AA
- Bartosz: here are some inputs from my side:
- AA is expecting that some sources are not handled by Android, this is how it was designed
- Specifically in Android10 they dropped Auto ducking because they reached the conclusion that they cannot control all sources
- Audio zones is limited to AA but we can still use external audio management (https://source.android.com/devices/automotive/audio#external-streams)
- Philippe: how do we manage Safety critical applications (ADAS for example) ?
- Piotr: this can be handled by the external management that Bartosz mentioned
- Gunnar: let's think of a multizone concept outside of AA because:
GENIVI Spring Virtual Technical Summit
- the virtual tech summit will be organized as half-a-day workshops scheduled to be either at US-friendly time or Asia-friendly depending the expected geo-location of the main participants (all times are CET)
- Tuesday 12 May afternoon (US friendly): president's keynote and state of the union followed by a couple of sessions about "looking forward" topics
- Wednesday 13 May afternoon (US friendly): AASIG VHAL engineering workshop (3 hours)
- Thursday 14 May morning (Asia friendly): AASIG Audio HAL engineering workshop (3 hours)
- Thusday 14 May afternoon (US friendly) : Cloud & Connected Services engineering workshop (3 hours)
- this is still TBC but please make a note in your calendar
Decisions taken in AUDIO_HAL_CW13
- Agenda for AUDIO_HAL_CW14:
- Slides from Wassim about the two models inside and outside of AA
- discussion of the cases/features that we are trying to solve
- checking the Audio Manager to decide whether or not it needs an update
- @all: Think and prepare diagrams of how you see the models of AA in control of the audio management and AA not in control of the audio management
- @all: Check what features and what cases do we have in order to validate both models
- @all: Prepare any questions about the Genivi Audio manager, mainly think if it's worth it to update it. Is AA audio manager missing some features?
- Unknown User (niskandar)focus on taking minutes of meeting
Thursday 19 March - 11:30am CET
Participants
- nadim, henric, piotr, wassim, suhasini, gunnar, philippe, stephen
- apologies: bartosz bialek
Agenda
- GENIVI Audio Manager Q&A
- how to control multizone from the system service (skipped, scheduled for next week)
- AOB
Minutes
GENIVI Audio Manager Q&A
Henric: I looked at the document, the AA framework also does audio management, how AM & AA both fit together ?
Gunnar: the main idea was to learn from the AM which solved this issue in existing production systems, the goal is not likley to reuse the AM code directly
Wassim: we have already systems using AM, if AA were used because it can meet the AM specifications, this would be fine
Nadim: I looked also at the document, what the AM is all about ? why is the rationale for using the AM ?
Gunnar: we said that reading the AM specifications is to learn about the concept and possibly reuse the concept
Wassim: we are doing automotive, does everyone have an automotive focus in the team ?
Gunnar: yes
Wassim: we need to look at system level, we can have a head unit, rsa, headphones for passengers,
Wassim: while each vendor develops their own ECU, the problem for the OEM to set/control/configure all the parameters in a centralized way, we need to control things in a centralized way,
Wassim: AM makes only the connection between sources and sinks and relies on other libraries to support the audio stream communications
AA offers the connection to Alsa at a device level, but fails to handle the connections at system level
this is the interested part of the discussion we should have: the handling of multiple connections is to be handle in the vendor partition (Audio HAL)
Wassim: I am interested in vendor agnostic (and OEM agnostic) solutions to solve the handle of multiple connections
Nadim: we would recommend to reuse the AM then
Wassim: you do not need an AM everywhere in the system, you need however a routing adapter in every ECU of the system
discussion on the architectural design of the AM (AM daemon, plugins)
Gunnar; the AM daemon can be used by everyone *unchanged*, the plugins have to be adapted to a given project details
Wassim: if you have multiple (audio) domains, you might also vendor specific buses for each domain, so you will need gateways to convert from one domain to another
Wassim: you informed the AM that you have two domains for instance and the AM will know about the routing to provide
Gunnar: if we look at the AA system, do I simplify too much if I say we want the AA system to produce all streams, controlling the mixing and priorities ? (to be re-captured, Gunnar Andersson please update)
Wassim: let me give an example use case
I have one button somewhere and I want to select the AA radio and route the audio stream to the device where I touched the button
Wassim: our goal is to give the capability of having all devices on the same SoC and having the external amplifiers in another location
Wassim: it would be nice also to have classes of sources (entertainement, announcements, etc.)
Gunnar: what is Tieto thought on this ?
Piotr: there are classes in AA, there is a hardcoded matrix to describe them since AOSP 10
AA exposes those to 3rd party applications, there is a framework layer and the HAL layer, are we fine with the features supported by the framework layer ?
discussion on how to use the framework features (audio flinger for instance)
is the framework functionality enough for implement what we need in the car ?
Gunnar: IMHO there are too much features in the AA framework (to be re-captured, Gunnar Andersson please update)
discussion on what we should standardized outside of AA
Wassim: could we iidentify and name the 2 following scenarii ?
1 android manages all sinks and sources, i.e. all audio management is inside AA
2 all audio management is outside AA, this is the so-called simple scenario where the audio management is done externally
Nadim: what is the expectation for the poc ?
Philippe: the objective of the poc is to validate our design and check it supports the required flexibilty
Wassim: scenario 2 control can be in a different software partition or SoC or an external ECU
Philippe: we need to agree on a reference system design to implement the poc use cases
/TODO/ all participants to produce their own system design from which we will identify commonalities and agree on a reference system design
Gunnar: Piotr was already assigned a presentation on multi-zone management (for next week)
Nadim needs to understand better the problem
Gunnar: how to manage an external amplifier with AA is a use case by itself
- discussion on how to jump start on the reference system design
/TODO/ Wassim to initiate a wiki page describing the block-diagram of a possible reference system design
Thursday 12 March - 11:30am CET
Participants
- bartosz bialek (tieto), nadim, henric, piotr, wassim, ruslan, suhasini, gunnar, philippe
Agenda
- GENIVI Audio Manager - Q&A after review of the docs as discussed in last week's call
- PoCs design
- Project workplan
- AOB
Minutes
GENIVI Audio Manager
Nadim: stiil need more time
Piotr : still on my todo list
Suhasini: went to the architecture overview
Henric : on my todo list
Gunnar: if this is the right way of doing ?
Nadim: Philippe is right, it should be on the top of the list
Gunnar: the AM is how to design an audio system which is flexible
Wassim: do we agree we all need a vendor partition distinct and beyond from Android Automotive ?
PoC design
- Wassim: we need to define simple use cases and demos in order to work out the PoC design and derive how it could look like w.r.t. devboard, dev kit, etc.
- Philippe: reminds the team about the GENIVI test farm
- Gunnar: test farm is actually hosted by Renesas and uses R-Car boards, NXP board is with Gunnar only obviously
- Gunnar: explains some details of the lava test farm operation
- Wassim: would be interested to have links to the test farm
- link: lava.genivi.org
- Gunnar: the test farm it is not documented yet in the wiki, this topic might be for a later time though
- /TODO/ Gunnar invite Wassim to the BIT call where the test farm is discussed
Project workplan
- Philippe: reminds his concern from a management perspective !
- Piotr: we need to dig Android Multi-Zone concept -- specifically, how you could control multi-zone from a system service such a use case is missing currently
- /TODO/ Pior to prepare a material on how to control multizone for next week or following week call
Thursday 5 March - 11:30am CET
Participants
- Wassim, Ruslan (EPAM, automotive systems) Gunnar, Philippe
- apologies: Nadim, Bartosz, Piotr
Agenda
- GENIVI Audio Manager
- Audio HAL project workplan
Minutes
Roundtable
- Ruslan represents Andrii in today's call
GENIVI Audio Manager
- Q&A after review of the GENIVI Audio Manager docs is shitfted to next week's call
recap on the decision made by the team last week to get awareness on the GENIVI Audio Manager
Gunnar: the code might not be reusable as such in an Android Autmotive context, but it is worth understanding the archictectural design of the Audio Manager
Gunnar: Android is lacking the flexibilty audio systems have in a car, this project intends to enhance this
Wassim: there are deployed systems using the linux based GENIVI Audio Manager or using Android Automotive, who should take care of the audio management in the Android context ?
Gunnar: there 2 tracks one is to find out the audio management requirements in the Android Automotive context, track #2 is to achieve this with the appropriate technologies, we need to decide first what we want
Wassim: we need to define the common problem statement the Audio HAL wants to solve
Discussion on the Audio HAL project workplan
- Wassim: Audio effects are critical because they fall into the domain of custom interfaces in Android Automotive
- Wassim: the networked audio systems exemplify why a car is different from a smart phone
- Wassim: my vision is that with the networked audio systems Android Automotive will become an OS for distributed systems
- Gunnar: GENIVI is now focusing on system-level integration
- Gunnar: for the VHAL our objective is not only to provide more vehicle data (than the Google vehicle properties), it is also to dig into the Android Automotive based head unit and integrate with the rest of the vehicle systems
- Wassim: Android Automotive uses a wrapup around Alsa, we do not need to change this, what we should look at rather is how to solve the custom interfaces in a general way as the VHAL is doing for the access to vehicle data
- Wassim: what about graphics ?
- Gunnar: graphics came as a side track in the GSHA project, we look for instance at the combination of the Wayland (linux) stack and the Android Automotive stack, this was not in the AASIG actually, there is one company working on an implementation for this
- browsing through the collaborative tools for AASIG: wiki structure and jira project
- Philippe: explains the way we structure the work in the SIG from the identification of problems to solve, the brainstorming on architectural design options and possible prood-of-concepts to validate the architecture concepts and the definition of work breakdown structure for the PoC implementation
Thursday 27 February - 11:30am CET
Participants
- Nadim, Bartosz Cichosz, Gunnar, Andrii, Bartosz Bialek, Piotr, Stephen, Wassim, Suhasini, Henric
- apologies: Philippe
Agenda
- Networked audio devices
- PoCs planning
Minutes
Networked audio devices
Gunnar: Le'ts start with the networked audio devices overview. It can be good input information before discussing PoC details
Suhasini presents her slide deck (link)
Wassim: When you say streams, (and the red lines in the picture), what does it mean?
Suhasini: It represents a network connection of some type (any type). Streams, I mean an audio stream logically from one device to another here. On the next slide there is a mapping shown.
... mapping slide
... Automotive Audio Bus (A2B) slide.
A2B is an example of single master, multiple slaves setup.
It is routed according to your configuration.
You can for example configure that mic input must go to head unit. Protocol takes care of it according to configuration.
Problem: Physically speaking the head unit is connected to only one device. The system views it as a single device.
On the logical picture, one stream is connecting to more than one receiver. Viewing as a separate device is useful but...
Android 10 features has Audio Device Out but, but it's not clear if we can address (receiving nodes) as we want.
Wassim: One device visible but many logical channels...
Suhasini: Problem 2 could be diagnostics support. Checking if the node is in an OK state...
Control path should be able to support more than volume. E.g. a series of diagnostic routines that can be initiated.
ALSA has minimal support also for this.
Wassim: ALSA supports well the (audio) data flow but not control. GENIVI audio manager has the primary purpose to define such interfaces. (It is not handling the audio stream directly).
The drivers (from silicon vendor) might still be implement the control function itself.
Wassim: The vendor needs to provide a HAL for those control functions. If Android defined the interface, this might be implemented.
Summary: View each device stream separately inside Android.
Discussion yields that we should perhaps handle each stream separately, in addition to devices. (sink and source abstraction)
Agree that each single stream could have types (mono, stereo, multi-channel) but we don't expect to need to handle reconfigure channels.
Gunnar: In addition to differentiating the individual receiving devices I expect we need to consider that one or several streams per device?
... how to implement additional features ...
Wassim: GAM for example has a generic support for "Properties" from user to sink, and "Notifications" from sink to user.
Gunnar: Note: "user" here means the controlling software, either the application or the main system HMI that is defining the behavior we want to have from the audio system.
Wassim agrees.
Discussion on other needed control interfaces... example: firmware update?
More discussion..,
Suhasini: In order to transfer a certain stream e.g. phone input to some output, this in the end needs to be configured in hardware...
Initial configuration will be created by the vendor, but some kind of interaction with the driver is needed.
Wassim: Look at open documentation to AudioManager. The routing plugin abstracts the logic of this. The implementation is doing the connection between hardware specifics and the generic interface.
I have seen that Android also allows some kind of custom bus implementation, but have not yet seen if this solves it.
Bartosz (Harman) - short introduction. Android engineer, not specifically focused on audio however. Based in Poland.
Action (all): Read up on the architecture and interfaces of the GENIVI Audio Manager, since it was designed to make a framework for exactly this. Docs: https://genivi.github.io/AudioManager/
The ideas / design / APIs should be reusable. Possibly some code also, eventually.
Stephen: Are those docs also covering plugins?
Gunnar: To further understand plugins, try looking at code of the plugins: https://github.com/GENIVI/AudioManagerPlugins (The code of the AM daemon is at https://github.com/GENIVI/AudioManager/ )
Stephen finds another slide deck describing routing-adapter.
PoC planning
We had 2 main topics: Global Effects API and more flexible audio channel routing.
Gunnar: OK so the discussion we had before led forward in the audio channel routing topic. What about Global Effects?
Bartosz (Tieto) reports that the plans for the next Android version include new interfaces that seem likely to solve the global-effects problem.
It is possible therefore that we do not need to work out solutions for global effects: LINK 1
, LINK 2, LINK 3
Adjourned at 12:25.
Thursday 13 February - 11:30am CET
Participants
- Suhasini (Analog Devices), Henric (Bosch), Nadim (Mobis), Bartoz (Tieto), Philippe
Agenda
- Overview of last week's F2F outcome
- Call for participation
- AOB
Minutes
Overview of last week's F2F outcome
- Philippe presents the outcome of the meeting using this slide deck
- Meeting was productive and the team had a very good collaborative spirit
- Minutes of the F2F meeting are here
- Nadim: minutes are very helpful
- Philippe: asks participants for their feedback and comments in the wiki
- Suhasini: I would like us to refine further the scope of the PoCs
- Suhasini: I would like to understand better what was discussed on AVB in the meeting
Nadim: Bartoz, Gunnar and Piotr wanted to know about which technologies Analog Devices has in mind for the networking of audio devices, this is why AVB popped up
Call for participation
- Philippe calls for participation for refining further the PoCs scope and initiating architectural design work
Suhasini: /TODO/ in 2-week time, will give an overview and a problem statement for the networking of audio devices
Audi HAL call schedule
- Next call will be scheduled on Thursday 27 February at 11:30am CET
Audio HAL F2F Meeting 4-5 February
23 January 2020
Participants
- Andrey, Piotr, Wassim, Henric, Philippe
Minutes
upcoming F2F meeting
- agenda is here, Audio HAL part is scheduled on Day 2 morning
- review of high priority topics selected for discussion at the F2F: Common HAL & Source Management
- assignment of preparation work done (although one is TBC)
AOB
- CES debriefing: GENIVI showcase at CES went out well, GENIVI hosted 1400 people, feedback about the event was very positive
- AMM: the Spring AMM will be scheduled in May, dates and location should be announced soon
19 December 2019
Participants
- Andrey, Suhasini, Piotr, Stephen Lawrence, Gunnar, Philippe
Minutes
upcoming F2F meeting for the Vehicle HAL project
- decision on the dates was made => 4-5 February, at BMW, Munich, Germany
- Philippe: we have an opportunity to coschedule (and colocate) a meeting for the Audio HAL project
- /TODO/ all fill in the participation table and indicate whether you would be available and interested by joining a meeting either F2F or via telco
Review of the list of prioritarized topics (continuation)
13- BT handsfree
Suhasini: it is rather a question of multisource management
Andrey: would agree it is related to multisource management
Gunnar: on the mobile phone, the BT role is the server role, the car is more the handsfree role
Andrey: we need to clarify the automotive use cases
the group should work on use cases but there are more fundamental questions to be solved on multi source management
14- Android does not implement all features required by customer.
Andrey: this is about multisource management, support of external devices, limited number of priorities, etc. the various bullet points can be distributed over the table
/TODO/ Andrey distribute the bullet points among the table
Piotr: what are the priorities you are talking about ?
Andrey: these are the priorities related to phone calls, I will clarify this when I do the split (phone calls vs. not phone calls)
15- Android Audio subsystem is developed only for infotainment purposes. Safety-related features need to be implemented in another RTOS
Andrey: thinks the safety-related audio features are not in the scope of this group
Piotr: we need to keep in mind that there is a higher priority audio functionnality in the system
Gunnar: it might be worth discussing how AA can use external components that manage the higher priority audio functionnality, these are system-level considerations
/TODO/ Gunnar add HERE what was discussed in this call about the system level architecture Thanks
Gunnar: do we look for solutions on AA side first or do we look also at the general system design for audio ? my recommendation is to start something concrete at the design level
16- Android way of extending its functionality is developing vendor extensions without modification of the framework
Piotr: we need to avoid changing the framework because there is a lot of manual work necessary to track the changes that need to be taken into account
Andrey: agreed
Andrey: we have a mechanism in Harman to follow up the changes in AOSP
Gunnar: we need to start the design work asap and as a general principle we do not change the framework
17- Google is still very “hand wavy” about solution
Gunnar: WR has a lot of work to implement the audio HAL by themselves
Piotr: people look at the code of android directly, I would like to get some tooling to cope with the fact the doc will not be provided by Google anyway
Piotr: tool can take care of the configuration setting, helping with work we do currently manually
Andrey: we need to adapt to the Google way, we cannot change this
Piotr: we can translate HIDL to Franca and provide a more flexible way of implementing things
Andrey: we are considering similar ideas within Harman
Gunnar: agreed, it is part of the discussion on how to use a common interface in multiple systems (with the remote interfaces, not only on AA), something like CommonAPI, there is a generic discussion to have there
18- Genivi ?
Gunnar: in my opinion this is a question about the role of GENIVI, we need to think about what could be the outcome of the group, as I said before, the question is rather to find solutions, it can be anything from a formal specification, to guidelines and even code
- category changed to "Deliverables"
- Philippe: IMHO this is also a question about the recycling of earlier work done by GENIVI, e.g. the audiio manager, we said we would organize a presentation of the GENIVI audio manager and the automotive use cases it supports
- /TODO/ Gunnar plan a presentation of the GENIVI audio manager in Q1, 2020
list of prioritarized topics
- /TODO/ all review the minutes of 12 & 19 December together with the list of prioritarized topics and identify any gap / missing points in the list
Next call
- Thursday 23 January, 15:00 CET / 20:30 India / 6am US Pacific
- agenda
- review comments for the list of prioritarized topics
12 December 2019
Participants
- Andrey, Suhasini, Piotr, Jimhyuk Jung (mobis), Patrick Carlisle (mobis), Stephen Lawrence, Giovanni, Gunnar, Philippe
- apologies: Wassim
Minutes
Roundtable
- everyone delivers a quick personal introduction
- Jimhyuk works for mobis technical center Europe (MTCE), he is an IVI expert
- Patrick works for mobis technical center America (MTCA)
- Piotr indicates he would like to have a less hardware aware Android
Review of the list of prioritarized topics
- list upated online, each contributor explains the rationale for setting the priority he chose
- 1- Networked Audio Devices: Support for the configuration of networked audio devices. Currently Android can view it only as a single device/sound card. An audio network in car might have multiple devices that needs to be controlled by the Android OS
- 2- Overall configuration management
- Piotr: there is no way to automatically validate the configuration
- Andrey: IMHO there is more important things to solve than the configuration
- 3- Common Audio HAL
Piotr: audio HAL is not so HW dependent because of tinyalsa, the porting the common audion HAL should be limited to tinyalsa
Andrey: I do not think whether we can make a common audio HAL, it would be great to have such a common HAL for using eAVB ?
Piotr: some hw vendors are working on it, must be feasible
Gunnar: this is about finding the common part, this is about avoiding duplicated effort to provide the same functionalities, this about reducing the amount of effort for each vendor to implement the HAL
- 4- Audio Data Transfer
Suhasini (not captured)
Piotr; utilizing a DSP is very hw specific, this is why I put it a low priority, however it is important, it should be high priority
Gunnar: this is about finding the right abstraction
Piotr: we need to talk about interfaces / abstraction to create audio effects
- 5- Equalization
- all agree it is high priority to have a way to control the equalization
- 6- Audio calibration
- Suhasini; audio calibration sounds to me like controlling global effects, not as big a problem as others
- Piotr: agreed, could be covered by the same interfaces, there is a need for the a generic interface
- 7- Controlling audio effects
- the above 3 items could be covered by the same interfaces
8- Latencies
Suhasini: for Analog Devices, latencies are important because we have a lot of time sensitive applications, currently we are doing these on the DSP, we would be interested in moving some of them on the Android system
Piotr: IMHO the latencies question is pretty hw specific, it cannot be solved in a general way, this is rather Google responsibility with their framework
Piotr: another problem is shared audio on a virtualized platform, not sure we can solve this problem in a joint effort
Gunnar: in the GENIVI Hypervisors (HV) project, Opensynergy is behind the proposal for a set of standard interfaces for the audio platform, we could dig into it if there is an interest in the Audio HAL group
/TODO/ Gunnar organize a presentation of the work done by the HV project on the standard interfaces for the audio platform
Andrey: we encountered a problem with latencies when we switched to hypervisord, actually we had problems with latencies AND scheduling
Gunnar: it is a difficult challenge, agreed
- 9- Multiple audio channels
- Bosch indicated their challenge is to adapt the AA framework to their HW I/O, e.g. they have 4 audio channels while they need to present them as 2 audio channels to AA
Piotr: it is hard for me to judge because it is very specific, my claim is that it can be solved with AA
Andrey: I do not know it was feasible
Philippe: this needs further investigation
10- Early audio for RVC
Andrey: early audio RVC will be rather handled by the HV or RTOS, because there are safety requirements which cannot be achieved by AA
Suhasini: what is the audio application for RVC ?
Andrey: (not captured)
Stephen: there might be one opportunity for standardization, some people manage the early audio on RTOS and then switch over to AA framework. Can the methods on Android side by standardised? This also applies to other 'early' features such as video for reversing camera.
Piotr: one possibility is to have 2 audio HALs, one "simple" audio HAL for early audio and one full feature audio HAL
Philippe: iy would be good to look at a standardized way of switching modes
Andrey: we still have the safety requirements for this
- 11- Source Management
- Multi‐source multi‐sink, etc.
- Piotr: this is in the framework, it maybe not be worth looking at it as first priority, this is why I put a low
- 12- Audio focus
- Andrey: it would be great to get applications on AA to request audio focus (not optional audio focus), usually automotive use cases have a limited
number of sound applications
Patrick: at Mobis we have our own sound manager and pass it to AA, adding a little man between the sound applications and AA works well
Patrick : I do not know much effort we put in this tool though
Gunnar: IMHO the concepts supported by the GENIVI audio manager component might be relevant for this issue
/TODO/ Gunnar contact the "right" person (i.e. who knows the GENIVI audio manager component) and organize a presentation of the concepts supported
- review of other topics will continue next week
F2F opportunity in early Q1/2020
- Philippe: the Vehicle HAL project intends to have a F2F meeting in early Q1/2020, BMW will host the meeting in Munich, Germany
- Philippe: it might be an opportunity for the Audio HAL project to meet F2F at the same date & location
- /TODO/ all check about your availabilities and the clearance of travel expenses
Next call
- it will be scheduled on Thu 19 December at 11:30am CET (usual time)
4 December 2019
Participants
- Andrey, Suhasini, Wassim, Stefan, Maria, Pontus, Gunnar, Philippe
Minutes
Introduction
- Philippe reminds all participants to identify additional extensions to Audio HAL that would not change too many things and identify a part of the Audio HAL that could common for all SoC vendors
- this analysis needs more work and consensus reaching within each participant's organization
- Suhasini asks whether the list of topics is available in the wiki
/TODO/ Philippe merge all inputs from minutes and slide decks into a single wiki pageDONE- page is List of prioritarized topics for the Audio HAL
- tentative categories have been added: configuration, common audio HAL, performance, multi-source management, multi-OS management, extensions. All categories are To Be Confirmed (TBC) and the mapping of categories to topics are all TBC.
- /TODO/ all participants review the list of topics, categories, mapping and add priority (high, medium, low)
Configuration & Common Audio HAL
- Piotr shows the slide deck he presented at the tech summit in order to exemplify what our workplan should consist of
Next call
- Thursday 12 December at 16:00 CET / 7am PST (unusual time)
F2F meeting opportunity
- Vehicle HAL project will meet F2F at the end of January / beginning of February, this is an opportunity to colocate a F2F meeting for the Audio HAL project
- please look at Vehicle HAL F2F - 4-5 February 2020 - Organization and fill it in (you may have to ask for credentials)
21 November 2019 - Project Scoping - Second Call
Participants
- Andrey (Harman) Sw architect
- Suhasini (Analog Devices) SW engineer for the car infotainment
- Wassim (BMW) Audio team SW architect, works on cross-ECUs function, audio manager, specific implementation, needs to clarify his position w.r.t. the maintenance role for GENIVI audio manager
- Stefan (Bosch) component maintainer for Audio at Bosch Car Multimedia, Hisldesheim, Germany
- H. Carlsson (Bosch) Bosch Car Multimedia, Lund, Sweden
- Bartosz + Piotr (Tieto)
- Gunnar
- Philippe
Minutes
Introduction
Philippe welcomes participants and details the agenda
Philippe recommends the review of the presentations delivered by Tieto and Windriver at the tech summit in Detroit where they presented their return of experience on Android Automotive, in particular on Audio HAL
please look here, search for "return of experience"
Introduction to Audio HAL
- Bartosz presents this slide deck
- We have more than 1 Audio HAL module in the system. There is always primary audio HAL that handles primary output (car speakers) and primary input (mic, aux, tuner, etc.) and in Android Automotive usually we also have USB audio HAL (dedicated to USB audio) and A2DP audio HAL (dedicated to Bluetooth A2DP). Different Audio HAL modules don't talk directly to each other, but the routing/bridging between them is handled via AudioFlinger/AudioPolicy based on audio policy configuration (where routing topology is defined).
- slide 6: grey boxes are external to AA
- Wassim: question on slide 14 about configuration
- Bartosz: configuration files are in the system partition or in the vendor partition, we lack a good concept for configuration, and for the persistence of configuration as well, support tools would greatly help
Gathering of list of topics for the Audio HAL project
- Bosch: we have issues with the BT audio stream / BT handsfree but I am just starting digging into it
- Gunnar: I am surprised that the BT handsfree is not part of the Google AA code
- Bosch: there is some support in the framework to support handsfree
- Gunnar: is the basic hansfree profile implementation part of the BT support in AA ?
- Bosch: pairing the phone with the device works but there is a need to connect the audio
- Andrey: shows a list of questions / problems, look at the list below
- Android does not implement all features required by customer.
- Audio focus can be lost at any time
- Limited number of sources
- Limited number of priorities -2
- No easy way to support external amplifiers
- No priorities of phone call types.
- Android Audio subsystem is developed only for infotainment purposes. Safety-related features need to be implemented in another RTOS
- Need to share the same hardware between 2 OSes
- Running Android as a virtual machine inside RTOS leads to problems with scheduling of audio processes
- Android way of extending its functionality is developing vendor extensions without modification of the framework.
- But in some cases you need to modify framework. There is a process for this, but slow. And changes needed to one customer / developer is not needed for another. Even if you do this, you have to wait until Tier 1 merges this change to its drop
- You have to retain compatibility for third-party apps
- Android does not implement all features required by customer.
- Gunnar: what are the most important topics to tackle on ? maybe there is a particular problem to look at first
- Wassim: there is some topics that are low-hanging fruit
- Stefan: IMHO some of the topics are not related to Audio HAL but rather system-level architectural issues
- Gunnar: Audio HAL is much more complex than the Vehicle HAL
- Gunnar: in the Vehicle HAL we are finding solutions that can sit aside from AA, we do not intend to make a fork which would provoke a lot of
integration problems - Stefan: priority of sources varies among OEMs, we would need a better configuration capability from Google to handle this better
- Bartorz: extending the framework is fine provided you do not change the APIs, the HAL is actually something where we can put additional stuff "for free"
- Bartoz: this is a much bigger challenge to work out a common Audio Hal and port it to different SoCs but it is interesting to do
- Gunnar: agreed
- /TODO/ all participants identify additional extensions to Audio HAL that would not change too many things
- /TODO/ all participants identify a part of the Audio HAL that could common for all SoC vendors
- /TODO/ Gunnar and Philippe create the wiki page to gather the priorities of topics of interest listed so far (from minutes of first meeting and presentations delivered by Tieto and Windriver at the tech summit)
Adjourned: 18:20 CET
7 November 2019 - Project Scoping - First Call
Participants
- Andrey (Harman) principal architect, Android expert, based in Nijni-Novgorod, Russia
- Denis (Harman) manager of Android COE team (Center Of Excellence)
- Alexander (Harman) workis in the COE for low level Android
- Suhasini (Analog Devices), automotive infotainment software engineer, based in Bangalore, India
- Piotr + 1 (Tieto) mobile devices, automotive HW layer of Android, based in Poland
- Maria + 2 (Bosch Car Multimedia) manager and two SW engineers
- Gunnar (GENIVI)
- Philippe (GENIVI)
Minutes
- Philippe reminds the objective of the call (or of the first series of calls rather) which is to define the scope of a Audio HAL project
- Suhasini: we have not much experience with Android Automotive (AA), we are interested in AA because AA is now in the head unit
- we have a portfolio of audio processors and an automotive audio bus to distribute audio within the car, we want to configure our proprietary audio bus SW stack to use it with AA as we did for Linux
- we want to determine whether we have a problem of bandwith when using our proprietary network with AA, one approach is to use shared memory transfer
in the audio HAL
- Piotr: we identified the following list of problems / tasks with AA when integrating the HAL for silicon vendors and OEMs
- configuration of Automotive overlay, there is not much tool to configure the AA system
- there is no way to control the equalization, i.e. no simple way to control global effects for output streams (by default Android application controls its own tracks)
- there is no Audio calibration interface
- there is a need for a generic interface for controlling audio effects on HAL level, global effects are designed for input streams but control over them is limited by interface
- early audio for RVC (Rear View Camera) or other services
- configuration management component for TinyALSA
- Audio Focus doesn't forbid to interrupt Audio, Android 10 provides additional interface for Automotive to solve this problem
- Suhasini: we would like the following topics to be investigated
- Audio data transfer / streaming to a co-processor for post processing of audio from the HAL/other Android layers
- Support for the configuration of networked audio devices. Currently Android can view it only as a single device/sound card. An audio network in car might have multiple devices that needs to be controlled by the Android OS.
- Bosch engineer: we are working on two things, with the audio HAL we received from our silicon vendor
- 1- patches for handsfree management, combination of BT stream with other streams, how to enable the correct audio routing, documentation on the Audio HALis missing, it is difficult to find examples on line
- 2- I/O for our Bosch HW, the challenge is to adapt the AA framework to our HW, which has for instance 4 audio channels while we need to present them as 2 audio channels to AA
- Gunnar: how many of you are familiar with the Audio Manager of GENIVI ? I would recommend you to look at the features supported by GENIVI Audio Manager because it has been designed for automotive audio specifically
- Gunnar: introduces shortly the Audio Manager daemon and plug-ins
- Harman engineer: using Audio Manager am implementation designed for linux would raise an issue because of the Google CTS
- Andrey: safety related audio features can likely not be implementted in AA because AA is not a real-time OS
Next steps
- Gunnar: @Harman - would you consider making a list similar to what Tieto presented ?
- Suhasini: having an overview of the AA audio HAL would help ? can someone give such a presentation ?
- /TODO/ Harman & Bosch provide a list of topics that could be worked on jointly
- /TODO/ Philippe create a wiki page DONE
- /TODO/ Piotr prepare a short presentation on the AA audio HAL so that we share the same understanding of Audio HAL features
Next call
- we will have calls every other week, on Thursdays at 11:30 CET
- next call is scheduled on Thursday 21 November, a calendar invite has been sent