PoC Milestones and Work Breakdown
AASIG (Audio-HAL) Workshop at 20th All Member Meeting (agenda planning)
List of prioritized topics for the Audio HAL
Multiple-zone audio management - System Level Audio
Next Meeting - TBD 2021 (summer vacations pause) - 11:30am CEST
Click to Join Webex meeting
or join by entering the meeting details manually:
- Meeting number (access code): 297 637 101
- Meeting password: XCtdZmvs248
Or join the meeting by phone with
- Access code: 297 637 101
- Meeting password: 92839687
Agenda
- Discuss design of audio→external system, WebRTC based demo and "desktop audio development environment"
Thursday 15 July - 11:30am CEST
Participants
Gunnar Andersson Unknown User (wassim.filali) Suhasini Raghuram
Apologies
Piotr Krawczyk Philippe Robin Stephen Lawrence Unknown User (niskandar)
Minutes
- RPC interface. What is in there? Is it volume, settings, ...?
- Gunnar: Shared work to define this. Let's set up a Wiki page to list all the required methods and get/settable properties.
- Gunnar: I have also mentioned the possibility to use the CVII:VSC IDL to describe the interface, but this is a secondary goal and not where we need to start.
- Wassim shows a lot of interest in this, the development and use of a standard IDL, and has experience working on topics of IDL, format-conversions, documentation, etc.
- How to transfer PCM data between "Host" and VM? Is it the socket approach as discussed before?
Thursday 8 July - 11:30am CEST
Participants
Piotr Krawczyk Gunnar Andersson Stephen Lawrence Philippe Robin
Apologies
Unknown User (niskandar) ( + 1 more week )
Minutes
- Discussed content of AMM demo See PoC Milestones and Work Breakdown (AASIG, AHAL, audio-control)
- Piotr indicates the Android side can be done in time. On host side we expect to have something, scope to be determined.
- Piotr to send description of project/plan to aasig mailing list and ask Wassim for input on the host-side of the plan.
Thursday 1 July - 11:30am CEST
Participants
Suhasini RaghuramPiotr Krawczyk Gunnar Andersson
Apologies
Unknown User (niskandar) (+ 2 more weeks)
Minutes
- The full diagram of discussed plans has been posted by Piotr.
- It includes Audio-offloading and desktop development environment ideas, as discussed.
Thursday 24 June - 11:30am CEST
Minutes
- Very short meeting due to apologies and absences
- First draft diagram by Piotr coming via email. To be shown and discussed.
Thursday 17 June - 11:30am CEST
Participants
Unknown User (wassim.filali) Suhasini Raghuram Philippe Robin Piotr Krawczyk Unknown User (niskandar)
Minutes
Discussion about the work
- The main summary is that we need more people to implement
Discussion about PoC Milestones and Work Breakdown (AASIG, AHAL, audio-control)
- Thanks to Wassim for making this page somewhat up to date
- The talk is more oriented towards the big picture
- The big picture: Shifting from Android as device centric based operating system (like with the smartphone) to car based operated system
- Then other topics would be in that direction
- for example extracting the audio, effects, offloading, control, Bluetooth source, etc.
- we shifted from what's needed for the features that we want to implement to what is needed to implement that direction
- Where would be have development and programmers?
- We worry less and less about what Android is doing when we go to the direction of the border of the OS
Concrete steps for next week. 2 major tasks:
- Making the documentation and diagrams of what we intend to do (check the WBS)
- Track related features in new android versions to check that there are no surprises or something that would block us
- Please check first with your NDA not to break it
Goal of the development
- Should be more concrete than the big picture such as
- configuration of audio zones
- environment for building the framework inside the system
- environment for building the framework outside the system
- Piotr: I would start with high level design for these tasks first
- like controlling audio zones, volumes, context
- design of trout based development (from an architecture point of view of course this is not important)
- but does trout support audio?
- it's a nice basis because the framework will not change, so the development can be taken to other HW easily. for example in an emulator, the framework was changed
- cross VM is an advantage (it's used in Trout)
- closer to what we want to develop in future
- closer to real use case
- This is where the trend is going
- get a portability of Android with trout (but not other OS)
- we need to check what is the real platform that we would work with in the future
- we need to check the good reasons to select it
- Wassim: I would focus more on policy and logic management specially for our GENIVI group
- Focusing also on versions of Android in the background and what would they offer
Thursday 10 June - 11:30am CEST
Participants
Unknown User (wassim.filali) Suhasini Raghuram Gunnar Andersson Philippe Robin
apologies: Unknown User (niskandar)
Minutes
Discussion on the workbreakdown structure, PoC Milestones and Work Breakdown (AASIG, AHAL, audio-control) updated online
Thursday 27 May 2021 - 11:30am CEST
Participants
Unknown User (wassim.filali) Gunnar Andersson Philippe Robin
apologies: Unknown User (niskandar)
Minutes
Discussion on the project organization and workplan and on how to foster engineers' engagement in day2day work, outcome of the discussion will be reported in the next call
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Gunnar Andersson Suhasini Raghuram Unknown User (niskandar) Stephen Lawrence Philippe Robin
Minutes
Small discussion about the AMM
- Interest in virtualization, trout,
virtio
Discussion about virtualization, virtio, networking virtualization, TSN area in the hypervisor group
Discussion about the implementation
- Have we moved away from topics that are interesting for Analog Devices? A2B connected technologies
- Maybe we can get back to these topics when we are designing hardware POCs for the virtualization
- There some interests of developing Linux drivers for A2B not only Android
- Also standardized way of connecting A2B
- Idea of diagnostic protocol as a carrier for end user functionalities
no call on Thursday 13 May (due to a bank holiday)
May 4-7 Virtual AMM - AUDIO HAL sessions
- 5 May - technical update: recorded session (please go to AUDIO HAL section), Android Automotive Audio slides
- 6 May - technical workshop #1: Recorded Session (please go to AUDIO section), PoC presentation
- 7 May - technical workshop #2: Recorded Session (please go to AUDIO section)
Thursday 29 April 2021 - 11:30am CEST
Participants
Philippe Robin Gunnar Andersson Piotr Krawczyk Suhasini Raghuram Unknown User (wassim.filali)
apologies Unknown User (niskandar) Stephen Lawrence
Minutes
preparation of AMM last session - discussion on the topics for next 6 months
- virtualization => how can we minimize changes to Android (virtio, trout)
- simulation => how can we simulate HW acceleration in a seemless environment together an Emulated Android
- are we chasing after Android 12 or are we ahead of Android for the external source management in an automotive environment ?
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Philippe Robin Gunnar Andersson Piotr Krawczyk Suhasini Raghuram Unknown User (wassim.filali) Unknown User (niskandar) Stephen Lawrence
Minutes
Synchronizing from last weeks' status
- Wassim: We presented some suggetsions and discussed how they can help, what are your impressions?
- Piotr: Audio Weaver was specially a quite interesting topic and it seems widely used.
- Wassim: you can use Audio Weaver for free with https://elinux.org/R-Car/Boards/M3SK development kit
- Wassim: you can use it on PC but it's based on Matlab
- Wassim: the goal is the smooth transition between the demo/emulation, and the product. Concept is requirement as product model.
- In the future a researcher would just for example provide an effect model and this would be a requirement to the suppliers
- Specially that we are using Android. We can have a ready Android system, and a ready effect model. What is left is to connect them.
- Wassim: this is the picture posted here Car Audio System Emulator. I'll take some time to create ppt Genivi slides
more discussion about the demo but with WebRTC and Trout
- Piotr: It's a nice this audio solution (refering to the Car Audio System Emulator). To have the effects controlled with a model outside of Android.
- Piotr: In the end, we could have the WebRTC interface showing Audio coming out of Android and Effect controlled by the Audio Effect HAL and allows the user to apply additional effects on top of the effects.
- Wassim: yes and we could also have them decoupled: PC simulation where the decoupling happen but browser is GUI.
- Piotr: one question is about the LibPulse Audio
- Wassim: libpulse audio is simply a tool. What is more important is to show:
- if we have in the car many zones/speakers/etc.
- the user would see it on his browser the streams the channels etc.
- the user would choose what to listen to for example
- the architecture of the Car Audio System Emulator is independent of the PC but can be patched and used to control
- Pitor: further question: should the ECNR control interface be exposed? and to what level are they exposed?
- Wassim: as long as Android allows to do it, should be enough.
- Wassim: as long as something can be controlled from hardware, it should be controllable by software
- Wassim: imagine that every app has its own ECNR and multiple applications would need to run at the same time → this is why it's more interesting to have a system ECNR
- Pitor: common solutions don't utilize DSP firmware by default
- Piotr: if we talk about Trout , it's not an emulator, it's just an Android instance and most of the HAL will be virtualized. We are not connected to the pulse audio here for example. to keep with the same picture, we would do something different
- Wassim: completely okay, we don't have to use the pulseaudio server
- Piotr: we can think about not using ALSA, because of the virtualization.
- Piotr: this virtualization would be separate from browser and ALSA.
- Piotr: android emulator - virtual android
- Pitor link for trout and virtualization https://source.android.com/devices/automotive/virtualization
- Wassim: we didn't solve the control path yet
- Wassim: between the android and browser how would you see that? Json?
- Piotr: audio control already implemented in trout, let's check a solution in that direction. Since Trout already has solutions for several audio control
- Wassim: we need to look how easy it is to deploy the grpc on the browser
- Wassim: 3 demonstrations so far: raw audio with float sample packets, Opus Codec, WebRTC not yet fully running but it's client 2 client (looking like an overkill)
- Piotr: I would say for demonstration purpose, whatever is currently working would be accepted,
Jira ticket AASIG-124
- We need to edit the diagram and the names and rework it and that should be the demo concept.
- Audio weaver: candidate for virtual effect platform to start a professional demo. Later we can work with python audio effect but this needs more work but is open standard.
Genivi virtual meeting 04-07 May 2021
- Need to gather some topics for next meeting (Audio HAL CW 2117 - 29.04.2021)
- Webex is preffered as a meeting software.
- Last time some people could not join (because the meeting was hosted with Zoom)
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Philippe Robin Gunnar Andersson Piotr Krawczyk Stephen Lawrence Suhasini Raghuram Unknown User (niskandar)
Minutes
Overview of the last weeks
- Piotr: status from last week regarding the demo (part of the AASIG-93 but with a twist)
- Demo is working on Trout
- WebRTC is now being connected
- Confident to be able to show something in two weeks
- Expecting more commits in https://github.com/SoundHacking/web_py_loop
- Got more support about this task from Tieto → Output should be shown
- Gunnar: So where are we now exactly?
- Piotr: Next week we would have something to show and have a better overview of what we have and what is missing.
- Gunnar: we need to check with Wassim what was his vision in order not to have an overlap and re-use different parts → Todo
- Gunnar: Main idea was to create a development environment where we can try different audio solutions but the heart of this would still be Android → todo capture this in Jira
- Suhasini: 3 questions about the virtual meeting 04-07 May 2021
- What are the other slots used for (for example vehicle data related)?
- What will the Audio HAL do in the slot exactly?
- What updates are we planning on giving in the introduction
- Gunnar/Philippe: the slots are being now finalized we'll update you asap. For now:
- One slot is on Wednesday (05.05.2021) 5 minutes introduction
- Time slot 15min about the External Audio Demonstration
- Time slot workshop 1 Q&A Audio design discussion 25min
- Workshop 2 (Wassim moderating) on Friday (07.05.2021) 15:25
- Introduction shall be about what we have done, what we are doing, and if you like to join. Prerecorded video. At some point you will be contacted by Tom (Audio/Video/Networking) between 27 and 28 April 2021
- Fosdem update: we could bring it up in the introduction on Wednesday
- Stephen: DSP concept from last week. Asked internally
- There is HIFI support but it's proprietary
- Audio Weaver is supported in Renesas
Todo
- Unknown User (wassim.filali): To synchronize with Piotr Krawczyk about the vision and re-usability
- Unknown User (niskandar) : create from the past minutes of meetings a Jira ticket and add it to the sprint
Thursday 8 April 2021 - 11:30am CET
Participants
Stephen Lawrence Gunnar Andersson Unknown User (wassim.filali) Philippe Robin Suhasini Raghuram
Apologies
- Wassim has investigated webrtc with some simple implemementations
- Wassim: This could be transformed into a "proper" project. Maybe hosted on github/GENIVI?
- https://github.com/SoundHacking/web_py_loop – Gunnar Andersson to review if this fits into GENIVI repos.
- Documentation here: https://www.homesmartmesh.com/docs/sound/
- Wassim describes two designs, one on WebRTC, and another which is more decoupled and simply uses fixed parameters for the encoding. In an environment where the bandwidth is assumed to be more controlled and less variable, then the dynamic features of WebRTC are not needed.
- Both ideas are implemented and can be compared. All examples use the 00_backend (loopback) as server except for example 07 which uses 00_signaling server. Wassim to update readmes.
- Interesting tool https://w.dspconcepts.com/audio-weaver
- Development tool with target framework to execute the scenarios (on-board firmware)
- Audio experts make a flow using GUI tool, by connecting and configuring processing blocks.
- Not open-source. Likely business model is target license-fee for production use
- Some DSPs, are supported.
- For ST Electronics hardware, the license is included.
An inexpensive ST Microelectronics dev. board can be purchased and used directly: STM32f407 user guide.
(but there is also the option to run it on PC only) - ... discussion about which other DSP hardware platforms might be supported. This is not clear on the web site (need to contact the company?).
- Wassim says he knows it supports production grade DSPs, but apparently these are not public information(?), so we cannot describe it further.
- Stephen: I found that Cadence is listed, that's all
- You can generate the config/script and send it to the target systems.
- The target systems have an audioweaver DSP code.
- Documentation is available.
- Suhasini: Can the tool run the scenarios on the PC (a simulator/emulator environment)?
- Wassim: Yes, all "blocks" can be executed on the PC with a "x86 target"
- It might be interesting to support our "development environment" (described above and in minutes 2 weeks ago) integrated with this?
- Wassim: (bottom line) Whether it is exactly this one or another, this type of development environment would be ideal.
Thursday 1 April 2021 - 11:30am CET
Participants
Gunnar Andersson Unknown User (wassim.filali) Philippe Robin Suhasini Raghuram
Apologies
Minutes
- review of the planning of presentations at the upcoming AMM
- TODO Wassim and Suhasini will prepare a (5 to 10' mn) report on Audio HAL project status
- TODO Wassim will contact Piotr and check with him how to deliver a demo on External Audio management at the upcoming AMM
- Wassim: Android 12 is now being published
| Anchor | ||||
|---|---|---|---|---|
|
Thursday 25 March 2021 - 11:30am CET (AUDIO_HAL_CW2112)
Participants
Piotr Krawczyk Gunnar Andersson Stephen Lawrence Unknown User (wassim.filali)
Apologies
Unknown User (niskandar) Philippe Robin
Minutes
- Piotr discussing that the HAL implementation might just as well target Trout since it is progressing quickly
- Google's virtual platform for Android "Trout", is likely to implement VIRTIO-SND
- Trout is based on CrosVM (developed as part of Chromium). CrosVM is in turn based on KVM when running on Linux.
- Trout is great and will make HALs portable without/less rewrite (Piotr)
- ...but the complexity of hardware compatibility and porting then lands on Hypervisor and Virtual Platform implementations instead (Gunnar)
- (Note Automotive Virtual Platform Specification already requires VIRTIO-SND in the working-copy, to be released in new version soon)
- VIRTIO-SND proposal is merged to VIRTIO master branch but no VIRTIO v1.2 document exists yet. Seems very likely to be included since it is merged.
- VIRTIO-SND kernel driver exists but implementations on virtual platforms (QEMU) still lacking. Part way done on Trout it seems...
- It is being developed but we can't rely that it will be done for AMM demo.
- Plan is therefore to use a simplified VSOCK transfer instead, exporting the audio. (A step in the right direction since VIRTIO-SND implementations are likely to use VSOCK)
- The plan remains to start building a framework for external audio and as a first implementation set up a WebRTC server running on the "host" that can consume the data from the VSOCK connection.
- The usage of WebRTC also fulfils a desire from Wassim to build a flexible simulation/processing/tinkering framework that can be executed on developer machines by way of a Web browser providing the user interface. (i.e. not only production, but development tools support).
- Why WebRTC?? Summarized:
- Easy demo - just "send a URL to someone, they open it in web a browser". It is also convenient way to create a user interface
- Bigger context: Create an abstraction of target sound environment, which may develop into a flexible (desktop) development environment for exploring/developing audio related functions.
- (Gunnar added offline): The fact that "real" Android is still part of this abstract development system makes it relevant for developing and testing real audio functions. I imagine you could even do Hardware in the loop setups...
- But also: WebRTC might be a possible actual "production" protocol for audio connection to a remote server. (Example use case that needs this: off-board speech processing)
- Pulseaudio? Might be an easy solution to implement some of the control channel parts for the demo.
- NEXT : Need to start breaking down the design into more details, included software components
- 1) Find existing implementations of component parts (e.g. WebRTC – Unknown User (wassim.filali)mentioned there are implementations available, let's point to them)
- 2) What parts still do not exist - need to be developed
- And of course: Confirm the overall demonstration use case and plan to guide the breakdown.
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Gunnar Andersson Philippe Robin Stephen Lawrence Suhasini Raghuram
Apologies
Unknown User (niskandar) Unknown User (wassim.filali)
Minutes
- Wassim sent a link before the meeting. A very useful documentation page describing the principles for Audio Focus in the latest AOSP is this page:
https://source.android.com/devices/automotive/audio/audio-focus - Quick updates for Suhasini who is 'glad to be back'. Continued focus on external-audio track is good.
- We reviewed last week quickly and decided to do offline reading of the documentation and come back next week for more discussion.
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Chris Simmonds (part) Gunnar Andersson Philippe Robin Piotr Krawczyk Stephen Lawrence Unknown User (wassim.filali)
Apologies
Minutes
CI/Testing
Gunnar: Shortly about Go issues sorted out. Both Lava and Go.CD updated to latest versions. Go now behind NGINX proxy (it used to handle its own SSL/TLS but this is more standard). Some breakage to sort out -- large artifact transfers were being truncated, etc. NGINX settings sort this out.
StephenL: We have set up full CTS & VTS in Lava. Seems very large and running many tests. How could we reduce the tests to the most relevant. But in some cases the tests still seem not that comprehensive.
Does Chris have experience?
Chris: There ought to be a test for every API. Admit that some seem trivial however. Just call the function and see if there is a reply, basically.
StephenL: So for audio tests, it won't really test behavior
Chris: Yes, the tests cannot monitor the output, so some might pass even if no sound actually comes out, etc.
Effects / DSPs / binary blobs
Wassim: I looked at offloading to DSP in Android 11. I can find the effects headers. Where is the implemntation -- DSP binaries?
Chris: AOSP should ships some basic effects, with source code. (algorithms on general CPU only, since DSPs are very specific). DSP code is likely to be binaries without source code, and therefore can't ship with AOSP.
SL: Agree, silicon vendors ship binaries and often a DSP development kit to write code, and so on. But it's likely even 3rd parties are involved and selling implementations of special effects/processing.
In summary: Binary implementations come from elsewhere, either as part of BSP delivery from Silicon Vendor, or even third party.
Design and demo of external audio system
Wassim: We haven't discussed using WebRTC before, let's We'd like to create an isolated/simulated environment (e.g. dockerized). Demonstrate how to play/control non-local audio.
Proposed Idea: Use a browser as the receiving audio device. (And UI)
Web audio API - volumes, effects, filters, extract channels... even different zones set up?
You could forward to WebRTC inside Android. Android WebRTC -- see link TODO It seems easy to stream to WebRTC. (but that's not quite the same, see below)
Wassim shows example WebRTC user interface with volume controls etc.
Piotr: What is volume control controlling here?
Wassim: It happens on the local machine.
(discussion: it should affect the Android based source instead?)
Implementation alternatives
1. Reuse Android WebRTC implementation
2. Don't use WebRTC implementation from Android, extract audio as before (via plain socket) to some external server. This external server may also act as a WebRTC server and could be implemented using any non-Android implementation (i.e. likely Linux, and likely packaged in Docker of course). The WebRTC server sends audio over WebRTC but also hosts the web page of course, that is the client user interface, shown in "user's" browser)
Wassim and Gunnar prefer alt. 2). This simply extracts audio from Android and the fact that WebRTC is used is a non-Android related detail in this demo setup.
Piotr/Wassim:
Discussion on full round-trip, e.g. user changes volume or other setting on user interface - this might affect audio server, but in some cases need to be fed back also into the Android system.
Gunnar/Wassim:
Web page as user interface would be like a remote-control case. Agreed. Wassim notes in that case Android could run "headless" for the purposes of the demo.
Gunnar: A more car-like setup is that the user interface is still provided by Android, only the audio processing and playback may be on an external system.
Who is skilled at web development? Wassim is. Piotr would need to spend some time learning more.
Wassim: Can we try running the emulator headless (in a container)?
Piotr: I would expect it will fail when not having the graphics access.
Wassim: I'd prefer a simple remote-server setup, ideally a collection of docker containers.
... (discussion) running full emulator UI from docker might have issues (theoretically possible, but why?)
... might be OK to just run emulator on host without container - but it should be just a simple launch
... note that the emulator is in fact a virtual machine.
Alternative - Android in container has been shown before. Is it possible to build on it? Probably not based on automotive build out of the box There Android executes directly on kernel (container) and not a VM. Links?
Demonstration scenarios
- User interaction is desired on Windows (typical corporate environment)
- User interaction could be with the Android UI --> requires the emulator to be running on the Windows client machine. The audio/WebRTC server has been assumed to be on same machine also, but since the audio is over TCP/IP it could theoretically be elsewhere.
- User interaction could be via Browser only --> no requirement on where the Android system + WebRTC server actually runs. Deployment on some Linux server might be nice if it can be done headless.
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Gunnar Andersson Philippe Robin Piotr Krawczyk Stephen Lawrence Unknown User (wassim.filali) Unknown User (niskandar)
Minutes
Introduction from Philippe
- Have a look at the detail from Jira and update the tickets
- Check the meeting in May - Virtual all members meeting
- 4th of May until the 7th of May: starting to gather/fill the content. Provide an introduction and demonstration of particular topics
Jira going through the Audio HAL board
- AASIG-70: needs a bit more time (a couple of days) to be updated.
- AASIG-75: Design is in progress but there are more relevant questions first
- What is the expectation here? we were still with the emulator idea? What is the purpose of this ticket?
- Detailed discussion
- Communication between the host and the emulator
- Projected mode and Media Sync Service
- Networking between host and emulator
- ECNR - Echo Cancellation and Noise Reduction - before or after HAL
- Can be used in bare-metal Android solution, Also Android AEC
- Summary, 2 different points
- Own Microphone to Android: goes through Audio HAL
- External Microphone to Android: this can go through the "custom sync service" → offloading topic
- AASIG-92: we studied it and we understood that we need an AVB stack or A2B but do we do more investigating here?
- AASIG-104: Wassim's opinion x86 is real heardware and this for now fully okay instead of targeting a hardware because anyway the target hardware chosen here will not be used in the car
- News about the hardware lab: There are functional boards with RENESAS
- This might be interesting when we pick up the topic of the audio latency
- removed from the sprint
- For the Virtual All Members Meeting
- Idea from Piotr will be first analyzed by Piotr and then presented in the next week
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Minutes
Thursday 18 February 2021 - 11:30am CET
Participants
Unknown User (wassim.filali) Stephen Lawrence Philippe Robin
apologies: Gunnar Andersson, Nadim Iskandar, Suhasini Raghuram, Piotr Krawczyk
Minutes
Discussion on AVB/TSN and hypervisors and virtio between Wassim and Stephen; will continue next week
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Unknown User (wassim.filali) Gunnar Andersson Unknown User (niskandar) Stephen Lawrence Philippe Robin Suhasini Raghuram Piotr Krawczyk
Minutes
Update regarding the platform
- we got the original manifest
- it looks simple to integrate into the scripts that we have (this is the next steps to be done here)
- As a reference here is a link to the different repositories
- This new repository, aasig_local_manifests is for local manifests (to define hardware target specific files, and feature variants such as audio, vhal, etc.)
- This repository, aasig_dev_configuration has the build configurations (mk files) for vhal and audio variation
Discussion about the stories
- Main Question. What is more interesting, how do we go futher?
- Networked Audio (AVB Workshop and Discussion)
- Piotr: maybe if we can study topics outside of Android to offer solutions that are not on the plate of Android. This could be even added to Android.
- Android control (Android and System Level Audio)
- How can we deal with this? how much custom code do we need?
- Wassim: I think here there are more hanging fruits
- Both approaches shouldn't block each others.
- Networked Audio (AVB Workshop and Discussion)
Jira showSummary false server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-119 - Overview about the issue and the goal
- there are some alsa and tinyalsa plugins inforamtion that we can check
- Once we understand the limitation (questions in the ticket) we can close the ticket, this would need to look into the links gathered
- DoD would be a kind of a presentation that could allow us to start a conversation/discussion
- maybe add an entry into confluence and turn it into a presentation
- Discussion about the
Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-93 - Regarding the whole concept of the AVB, do we have a demo to show that the concept works?
- Piotr: i'm interested in showing this in a complete stack. So an approach to form a complete solution, first we start with a diagram to define:
- what would be the interfaces with Android
- what would be the alsa gateway
- what would be the AVB software stack and alsa solution
- etc.
- if we can create a diagram of all of this, it would give us an overview to allow us to create a complete demo about the solution.
- There are many points to check and explore such as different suppliers or silos for different parts of the stack
- What is the definition of "Offloading"
- Shared screen of Stephen
- just to check if such documents are interesting to continue the discussion of the AVB approach
- This will help towards the ticket
(creating an overview)Jira showSummary false server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-94
- Feedback from FOSDEM
- A lot of participants 300-400 and Q&A was successful
- Thank you for the guys who participated
- Suhasini will be missing the next 4 meetings
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Unknown User (wassim.filali) Gunnar Andersson Unknown User (niskandar) Stephen Lawrence Philippe Robin @Chris Simmons(2net)
Minutes
Discussion about
| Jira | ||||||||
|---|---|---|---|---|---|---|---|---|
|
- Would there be any limitation? maybe apart from ASIL limitation
- References were added to the task detail
- Configuration options range from plugins or configutation language (xml configuration file for examle)
- XML configuration file disadvantages:
- one person deciding what can be configured (limitation by the developer)
- static or compile time let's say
- Configutation with C++ policy advantages:
- dynamic loading of the classes according to (I opened the roof of the car, now I want the configutation to be changed on run-time)
- more freedom to configure
- extendibility
- It would be easier to implement conditional programming (if the roof is open)
- XML configuration file disadvantages:
- So back to the original discussion of this task: we need to check if Android policies are good enough or do we need to extend our own
- But the decision to use XML and C++ policy is a different decision
- There is already a bit of documentation in the implement policy on android
- We need to get a real configuration file to see what it would look like (a simple file but with different examples) such as roof open, driving mode change, etc.
- This is different than the audio focus matrix defined in the multi-zone audio focus. We could modify CarAudioService.java
- We could modify more files but then Andoird would be not certified. There is of course advantages and disadvantages but let's check our options that keeps us certified first.
- We need to check if the Audio Focus matrix is modifiable (check with Unknown User (bartoszbialek))
Discussion about the AVB topic
- as mentioned last week, there are still open points like how do we integrate ALSA devices, how would be do cross ECU communications, etc.
- The problem is, is there something more we can introduce?
- Wassim: imo, there are no longer low hanging fruits in this topic. We either go further or we drop the subject.
- It depends if there is someone that would like to try something and go further
Going futher
- We need to continue with this concept Android and System Level Audio, to check which option is better going further
Looking at the current sprint (AASIG-92)
- audio transfer to external ECU
- There are still some problems with the definition of Offloading
- There are some overlapping and similarities
- There is still a question of considering the Audio HAL as part of Android or not
- The way you control your dsp on the Audio HAL is completely outside of Android
- Piotr maybe meant that since we decided that the audio effect Audio HAL is outside of Android, what are the APIs that we can interact with
- Any android has Audio HAL implementation but the different is that the audio effect HAL expects the audio to be back (not like an amplifier where the audio is just leaving Android and not returning)
- also a good definition of Offloading would help
- implement Input
- implement Raw Stream extraction
Todo
- Unknown User (niskandar) : Add components to our epics and stories to diffirentiate them
- Philippe Robin : Could you please create a quick filter on the board for tickets with componen in (AudioHal). That would make it easier to navigate our dashboard. DONE please AASIG AUDIO HAL Board
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Suhasini Raghuram Unknown User (wassim.filali) Gunnar Andersson Unknown User (niskandar) Stephen Lawrence Philippe Robin
Minutes
Discussion about the AVB and next steps or topics to do (but we cannot reach the conclusion without Piotr)
- let's refer back to AVB Workshop and Discussion.
- Networked audio devices: detailed good enough the AVB (Topics and priorities)
- AVB is not a solved problem but there are no more comments
- If we say Android needs to talk to ALSA devices, and AVB is a network between them, then the networking problem is solved
- If the AVB is sent to another ECU, then we also need the control → point number 2
- The focus should be on how cross ECU should work
- The third points: Audio Effects should have two parts, hardware audio effects and virtual effects
- Not sure if we are taking notes only or should we write this in the epics.
- Gunnar: is it a solved problem in the industry? Is there a real solution that can be used right now in production?
- Stephen: from a hardware point of view, AVB had a lot of improvement
- Software side, in early day it was considered too complex but people are demystifying it
- What track should we look at from our side?
- Wassim: interest: point number 2 is a low hanging fruit. Point number 1 is a bit more complext and needs a hardware etc. Audio Effect would be the next priority
- Gunnar: Audio project is more about defining the functions and API for transfering and controlling audio between the sources and the sink.
- the point number 2 is more like a remote control (from a smartphone for examle). But this topic was in GENIVI many years ago as part of multimedia. But we should pick up existing work to make it happen again if we go in this direction.
- media and media control should be separated from pure audio design
- Wassim: we have media control and volume control. volume is managed differently in Android. so we could revisit the Media control but the interfaces are already present in Android. The question here is more : i have an external amplifier and i want to control the volume without loosing the sync with Android: this is an open point. → so point number 2 is more like an Audio Control subject. We can look at Android and System Level Audio
- Gunnar: but so we could work on an interface to have in the infotainment system that would have control over the master volume and other media controls
- Wassim: the image in Android and System Level Audio is a bit outdated. After some research, Android does provide an interface for the media control but not for the volume
- Gunnar: MQQT could be used as a carrier to expose some information to the mobile phone for a remote control of the media for example
- Wassim: yes but when we receive a signal, do we bypass Android or we would feed it in the Android service
- Gunnar: so we would need to investigate what Android is exactly providing in order to check if we need an alternative
- Wassim: yes the first step should be a deliverable review document detailing what Android can and cannot do regarding this control API.
- Agreed (from all)
- Gunnar: maybe we can also take more opinions
- Google partner would give a benefit of the pre-release notes in order to check what are the upcoming changes in the APIs
- Gunnar: then we have a direction to go → learn what the system can do.
- Wassim: First question that comes to mind: the car audio manager could have pre-defined policies. If we want we can add plugins to it. The question is: does the car audio manager allows C++ custom plugins or pre-defined XML files?
- Stephen: There should be something like this because of all the cases of early audio in Android systems
- TODO: check the Jira if the discussion was captured in tickets of Jira to work on them
- Created a Jira ticket for this. @All: please have a look and create sub tasks to provide feedback about the topic in parallel to Wassim
Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-119
FOSDEM presentation update
- Video submitted and is added to the presentation
- Piotr was requested to also be added as a host (but still no reply)
- The video is clear about the presentation and then Q&A session will follow where Suhasini and Piotr would join as hosts.
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Suhasini Raghuram Unknown User (wassim.filali) Stephen Lawrence Piotr Krawczyk Gunnar Andersson
Apologies
Minutes
Review of the backlog for audio HAL
- epic: audio HAL implementation - BT (
)Jira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-79 Jira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-75 Jira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-76 - we look at the diagram attached
- Piotr: native mechanism to connect to a BT device is shown on the diagram
- Potr: a second option is to connect the BT device to the platform directly and use the BT infrastructure and pass the audio data to Android
- there is a need to update the diagram
- discussion on the options
- discussion on accessing the hw
- accessing the dsp through 2 different VMs ?
- either you own the dsp on one machine only and you do not have a virtualization problem
- or you run the dsp as a service like a networked device
- Piotr mentions the work done by Opensynergy and Qualcomm with Google on virtualizing Android Automotive
- TODO Wassim and Piotr will capture the discussion on a tech note
- Piotr: added offline (in a email) the questions he raised during the call
- BT headset handling should be realized by Android stack or host OS - virtualized/containerized environment, I guess we agreed here that such case should be handled by Android and its BT stack
- how to properly utilize DSP for BT headset, especially in virtualized environment, DSP as a service?
- Google is introducing GKI (Generic Kernel Image) concept - how this could affect DSP and ECNR usage?
- ECNR for BT headset - how this should be handled, should it be?
- Jira ticket commented
Todo
- Unknown User (wassim.filali) Piotr Krawczyk capture the discussion from a technical point of view.
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Suhasini Raghuram Unknown User (wassim.filali) Stephen Lawrence Piotr Krawczyk Gunnar Andersson Unknown User (niskandar)
Minutes
Philippe: talking about the FOSDEM presentation, if some time is left we could mention the backlog tasks.
FOSDEM presentation
- The presentation will be recorded tonight
- Piotr already has a recorded Video
- Meeting done on Tuesday regarding this topic
- Brief slide overview done
- Differences between Android on phone vs Android in automotive
- Mention some of the requirements in different automotive branches (mentioning connected cars, ADAS, Powertrain, etc.)
- Aim is: can Android Automotive cater to all these requirements?
- Introduce design options (system controlled inside or outside of Android)
- Common problems in both design options, our starting points
- Extracting and injecting RAW strems (slides to follow with the POC and Demo) - This would be done by Piotr
- Audio network options (slides about previous problems in network: time, delay, etc.) and their solutions (standards solving the problems)
- suggesting Automotive Audio Bus (A2B)
- Further investigations (listing other topics - in our backlog) and what could be done as part of this aim
- Team currently working on this and more information to follow
- Live Q&A will follow with Matrix chat system, specifically here is the link for FOSDEM matrix
- In addition there will be a voice or real time interaction with the moderators etc
- Remarks:
- Philippe: license info is missing on the front page ("The content on this presentation is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License")
- Gunnar: link for how people could join us (link to the wiki)
- Stephen: we might have to switch to Element if we stop using Whatsapp (Element.io)
Tasks in the backlog topic
- New Sprint AASIG Sprint 6 created
- let's start by defining a road map for this year
- We soon will have some milestones from the board meeting
- Discussion about what to do first. Suggestions:
- Road map for the project with what is needed from a technical point of view. Create priorities. Fix the sprint according to resources.
- Check the resources and decide what current tasks are being worked on
- Piotr shared his screen to check the current backlog and epics.
- let's go through the issues or tasks that are in-progress and check how we can close them or postpone them etc.
- AASIG-70: This will be closed soon (Piotr)
- AASIG-86: Are there any artefacts that we can present, do we go futher on this task → moved out of the sprint to the backlog
- AASIG-92: This is more interesting than AASIG-86 but more complex. Maybe we can be break it down to studies and overtime consider how to implement it. The first step seems doable though. The DOD (definition of done) is to define the API and protocol. maybe let's break it into sub-task to add DOD in the sub-task to show a diagram for example or a design, etc. → moved to the backlog.
First let's define the sub-task (in the next session AUDIO_HAL CW03)
- Proposal for the structure of the source code as seen in Android development repository structure (overview was given by Piotr Krawczyk)
- we can have several repositories with manifest, with aasig_dev_configuration, and several repos depending on the services (vss, external audio mixing, vehicle plugin, etc.)
- downloading and building would be easily done.
- then we are modular and we can define the components that we would like to add to the build easily and we would share common parts.
- Together with Stephen: prepare action steps to prepare and implement this structure
- Gunnar will create the repos and support
- We also need to combine this with RENESAS build steps
- @All: Please have a look at the backlog to be prepared for the next meeting.
Thursday 17 December 2020 - 11:30am CET (AUDIO_HAL CW50)
Participants
Gunnar Andersson Philippe Robin Suhasini Raghuram Stephen Lawrence Piotr Krawczyk
apologies: Wassim Filali, Unknown User (niskandar)
Minutes
How to get help on the Kingfisher board
- kingfisher board is a 3rd party board, Stephen browses through the doc available on this web site below
- H/w manuals for Starter Kit boards are in link below
Fosdem
- CfP is there: https://fosdem.org/2021/news/2020-12-12-call-for-participation/
- Piotr and Susahini will prepare an abstract and submit by the end the week WORK-IN-PROGRESS
Title : Networked Audio in Android Automotive
Abstract:
With the evolving infotainment applications in cars, it is no longer sufficient to have just point to point analog connections between the various parts of the audio subsystem. And thus, OEMs are now switching to networked audio options for the infotainment systems. As Android is increasingly popular choice for the head unit operating system, there is a need to support networked audio devices in Android Automotive.
Piotr: I would mentioned here that there could be more audio sources in system that just Android and challenge is to make Android part of a complex system, maybe something like: …modern vehicle setup is built with number of networked components…
The GENIVI Android Automotive Special Interest group chose to work on addressing this. This talk highlights the work we have done so far and what we intend to do next.
Piotr: And here point out that we also tries to find out how mature Android in Automotive is: … GENIVI AASIG analyzes various scenarios of integrating Android in this complex setup and analyzes maturity and gaps of Android Automotive solution in this context.
The first problem the group identified was extracting raw PCM streams out of android context. The talk will discuss the implementation of this and the various design tradeoffs and decisions that were taken.
Piotr: Instead of first problem I would say first step like: The first step group took was to extract raw PCM stream out of Android context…
The next step would be to control a network from Android. Networks like AVB and A2B was considered for this. The talk will also highlight the work done so far for supporting A2B and what the next steps would be in getting the audio from the RAW PCM streams into A2B network.
Piotr: Maybe as a next step would be to connect actually Android to audio network. I would mention here also challenges like applying audio effects (ECNR, calibration, user EQ). In far future there will be also challenges regarding voice control. So something like: The next step would be to connect Android to audio network. Currently A2B and AVB are being investigated. Further investigations includes topics like: ECNR handling, calibration mechanisms, how to utilize DSPs in various system configuration (virtualization, containerization, multi HW)
Call schedule
- no call next week, the week after and during the 1st week of January
- next Audio HAL schedule is on Thursday 14 January
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Gunnar Andersson Philippe Robin Unknown User (wassim.filali) Suhasini Raghuram Stephen Lawrence Unknown User (niskandar)
Minutes
Virtual open source conference. GENIVI was recommended to give a talk (Fosdem: biggest open source conference). If we submit anything to the Embedded Mobile and Automotive dev room, we need to have strong code examples.
- Let's evaluate if we have a proposal from the Audio_HAL.
- We don't have to compete with Vehicle-HAL we can submit together. But we can submit our code and see if it's accepted to be presented
- We would need to talk to Piotr to check the current situation in Audio_HAL
- The last date for submitting the abstract is CFP deadline: Friday, 23rd December 2020.
- Description of the event: pre-recording video most probably (news Call for Participation)
- Summary: focused on code (link to github) but other related topics can be added.
- Potential candidate is the proof of concept on github.
- It is important to have something to show, to increase the visibility of this project and potentially get key resources to the Audio_HAL
AVB subject pickup
- From the analysis point of view we are done
- Let's see if we can talk about implementation
- if OpenAvenue is providing the AVB a complete Network Audio Synchronization out of the box, then there is nothing to do, but if not, then this is a potential improvement
- It can also be useful even if it's just showing the integration of the solution to Android.
- AVB can run on software only so the integration can be an implementation target.
- What about working on the integration to show the code before 23rd of December?
- it's gonna a bit hard to almost undoable
- A2B has a pre-liminary Linux driver that but we if we can configure this and prepare it for integration that would be a topic for the Fosdem
- is A2B compatible with I2S?
- yes I2S and I2C
- But sort of an emulated I2S and I2C
- is the A2B its own hardware transceiver? yes
- in summary: A2B is a bus transfer of I2S and I2C
- is A2B compatible with I2S?
- Another thing that we can try:
- I2C and I2S lines out of the Android platform and we can have A2B on an external board
- we connect the I2C and I2S to the A2B toolkit to control it
- Android (having: I2C + I2S_1 I2S_2) <==> A2B devkit <==> A2B remote1/remote2
- How does A2B operates if i want to send a stream to both remote 1 and remote 2? I2C has for example Left and right flag
- you could put it in bus slot and configure the slot to be picked up by remote 1 and remote 2
- bus slot is an A2B concept (converted data from I2S to A2B)
- the transceiver of A2B would then multiplex the data to different bus slot
- Yes this idea falls into the scope of Audio_HAL
- Could we take an ALSA device into the Linux driver?
- The Linux driver in the A2B is already supporting ALSA
- Details about the Bus and audio attribute (Unknown User (wassim.filali) + Suhasini Raghuram)
- We are capable in Android to map a bus according to the attribute
- So each of the remote can become a bus on its own
- it would be useful to send every Audio usage can go to different remote
- Isn't AVB a p2p?
- we can use AVB on a network and work with mac addresses
- AVB being derived from the network, it can do what a network does
- You have two technology to send audio through the network:
- One node to another (A2B)
- Network (AVB)
- so our discussion of the AVB in the Breakdown rework AVB page can have two solutions (AVB and A2B)
- These are the most attractive technologies to work on it
- We can propose this idea as a stepping stone for building a network Audio in Android (A2B + I2S/I2C)
- We can start with small code base and compliment it with plans for the future
- The only problem is that we are not sure how open the Android/A2B topic is
- The software part would be open but the development board (Android + A2B) is hard to get
- Actually this is an advantage point not an issue. We know that the hardware is needed but if we can show that our solution works with Software only, then it's very good
- Devkit software: free but no opensource → there are plans to make it open source (it's in the roadmap)
- Summary: There is an opportunity for Suhasini Raghuram to create a plan, presentation, and show implementation in the Fosdem (if we are able to meet the deadline 23rd December)
- repo + license should be ready
- This is nice because it might trigger the curiosity of some developers
- Need to check from Suhasini Raghuram if the open source part can be done
- Can we check that the presentation could be ready before 23rd of December but the code should be ready before the presentation
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Gunnar Andersson Philippe Robin Piotr Krawczyk Unknown User (wassim.filali) Suhasini Raghuram Stephen Lawrence Mohan Karthik Unknown User (niskandar)
Minutes
Continuing the session about Network Audio Synchronization and answering questions
- Question: is it possible to use AVB over standard Network adapter without HW support?
- No, not for high fidelity Audio automotive or pro equipment
- Question: what is the PTP precision?
- Theoratically it's 5ns but 5ns is not needed
- in 48 kHz sample, we would need a sample every 21us → to be out of sync we would consider 1-2 sample misses → 50us difference is then out of sync for 48 kHz sample.
- so if we can achieve a PTP precision of 5us then it's already very good.
- Question: do we need actually the 5ns?
- let's say we work with a packet of 6 samples, each sample is 21us → time of each packet is ~125us
- 5us is around 3% of overall observation so there will be jitter
- and we can calculate a tradeoff
- Question: Dilemma Latency vs Accuracy
- For example: user clicks on button, beep sound should be heard.
- but between interrupt and packet sending, the OS would add more delay (becaues the interrupts cannot send packets directly)
- so to reach the amplifier, the packet needs 3 ms to arrive
- But it can be that the amplifier is still processing another packet because the other packet is big → this creates latency
- but sending big packets also means that we can pack many samples in the packet and this would bring us more accuracy because the samples in the packet would have correct timing
- so it's a tradeoff between packet size and latency
- This is a big problem for Real Time systems such as:
- AEC - ENC (voice cancellation, noise reduction)
- What Latency would we need?
- for instrument 2ms
- smartphone playing 10ms
- streaming 1s
- phase manipluation of the speakers in different audio zones 1ms
- Lip sync? is the latency here not needed?
- here we divide the problem into two parts:
- 1: start of the video after user clicks: 500ms → this is enough time for the SW to synchronize video and audio
- 2: during the video and audio playing and user plays forwards or backwards
- 3: Humans to notice the audio, video difference: 20-40ms
- Question what is the top latency we can achieve in Android?
- 10ms in considered very good
- Latency is latency of transmitter + latency of network + latency of receiver
- For Example a class A limitation is 2ms P2P assuming a maximum of 7 network hubs
- But this is done with ovservation test so no real processing done in transmitter or receiver
- Question: PTP precision again
- software can be delayed regarless of the clock. so the bigger problems are not in the PTP precision:
- HW support: jitter of 5ns
- SW layer support (protocols on top of PTP: us jitter
- OS support: ms jitter
- Question: next steps? what would be the next steps regarding the SW? what to chose for the prototype?
- A good starting point is Open Avenue
- Linux AVB online documents can be found here (https://tsn.readthedocs.io/avb.html)
- Linux ptp stack (https://github.com/openil/linuxptp)
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Gunnar Andersson Philippe Robin Piotr Krawczyk Unknown User (wassim.filali) Johan Suhasini Raghuram Stephen Lawrence Mohan Karthik Chris Simmonds Unknown User (niskandar)
Minutes
Wassim giving a session about Network Audio Synchronization (many additional information came from Mohan Karthik)
The Problem
- The problem is that different ECU in the audio network have different clocks.
- The most accurate clock is one that looses 1 second every 15 billion years.
- if we had such a clock then there would be no problem in just synchronizing the clocks in the network.
- In current ECUs, the clocks are loosing or gaining 10us every second. So in a 44.1 KHz sample, it's around 1 sample loss every 2 seconds.
- These clocks have an accuracy counted in parts per millions
- Q&A that for humans to notice the difference, we would be talking about a different in parts per 100 not parts per millions
- Link to the Oscillator design can be found here
- These clocks have an accuracy counted in parts per millions
- This will create asynchronous time inside the system and could lead to noticeable sound problems
- One solution to synchronize the clocks is using software.
The possible solution: Software
- By implementing the PTP protocol (Precision Time Protocol) we could synchronize the clocks inside the audio system.
- for example Tutorial on Time-Synchronization for AAA2C based onIEEE Std 802.1AS™-2011 is more explained.
- The main idea is
- the sender and receiver should know what is the delta time between them
- This will allow them to synchronize the packages sent
- The PTP would allow the calculation of the delta time by sending specific values within the packages for the sender and receiver to use
- PTP however does not:
- prevent problems with Latency
- prevent problems caused by Jitter
provide a unified clock for the whole network. Stephen Lawrence : thank you for linking the correction: PTP master/slave clock sync with redundancy
- So another solution on top of PTP needs to be added to solve the other issues.
Some examples of the solutions mentioned
- Clock Recovery
- If I just sync my clock, I would lose the samples that were supposed to be playing now (already passed) → leads to cut in the sound
- Solution: Clock recovery not clock sync
- This means that we slow down or make the receiver's clock faster in progressively in order not to loose the samples, until the clocks are synchronized.
- Here we talk about a Skew in the clocks
- Resampling
- What if the clocks cannot be synchronized?
- Then we can do a resampling which will take the sample from the sender and resample it to the resceiver's clock
- This is however a very hard subject and there is currently a quest for the perfectest resampler
- This is caused by the new harmonies created by the resampling or in general the distortion of the original signal
- It also needs a lot of CPU power to be done
- Q&A: Why do I need to resample for example in the case of an amplifier, why can't I just make the playrate slower or faster?
- There is no market solution (cheap and easy) to change the frequency of the amplifier because of the complixty of the problem
- Usually all amplifiers are set at a certain frequency and there is no dynamic or adaptable playing rate
- Jitter as explained in Wikipedia
- So PTP does not solve the jitter problem, how can we solve it then?
- When we say jitter in this context we mean clock differences below 5ns.
- So PTP synchonizes the clocks up to 5ns, any difference below that is jitter.
- Actually the receiver would recover the clock by using the "presentation timestamp or PTS"
- In automotive domain, we mostly use a second layer on top of PTP → IEEE 1722-2016 - IEEE Standard for a Transport Protocol for Time-Sensitive Applications in Bridged Local Area Networks
- And we also use Qav to restrict the emission
Side discussion: AVB (Audio Video Bridging)
- AVB is about routing only (no control/ start/ skip/ pause)
- SRP (Stream Reservation Protocol) protocol was supposed to support the control but there is no dynamic protocol to control the media specially in automotive
- Usually some OEM do this with SOME/IP (Scalable service-Oriented MiddlewarE over IP) protocol but there is no standardization
- But it's also because there are some more controls like volume, surround, etc. that are not in SRP
- It's hard to catch up with technology
- Either you have an AVB hardware or not (you buy it) and then the vendor would give you the support needed and the explanation
- Presentation of AVB overview was shown.
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Gunnar Andersson Philippe Robin Piotr Krawczyk Unknown User (wassim.filali) Unknown User (niskandar)
Minutes
Going through the list of prioritized topics:
- Gunnar maybe We can also check if there are some conflicting priorities, like High from one party and low from another, and we try to clear any misunderstanding about the subject or see it in a different way.
- Wassim: Maybe each one of us can fill the table offline (those who haven't filled it yet) and then we can check if there is a priority conflict.
- Wassim: for now I suggest that participants take one or two subjects that they find interesting a sort of a brain storming session
Brain storming session (recorded in PoC Milestones and Work Breakdown (AASIG, AHAL, audio-control) - in "Brainstorming about breakdown rework")
- Wassim: everyone can pick 1-2 topics that we can work on that we think it's important
- Piotr: Networked audio devices and Audio effects
- Gunnar: To clear this up, I don't really have an opinion in this area, my role here is more of a facilitator. It would be interesting however to check how we can handle the audio streams differently than from within Android Automotive (Raw Stream topic)
- Nadim: Networked audio devices and Multizones
- Wassim: Mainly keeping up with the mentioned topic but with particular stress on interfaces:
- Interfaces definition and standardization of the networked audio devices
- Audio effects related to DSP offloading however from a point of view of interfaces not only the different audio effects
Conversation about the interfaces
- Wassim: the point is that after discussing with the VHAL team, I thought to bring the same concept of interface building to AUDIO_HAL. What interfaces can we build in AUDIO_HAL that can be used by everyone. What interfaces can we standardize in AUDIO_HAL. This is why I emphasized the interfaces topics in my points.
- Wassim: @Gunnar Can you please give some example from VHAL
- Gunnar: I think this is always a target of GENIVI and it might that in VHAL we have reached this point after many discussions and contributions from different OEM and parties.
- In AUDIO_HAL, firstly there isn't a lot of interest, secondly we first have to define what we need, test it with a prototype, adjust and build interfaces and then think of standardization
- Philippe: there is for example a renewed interest in multimedia streaming standardization, this could be a nice start
- Gunnar: yes this is a nice point of course, but this is a very comlex question or topic to handle.
- Gunnar: we already had once such a topic but it the conclusion is that even if a standardaztion was to be developed, it would not have been used because most of the multimedia sources have already their applications on Android. They asked to just use the applications and the API there.
- Gunnar: in general I recommend that we : start locally (in each of our companies) and check "what do we need that AA doesn't provide", then we start developing it. If it works, we check if this should be made as a standard, then we make interfaces for it.
- Wassim: For example, since we talked about the networked audio devices, what are the main use cases:
- External amplifier controlled by Android only (for example to set priority of what to play etc)
- Audio system controlled from other ECU in the car (for example remote control)
- Audio system controlled from a server
- Wassim: Android has a solution for let's say point 1. But what about point 2 and 3? Android did not plan to contro lthe volume for example from outside Android. So why don't we have a GENIVI Standard way of doing this?
- Gunnar: yes all of this makes sense, but we need to check the different solutions before we standardize the interfaces. We could also ask for point 2 and 3 above, are these for Android only? or are they for Linux?
- and if it's for Linux, then the target audience is very large and not restricted to attendees of GENIVI AUDIO_HAL.
- Wassim: yes this is correct, we need to start first with the work and then we can go on with the next steps. i think GENIVI AUDIO_HAL is a nice platform to do this in order not to have fragmanted solutions in our industry.
- Wassim: maybe we can reduce the ambitions a bit and not target for now ISO standards.
- Wassim: rephrasing the topics of interest to not include "standardization"
Last few mentions
- The webex meeting is now a reccurent meeting and this means that the link can be used several time.
Todo
- @ALL: update the List of prioritized topics for the Audio HAL with priority of the project
-
Unknown User (niskandar) : update the webex meeting link in the Audio Hal minutes
Thursday - 5 November - 11:30am CET
Participants
Gunnar Andersson Philippe Robin Stephen Lawrence Suhasini Raghuram, Unknown User (wassim.filali), Chris Simmonds
Minutes
Review of the list of prioritarized topics
- List of prioritized topics for the Audio HAL
- updated online
Wednesday 28 October - 9am-1pm CET - GENIVI AMM AASIG AUDIO HAL working session
GENIVI AMM AASIG AUDIO HAL working session:: slide deck and session recording are at GENIVI Virtual Member Meeting October 2020
Thursday - 1 October - 11:30am CEST
Participants
Gunnar Andersson Philippe Robin Stephen Lawrence Suhasini Raghuram Henric Carlsson
apologies: Unknown User (wassim.filali) Unknown User (niskandar)
Minutes
Review the work breakdown structure and update it to get a shared knowledge of the project status before preparing the agenda of the AHAL working session at the tech summit
diagrams on the top of PoC Milestones and Work Breakdown are still valid
Jira server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-70 Gunnar needs to try it , but not done yet, testing the raw stream extraction is part of the activity dedicated to integrate everything in the platform build
commented online, closed
| Jira | ||||||
|---|---|---|---|---|---|---|
|
Piotr: Bartosz is not available due to a production project, we should reassign it
Piotr: will take over the design overview activity so that we can start the discussion within the team
Jira server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-79 Henric will check whether someone at Bosch can do some work on this
assigned to Henric
Jira server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-79 Suhasini is ok to start this activity it if someone can help her installing the platform
Piotr is available for helping suhasini
Jira server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-92 discussion on how to tackle the topic
use socket api seems to be the simplest solution
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Gunnar Andersson Philippe Robin Stephen Lawrence Unknown User (wassim.filali) Suhasini Raghuram Henric Carlsson Unknown User (niskandar)
Minutes
Updates from last meeting:
- Restructuring the repositories talk can wait for when Tieto are in the meeting. For now the current repo is enough.
- Nadim's update: announcement about the pre-recorded presentation.
- Nadim: I won't be able to record the presentation and in general there will be less time for me to work on GENIVI related topics due to shifts in my priorities.
- Philippe: could you maybe suggest that we prepare the slides and you just record them?
- Nadim: I'll send this request yes.
- Stephen will try to update his topic by next meeting.
- Suhasisni's Update: waiting for the approval to purchase the board (decided on Qualcom).
Discussion about how to handle the push for GENIVI tasks to mangement:
- Gunnar: the key point is to integrate the GENIVI tasks in your daily tasks, and not treat it like a side project. Try to search for common points or wishes in order to contribute to the project while doing your daily tasks.
- Wassim: in BMW my argument is that the GENIVI Audio HAL is a better solution than the Linux HAL and investing time in GENIVI tasks leads to better product development.
Going through the list of Jira tickets
Bluetooth Device Integration in Android System
Jira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-79 - Gunnar: Is there an interest in this?
- We need to check the existing capabilities of Android first, what are our needs, and design a solution for this
- A lot of flexibility is needed usually in this subject, for example using the microphone for more than 1 thing, rerouting the audio, etc.)
- Wassim: yes i can confirm that this is an interesting subject for us but i would not take it all on my own, i could be assisting
- Suhasini: I'm also interested and I would take this ticket but I would require more documentation about BT Android first.
- Wassim: the BT android documentation is good but I think we should also invest in going to seminars of Android etc. and try to get the latest Android documentations (for example currently the Android 11 documentation would be good)
This would buy us at least 2-3 months before the official documentation come up - Gunnar: this also would make sure that we are not designing against new Android decisions or a redundant solution
- Wassim: unfortunately I cannot share such documents from seminars
- Gunnar: Is there an interest in this?
- Ensure measurability of full audio path latency
Jira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-86 - Nadim: I cannot continue investing this ticket because of the previously mentioned priority changes
- Gunnar: Some will have to take it up again when they are free.
- Offlading topics
Jira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-92 Jira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-93 - The goal is to identify the technology needed to get audio transferred out of Android.
- Maybe it makes sense to agree on common terms and definition in our GENIVI meeting that would let us describe the system/design commonly. For example what is vendor partition, Android domain, etc.
Question about the Webex link
- Suhasini: is the attached webex link a repeatable one? Usually when we click on it it says that the meeting happened in the past so the meeting is not a recurring one.
- Philippe Robin: Could you please create a repeating Webex meeting and link it on this page?
Gathered todos
- @All: Try to connect your daily work to GENIVI tasks
- @All: Contact Google or check in your company if there are already documentation about the latest Android
- @All: work on the repeatable platform setup in each of your companies (section: Additional build and run info in PoC Milestones and Work Breakdown (AASIG, AHAL, audio-control))
Thursday 17 September - 11:30am CEST
Participants
Gunnar Andersson Stephen Lawrence Henric Carlsson Piotr Krawczyk Unknown User (wassim.filali)
Minutes
Discussion on demo implementation status
Thursday 10 September - 11:30am CEST
Participants
Gunnar Andersson Stephen Lawrence Henric Carlsson Piotr Krawczyk Unknown User (wassim.filali) Philippe Robin
Minutes
Discussion on configuration & demo building & integration of the emulator project, followed by Jira review
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Gunnar Andersson Stephen Lawrence Henric Carlsson Piotr Krawczyk
Minutes
Discussion on feature content for MS3 demo (MS3 = virtual tech summit scheduled on October 26-30) Gunnar Andersson can you add the notes you took during the call ? thanks
Audio HAL session is scheduled on Wednesday 28 October morning 900 to 1300 CET
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Gunnar Andersson Stephen Lawrence Henric Carlsson Piotr Krawczyk Unknown User (niskandar)
Minutes
Going through the previous minutes of the meeting.
: @All, look into this ticket and check which board you would like to buy to use as a hardware. We can have an expensive board running Android 10 to emulate a real infotainment system, and a cheap board to emulate a receiving hardware. The best candidate so far is the Kingfisher.Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-104
- need an updated on these issues, but will be given next week (AUDIO_HAL_CW34)
Jira showSummary true server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-74 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-73 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-75
- No progress on these tickets
Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-91 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-87 Jira server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-86
Talking about the goals and the future
- There is a delay in reaching our milestones and the deadlines we have set.
- our goals are good and we should try to reach them and reach the results that we set for the milestones.
- In general we need a plan or a guide to check how is everyone taking part in regards to the goals we set.
Absenses
- Gunnar will not attend AUDIO_HAL_CW34
- Nadim will not attend AUDIO_HAL_CW34 because of a QT/QML training and AUDIO_HAL_CW35 because of an EA/UML training
Gathered Todo's:
- @ALL: Please have a look at ticket
and comment which board your would like to work withJira showSummary false server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-104 - @ALL: Please think of a plan or a guide on what members are planning to do for the future regarding goals, milestones, etc.
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Suhasini Raghuram Gunnar Andersson Stephen Lawrence Unknown User (niskandar)
Minutes
Going through the previous minutes of the meeting.
- briefing of why would need to study Alsa.
- briefing of the concept of the external PC as a "real HW".
- Gunnar: yes but i think the goal is to have a hardware as close to production as possible.
- Another goal is to learn something from the emulator.
- But I still agree that the external PC can be an intermediate solution.
- Not sure however how ALSA will fit this.
Through the Jira tickets (comment is in the tickets already)
Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-104
There was an update from Stephen about the Kingfischer, followed by suggestions from Gunnar and Suhasini.
Stephen was also explaining why some of the board manufacturers are restricted in the features they provide.
The decision is to investiage several other boards for feature list specially to check if they provide (A2B, I2S, S/PDIF)
The board doesn't have to have Android, and doesn't have to support multiple ways of routing the audio.
We just need to prove that we can reach the amplifier and process different audio channels.Jira showSummary true server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-74 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-75 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-73
Briefing and last status update of these issuesJira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-91 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-87 Jira server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-86
Briefing and last status update of these issues
Does it make sense to measure the latency on the emulator setup with an external PC as well? As a reference point, to check stability, variance, etc.
Gunnar: the main point is to have a method of checking the delay and check if the delay is stable, measurable, reliable. If this happens, we can design anything around it.
In any case, in most application with videos, the video processing is slower and usually we delay the audio. So a big delay is not a big issue for most application.
But for safety critical applications, like beeping sound from the rear camera, we need to have a reliable and stable value of the delay in order to work around it and deliver a safety critical application.
Gathered Todo's:
- --
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Philippe Robin Suhasini Raghuram Unknown User (wassim.filali) Unknown User (niskandar) Stephen Lawrence Piotr Krawczyk
Minutes
Going through the previous todo list:
- Gunnar will not be attending this meeting and the next one (AUDIO_HAL_CW31). So his todo is postponed.
- Stephen will find more information about the Kingfisher. In general, to check what hardware we can use, we can look at the R-Car website from eLinux.org. For example here the Kingfisher is shown. This week the focus was more on the hardware farm of Genivi.
- Piotr will also go on vacation and will not attend AUDIO_HAL_CW31 and AUDIO_HAL_CW32.
Going through the Jira tickets (comment is in the tickets already)
Jira showSummary true server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-74 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-75 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-73 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-91 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-87 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-104 Jira server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-86
Topic about connecting the emulator to a PC
- x86 is also a "real hardware" so we can use it for testing while an audio board is being prepared. This would also provide an alternatives for people who have no access to a board, and an alternative until the GENIVI hardware farm is finished
- One example can be: connect the PC with android emulator to two other PCs with TCP sockets. Then we can send from the android emulator different zones to the different PC and check what we can do with the data. Maybe someone with more experience of GENIVI audio manager can better connect the sockets
- Maybe we can do it with Alsa not only GENIVI audio manager
Discussion about the emulator and abstraction
- Wassim: Alsa provides some network capabilities we need to have a better idea how alsa works and if we can use it → todo
- Piotr: yes but we are trying to create an emulator that is as independent as possible. Not sure if Alsa is for now a good starting point, maybe it can come later.
- Wassim: regarding the emulator, now we are using the PCM sockets but what about using AVB? and PTP on top of it?
- Piotr: We can think of these for the future but now it's better to start without any dependencies. → todo bring this subject up in next meeting and check if we can create a Jira ticket from it
- Piotr: The most important thing is to build an infrastructure from the begining, for measuring many variables such as latency, lipsync, etc. → todo add this to the Jira tickets
- Piotr: Even if we are using AVB we need some sort of a bridge to feedback to Android because Android defines that the Audio manager should know its own latency, that will give an advantage and allow the third party paplications to work better. We need a bridge in any case (with and without AVB)
- Wassim: In that case it's even better because AVB has many restrictions, and if we can have a bridge that feedsback such measures (latency, lipsync, etc.) without AVP and PTP, it would be benifitial.
Gathered Todo's:
- Unknown User (niskandar): bring up the subjects discussed during this meeting for next week and check if we can add Jira ticket.
- Studying the capabilities of Alsa
- replacing the PCM sockets with sockets that have AVB and PTP in the emulator
- Connecting Linux Alsa or Genivi audio manager to the emulator
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Philippe Robin Gunnar Andersson Unknown User (bartoszbialek) Suhasini Raghuram Unknown User (wassim.filali) Unknown User (niskandar) Stephen Lawrence
Minutes
Alternative board for Suhasini, any upadte on that matter?
- Gunnar: Are you looking for an NXP or Renesas board?
- Suhasini: is there the option to remote access to the GENIVI hardware
- Gunnar: it's not yet setup and it's more like describe a job, let it run, and read the logs
- it's using LAVA and the code is under development
- it's a complex thing to set up and it might need some time, i don't see it happening in the near future.
- I suggest to look for alternative hardware with less capabilities
- we can also check the emulator from Tieto
- I'll also check with NXP with their support of Android 10 → TODO
Main topic: Implementation, proof of concept and Demo going through the issues
- Discussion about the following tickets was followed and recorded in the respective tickets
Jira showSummary true server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-74 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-75 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-73 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-91 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-87 Jira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-104
Hardware discussion followed:
- Gunnar: in order to run the demo on real hardware, even a simple one would be suffiecient for now. We would need to know how to build the files and create the manifests to compile it on the hardware
- Gunnar to Bartosz: could you please provide these files and manifests?
- Gunnar: is there a desire for hardware board in Tieto at the moment or do you prefer that you stay with the emulator and others port the code and apply it on a hardware
- Bartosz: at this point, there are no requirements from our side, both aprroaches would work.
Discussion about the current emulator:
- Wassim: once we extract the streams from Android and them to the sockets we can theoratically receive them anywhere, even for example on a host PC where we could have tools to mix the streams, etc.
- Gunnar: yes but the story AASIG-104 is more about testing an external system. This would also take into consideration the low latency topic.
- Gunnar: Is there currently any limitation on how many sockets we can create?
- Bartosz: From the implementation side, we are not limited. We can provide any stream to any number of TCP sockets. It's now as many sockets as we have audio contexts
Hardware requirement discussion:
- Stephen: what hardware are you looking?
- Gunnar: doesn't have to support Android but something that can be connected to an external amplifier via a digital connection like A2B, I2S, S/PDIF (but not MOST). Something that would be similar to what is inside a car in order to get as close as possible to a real life scenario
- Stephen: I'll look into it, I can see this one the Kingfisher, it has AVB, MOST, and BT (but with BT limitation of channels) → TODO
Demo Next week:
- Philippe: is it possible to have a demo next week with the emulator?
- Bartosz: I think it should be possible. → TODO
Gathered Todo's:
- Gunnar Andersson : Check with NXP if they support Android 10
- Stephen Lawrence : find a simple hardware with multiple digital outputs refer to above discussion for more details)
- Unknown User (bartoszbialek) : prepare a demo using the emulator for next week
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Philippe Robin Gunnar Andersson Piotr Krawczyk Unknown User (bartoszbialek) Henric Carlsson Suhasini Raghuram Unknown User (wassim.filali) Unknown User (niskandar)
Minutes
Should the minutes be more like a transcript?
- Maybe capturing the decisions is better than transcribing
- it's faster going through the minutes than through Jira tickets
Going through the previous tasks
- A meeting should be setup apart from the original
- Maybe for the next meeting, we can go through the tasks that exist in the sprint and check the progress
Going through a tutorial of Jira
- We can see the stories in the backlog which contains the sprints for all the projects not only AASIG
- We can also check the AUDIO_HAL tickets from the project components
- Sprints are depending on the milestones. The current sprint here is made to coincide with the milestone in mid-june
- Backlog is for later meaning after the sprint would be taken (different milestone)
Going through a few stories to add them to the current sprint
- we should document the workplan, the decisions, the status in the ticket itself. In order for everyone to have a better overview of the ticket, implementation, etc.
- Unknown User (bartoszbialek) : Could you please add the workplan and the problems faced in this story
Jira showSummary false server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-73 - Piotr Krawczyk : Could you also please add comments about the workplan of
Jira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-74
: the story should have an assignee this is why we are assigning Unknown User (bartoszbialek) to it.Jira showSummary false server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-75 - latency investigation: it's nice to have to investigate this a bit early on in the project
will be taken by Unknown User (niskandar)Jira showSummary false server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-86
andJira showSummary false server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-92 Jira showSummary false server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-93 - we need to review and rewrite the tickets for this offloading topic
- we don't do a lot of work from design but more we work on the concept
- it's a topic that we would want to work a bit on
- Gunnar Andersson : setup a smaller team to redefine the Offloading topics Tue 10am : Done
- Suhasini Raghuram : if you can express what answers you would like to have from the group on
Jira showSummary false server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-92 - Are we looking for concepts that shows different system designs for offloading, or more like technologies
- It's more to enable the choices that we have
: running the demo on a real HWJira server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-104 - Who has a Renesas board currently? You can purchase it from the company in your country.
- Hikey is just stereo, Renesas has multi-channels. Please check the page Android development platform
- Piotr: we can start this activity after the multi-zone and reference to Audio-hal
- We can try to assign it during the next meeting
Setting up the environment
- does it make sense to use something like docker to make it easier for others?
- actually at first it was working with dockers, then it was dropped
- The instruction from android pages seem to be working and can be followed easily
- Maybe Docker can be useful for different platforms
- If we would have automated build, we would need to have a stable unified environment
- This is part of the ASIG goals actually, to create a stable common platform that we can use for different hardware. It should be also considered in the future
Gathered Todo's:
- Unknown User (bartoszbialek) : Could you please add the workplan and the problems faced in this story
Jira showSummary false server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-73 - Gunnar Andersson: setup a smaller team to redefine the Offloading topics. Will be Done Tuesday 30 June 2020 at 10am CEST.
- Suhasini Raghuram : if you can express what answers you would like to have from the group on
Jira showSummary false server JIRA columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-92 - Piotr Krawczyk : Could you also please add comments about the workplan of
Jira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-74
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Unknown User (niskandar) Gunnar Andersson Unknown User (zafirulhassan) Piotr Krawczyk Unknown User (bartoszbialek) Unknown User (zafirulhassan)
Minutes
Looking at the history, the best course of action here is to go through the Jira tickets and assign them to start working on them.
First Story
| Jira | ||||||||
|---|---|---|---|---|---|---|---|---|
|
- Nadim: where would be put the outcomes of some of the Jira tickets?
- Gunnar: we would put the outcomes in the confluence ordered by logical order. We build the confluence pages as we are finishing the Jira tickets.
- Piotr took the ticket
. The deliverable is to have a picture that shows an overview of the needed SW involved in the Raw Streams extraction. It would be added here PoC Milestones and Work Breakdown (AASIG, AHAL, audio-control)Jira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-71 - Audio Hal discussion
: checking why do we need this point? Piotr will check if the AA Audio HAL is ready for raw stream extractionsJira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-72 - Bartosz took this ticket
: checking the multizoneJira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-73
would also be assigned to Piotr but it would take some time. It's about writing some code inside the HAL. This will be try to done with Bartosz and Piotr.Jira showSummary false server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-74
Going through other stories
- it doesn't make sense to continue tackling the other stories because we are not today complete.
- Bartosz will take this task
, so that we can have a ready overview picture for a better understanding. Please also add it to the top of the page PoC Milestones and Work Breakdown (AASIG, AHAL, audio-control)Jira server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-76
: why is this a story on its own? there are mixed points between this oneJira server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-92 Jira server JIRA serverId 121ddff2-c571-320f-9e4d-d5b9371533bd key AASIG-93
Platforms to use: Android development platform. We can also use an emulator.
Gathered Todo's:
- Unknown User (niskandar): setup the environment in order to be available for implementation
- Gunnar Andersson: please communicate with Philippe Robin and setup a meeting in order to improve a bit the translation of the Jira tickets.
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Unknown User (wassim.filali) Philippe Robin Gunnar Andersson Piotr Krawczyk Unknown User (bartoszbialek) Henric Carlsson Stephen Lawrence Suhasini Raghuram Unknown User (niskandar)
Minutes
Let's start by going through the points that we have gathered last time in PoC Milestones and Work Breakdown.
Our task is about breaking down the topics that we discussed last week:
- Wassim: How are we handling the breakdown in Confluence, Jira, etc.
- Philippe: I recommend to add the activities to Jira but keep here the overview
- Gunnar: important thing is to add the tasks in order to start working
- Philippe: I recommend to add the activities to Jira but keep here the overview
- Nadim: what about that we start from the User story, we write use cases, then requirements, and we register everything in Jira
- Philippe: we are not writing specifications, think of it that we are now tracking the activities that we need to do. We are not doing a full top-down approach.
- Gunnar: I think what we are also missing is the architecture side of these topics
- Gunnar: From my experience, it's better to create some sort of a guide or design for the programmer to have and ofc more details would follow, like interfaces, design, etc
- Piotr: are we detailing in sort of components or actions to be done?
- Gunnar: let's break down the overview in terms of components in order to see what components would need to be created new and what are already there
- Piotr: another point is about the Audio HAL if it would be redone or not?
- Philippe: so let's put them all these tasks in the backlog. We can pick them up later but we need to gather them now
- Gunnar: let's break down the overview in terms of components in order to see what components would need to be created new and what are already there
- Gunnar: regarding the question about the hardware, I think:
- We need to check what do we need for each hardware first. Then we check how much time we need with each hardware
- What parts of this work can be done so that it supports everything, and what parts should be done on each platform
- Piotr: we can create some sort of interfaces as common and then each platform would have an implementation
- Gunnar: once we know what is needed, we can estimate how much time each platform would take to create the implementations
- Piotr: extracting Row stream can be done using the emulator for now we don't need a hardware unless we need to target timing measures
- Stephen: Is there a list of features that the emulator has, in order to have a hardware that is very similar to the emulator and save time in porting?
- Gunnar: maybe this can be done from an architecture picture when we create it for each topic? Let's keep this point in mind.
- Suhasini: is the offloading only to the external speakers (point 9)? can we not redirect the stream after decoding back to Android?
- Piotr: I think this offloading can be a topic on its own
Going through the points to break them down here. Notes added directly in Description of work table
- reorder of the topics was done
- some topics or sub-items can have open points that need to be addressed before the topic
- BT will be kept into once container
- Early audio would complicate a bit the point, this is why it's not considered in the point "Input and Control of external streams"
- The offloading needs to be handled a separate item to have some proper documentation about it because it's a big and important topic
- With multi-channel we specifically mentioned 5.1 so let's add it here to be more specific
- Looking at the point 9, it's because of the Android policy that is reducing the number of channels because of limitations
- One other point is to keep the Dolby stream encoded before sending it to the external speakers, because of the BW that an be saved
Gathered Todo's:
- Unknown User (wassim.filali) : create the scratch for the remaining topics in the Description of work table in PoC Milestones and Work Breakdown.
- @All: Check the page Description of work table and participate in updating the scratch. Due:
- Philippe Robin: create initial Jira tasks from the work packages
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Unknown User (wassim.filali) Unknown User (niskandar) Philippe Robin Gunnar Andersson Piotr Krawczyk Unknown User (bartoszbialek) @Henric Unknown User (zafirulhassan)
Minutes
Introduction to members and new member Zafirul
Went through the minutes of last week and the gathered TODOs
Hardware clarification
- Wassim: Do we have a common devKit hardware that we can use for Audio HAL and Vehicle HAL?
- Gunnar: Yes we do, although HW is more significant for Audio as the V-HAL SIG is currently working on a PC emulated environment
Discussion of PoC Milestones and Work Breakdown
- Wassim: The first table purpose: go through the list of prioritized topics and add them to the table, to create use cases/work packages, later we can divide them into more details
- We convert from topic to use case (keeping the topic ID)
- Started with topic 13 (++++)
- Then it's topic 8 and 9 (+++)
- Wassim: We will not continue with more topics from the list we only take care of ++++ and +++ and topics from the PoC milestones
- Philippe: We will not mimic the work here and in Jira, in Confluence would be a small explanation and a link to Jira where the subtasks etc will be created
- Description of work, important things to do, we transform them later to jira tickets
- Wassim: Second table: Milestones
- Philippe: milestones are not one to one with the one we presented though, maybe we can try to work with the scheduled date to speed up a bit the whole process in order to align with the milestones of GenIVI? such as: Reporting to google, etc. Philippe Robin: please correct me if i'm wrong, are these the milestones that you meant?
- Wassim: yes well Milestone 1 can be shorter than Milestones 2 because it needs less time to be finished.
- Wassim, Philippe: maybe now the focus is on the description of work and we can set the schedule later accordingly
Gathered Todo's:
- @All: Check the page PoC Milestones and Work Breakdown and prepare use cases for the topics and work packages that we've gathered in the table. This will allow us in the next meeting to detail the work packages and better estimate a schedule.
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Suhasini Raghuram Unknown User (wassim.filali) Unknown User (niskandar) Philippe Robin Gunnar Andersson Piotr Krawczyk Unknown User (bartoszbialek) @Henric
Minutes
Debriefing of the virtual summit
- Philippe: what was discussed with Sriram. I think it's an important conversation. Gunnar's concerns as well.
- Wassim: what exactly should we discuss?
- Philippe: sharing the minutes of the virtual summit.
- Concern: why are we not just using Android. Are there features that Android does not support?
- Wassim: What is the background of Sriram?
- One of the architect of GENIVI, now manager at a company that uses GENIVI products and solutions used in production already
- Wassim: The base questions was: why do we need the proxy? Why can't we do everything with the Android Audio HAL
- I think this is the question that we need to answer as it's the base of Sriram's question
- We cannot answer it now, we need to check if Android Audio HAL fulfills all the requirements
- Piotr: The proxy is (as agreed in the virtual summit) a placeholder for now
- Any functionality that Android Audio HAL cannot fulfill are done by this placeholder
- Philippe: maybe we need to document the features that we would like to try
- We need to try to do something outside of Android and check if it makes sense
- Wassim: we can answer the question of Sriram when we have a working PoC and try out specific use cases to check
- Philippe: we need to come up with more detailed work plan and we need to identify several stages and add topics to it. An example work plan would look like this:
- What do we do to handle the low latency? List our solutions
- Experiments to check our solutions with the PoC
- Answer the question: Can this be handled from within Android or do we need outside dependencies
- Not sure however if the low latency problem has anything to do with the audio manger. Maybe it should be dealt alone.
- What do we do to handle the low latency? List our solutions
- Gunnar: there are some topics such as low latency, the ability of the application to control where the audio is going, the safety control audio, these are some of the topics that can be added outside the android platform
- Nadim: is this work plan within the milestones?
- Philippe: yes exactly we need to expand on these milestones and detail the work to be done
- Philippe: maybe we can create some sort of a WBS to start with it
- Unknown User (wassim.filali) : create a confluence page to place the ideas and the WBS
- Unknown User (wassim.filali) : communicate with Piotr Krawczyk and Unknown User (bartoszbialek) and others to get the ideas for the next steps → Initiate the WBS creation
- Unknown User (wassim.filali) : create a confluence page to place the ideas and the WBS
- Nadim: what about the milestones are they still the same
- Philippe: well we cannot change the dates for sure
- Wassim: the milestones will not contradict each others so let's keep them as they are
- Philippe: well we cannot change the dates for sure
Discussion about hardware
- Wassim: Do we create the PoC using software simulation only or do we need a real real HW? (Unknown User (wassim.filali) : please correct me on this question)
- if we try on emulator without a real hardware then it's not really a proof
- If we start at hardware we might get stuck with some hardware issues
- Piotr: Maybe we can get the input of Sriram here as well
- Philippe Robin : can we get the input of Sriram on this? Please wait for Wassim to better clarify the question.
- Gunnar: can we do both?
- Unknown User (niskandar) : Add path to the proof of concept code https://github.com/GENIVI/android-external-audio-mixing
- Piotr Krawczyk : could you please add to the read me note, how to compile on virtual server
- Gunnar: We have several suggestions but if you need as a company a certain hardware then we can select it
- It's more about who is providing the software to support what we are doing
- Piotr: Qualcom is out of scope, it would be very nice cause it needs minimal efforts to have nice solutions but their support is rather minimal for us. They are more mobile oriented.
- Henric: preference is Renesas
- Suhasini: no preferences for the hardware
- Nadim: no preferences for the hardware
- Hikey is low cost with support
- As a vote: Renesas seems to be the first choice and Hikey as the second choice
- It's more about who is providing the software to support what we are doing
- Wassim: how will the hardware be accessible?
- Philippe: it makes sense if each member would have a board on the desk if you want to listen to the audio
- Gunnar: there will be a global testing center with Renesas' support where we can do some testing on it, but if we want to listen to something a hardware board is better
- Gunnar Andersson : which HW from Renesas to order?
Gathered Todo's:
- Unknown User (wassim.filali) : create a confluence page to place the ideas and the WBS
- Unknown User (wassim.filali) : communicate with Piotr Krawczyk and Unknown User (bartoszbialek) and others to get the ideas for the next steps → Initiate the WBS creation
- Philippe Robin : can we get the input of Sriram on this?
- Unknown User (niskandar) : Add path to the proof of concept code https://github.com/GENIVI/android-external-audio-mixing
- Piotr Krawczyk : could you please add to the read me note, how to compile on a virtual server
- Gunnar Andersson : which HW from Renesas to order? – answered on this page: Android development platform
Thursday 14 May - 9am-12am CET - Virtual Technical Summit Workshop
Minutes
- minutes are here
| Anchor | ||||
|---|---|---|---|---|
|
Participants
Suhasini Raghuram Unknown User (wassim.filali) Unknown User (niskandar) Philippe Robin Andrii Chepurnyi Harald Bartholomae Ruslan Shymkevych Gunnar Andersson Piotr Krawczyk @Henric
Minutes
Going through the presentation of Wassim
- The presentation is work in progress
- The main message is that we are extending the design of google and if there is something not supported by google, we try to make it common or standardized
- We need to decide on topics to discuss during the summit.
Summit next week discussion
- Unknown User (niskandar) : reorder the list of prioritized topics
- Piotr Krawczyk Unknown User (bartoszbialek) : could you please prepare the demo for the summit?
- Piotr Krawczyk Unknown User (bartoszbialek) : could you please prepare a few slides about the demo as well
- Wassim: Could the slides be finished before Wednesday evening in order to add them to the main presentation
- We have 4 slots:
- Introduction 30min: presentation of the AUDIO HAL + slides of demo + demo
- Slot 1, 45min: PoC details discussion, define Next Milstones (AM integration) (avoiding And modifications and long term API support strategies)
- Slot 2, 45min: PoC prerequisites (external raw streams, Injecting input streams)
- Slot 3, 45min: Going through the list and refining the priorities. Asking participants about their opinion and feedback
- Other topics:
Early Audio
Audio Calibration / Equalization
Controlling Audio Effects (Google might improve it in next version)
Unknown User (niskandar) : update that the investigation in the global effect was stopped because google will release new api
- Ruslan: We should not replace components of Android IMO. It's too expensive to maintain the components replaced.
- Wassim: Yes for sure we are trying to avoid component modification and trying to follow long term strategies of Android (by keeping up-to-date with where they are heading)
Gathered Todo's:
- Ruslan Shymkevych : Get in touch with Unknown User (niskandar) for adding the different sections of his email into the wiki
- Unknown User (niskandar) : reorder the list of prioritized topics
- Piotr Krawczyk Unknown User (bartoszbialek) : could you please prepare the demo for the summit?
- Piotr Krawczyk Unknown User (bartoszbialek) : could you please prepare a few slides about the demo as well
- Unknown User (niskandar) : update that the investigation in the global effect was stopped because google will release new api
Anchor AUDIO_HAL_CW18 AUDIO_HAL_CW18
Thursday 30 April - 11:30am CET (AUDIO_HAL_CW18)
| AUDIO_HAL_CW18 | |
| AUDIO_HAL_CW18 |
Participants
Philippe RobinSuhasini Raghuram Unknown User (wassim.filali) @Henric Ruslan Shymkevych
apologies: Piotr Krawczyk Unknown User (bartoszbialek)
Minutes
Review of Ruslan's email of 23 April - archive
- Wassim: all the different topics listed by Ruslan are interested, my recommendation would be to add and structure them in the wiki
- raw stream section should go to Android and System Level Audio wiki page
- the other sections are more relevant for the List of prioritized topics for the Audio HAL
- Philippe: this can be done in coordination with Nadim who was assigned a TODO for review the list of prioritarized topics
Dynamic routing
Henric: in our mock-up we were able to enable the dynamic routing and when we did so, the USB audio stopped working
Ruslan: in EPAM implementation we do not have USB audio so I cannot help
Ruslan: it looks to me that an internal option to be turned on, this relates likely to the implementation inside the framework, AFAIK the USB audio feature is not dynamic
Philippe: it would interested to get TietoEVRY guys's opinion
Proof-of-concept code
- Gunnar: I invited Piotr to publish his code on github, but he has acknowledged the invite yet
- the code repo is android-external-audio-mixing repository
- code quality guidelines: GENIVI has such guidelines, please look at code quality guidelines and the presentation linked to the page
Virtual technical summit preparation
- Philippe: reminds Wasssim to reserve some time in his agenda for next week for preparing slides for the short status report on Tuesday 12 May and the workshop on Thursday 14 May
- Philippe: will send his recommendations offline
- Philippe: we will use Webex for the workshop on Thursday and an other webinar professional service on Tuesday 12 May (not zoom)
- NOTE: for the Tuesday sessions, there will be a rehearsal with the professional service the day before (Monday), stay tuned
- Philippe: asks Henric and Suhasini whether they have specific topics to discuss the workshop; they will check whether they can suggest something
- Henric: Bluetooth
- Suhasini: networked audio devices
Gathered Todo's:
- Ruslan Shymkevych : Get in touch with Unknown User (niskandar) for adding the different sections of his email into the wiki
Anchor AUDIO_HAL_CW17 AUDIO_HAL_CW17
Thursday 23 April - 11:30am CET (AUDIO_HAL_CW17)
| AUDIO_HAL_CW17 | |
| AUDIO_HAL_CW17 |
Participants
Philippe Robin Piotr Krawczyk Unknown User (bartoszbialek) Suhasini Raghuram Unknown User (niskandar) Unknown User (wassim.filali) @Henric
Minutes
Updates of the tasks done from last week: Wassim, Piotr, Nadim
- Unknown User (niskandar): create a picture for next week to show the different options of handling external streams in Android
- Henric: Andoird Patch API, has anyone worked with it?
- Bartosz: well actually this was used in Android 9 but in Android 10 it's deprecated and replaced by Android Audio Source.
It's a better API for external sources and has more control for example with events that can be sent to the external sources etc. It makes the external sources mix better in the framework - Bartosz: we can also create a sort of a loopback for handling external sources such as creating a recording of the external source and playing back.
- Bartosz: well actually this was used in Android 9 but in Android 10 it's deprecated and replaced by Android Audio Source.
Project proposal Henric: User Input Distribution and Coordination, if it's needed maybe we can add it to the list of prioritized topics.
- @All: investigate the project User Input Distribution and Coordination to check if it's needed
- Philippe: Actually we should update the list of prioritized topics in general
- Unknown User (niskandar) : Check the list of list of prioritized topics
Philippe two more points:
- Jaguar Land Rover (JLR) representatives working on Android Automotive joined the AASIG VHAL call this week and should join the All Hands monthly call next week. If they ever decide to join the Audio HAL project, we might have to think about the time of the meeting because they are based in the US Pacific
Project Status Report:- the Technical summit is nearly here, we must prepare a presentation about the projects VHAL and Audio HAL and in general update and prepare an agenda for the virtual meeting on the 14th of May.
- Unknown User (wassim.filali) : could you please work closely with Philippe and Gunnar in order to prepare a comprehensive technical report for the status of the project and an agenda?
- The Agenda can be found here https://www.eventleaf.com/geniviVTS. Some of the meetings are very interesting and will be broadcasted live by a professional company without the use of Zoom.
Gathered Todo's:
- Piotr Krawczyk : update of the PoC in AUDIO_HAL_CW18
- Unknown User (niskandar): create a picture for next week to show the different options of handling external streams in Android
- Unknown User (niskandar) : Check the list of list of prioritized topics and take care of updating it
- Unknown User (wassim.filali) : could you please work closely with Philippe and Gunnar in order to prepare an comprehensive technical report for the status of the project and an agenda?
Anchor AUDIO_HAL_CW16 AUDIO_HAL_CW16
Thursday 16 April - 11:30am CET (AUDIO_HAL_CW16)
| AUDIO_HAL_CW16 | |
| AUDIO_HAL_CW16 |
Participants
Philippe Robin Gunnar Andersson Piotr Krawczyk Unknown User (bartoszbialek) Suhasini Raghuram Ruslan Murtazin Unknown User (niskandar) Unknown User (wassim.filali) Andrii Chepurnyi @Henric
Agenda
- Discuss the gathered topics, who can work on which topics and which strategy
- Status update on the "access raw streams"
Minutes
Changes done in this page Android and System Level Audio:
- Gunnar: a bit of restructuring was done in order to better assess the project workflow:
- we need to first investigate the topics and choose what topics needs investigation
- Then we can apply analysis of both solutions to the topic
- definition were also added in order for all to agree on the terms that we use
Raw Streams discussion started
- Bartosz: we can get raw streams (unmixed streams) by bipassing the Android framework mixer.
- Applications can request for this, but by defaults all internal streams are going through the framework mixer.
- There is also the Android Audio API which is used for low latency streams that allows us to get raw streams.
- Gunnar:Unknown User (bartoszbialek) please add your comments in the page Android and System Level Audio
- Suhasini: what about the Network audio devices? can we use the device outbus for direct access of raw streams?
- Piotr: in summary, this API is used to handle addressable ports not for use of raw streams
- Bartosz: this is used for Android to capture external streams and represent them as internal streams but the audio path is longer (more latency)
- Raw Streams are both input and output streams
- Gunnar: when we talk about Raw streams do we want to take into consideration compressed streams
- Wassim: this is a link for the data format https://source.android.com/devices/audio/data_formats
- Piotr: the PCM format is mostly used and since we are starting with a generic solution, might as well not extend it beyond this scope
- Gunnar: what about the encoded and compressed streams?
- Piotr: multimedia framework is doing this and it's not part of the Audio HAL
- Bartosz: Android controls whether we want to take the stream and decoded using the CPU or not (this is called Offloading)
- Wassim: each stream has properties: depth, container size, compression, etc.
- here is a page of more attribute
- gunnar: Unknown User (wassim.filali) : please write down the decision about supporing the comrpession in our scope in the page Android and System Level Audio
- Wassim : As described in the Android data_formats page, Compression is a Property of each audio stream. Although audio streams are most of the time uncompressed (PCM) due to avoidance of compression decompression cpu time loss, it might happens that a source stream is provided compressed by default. In that case the common formats are FLAC (lossless) and MP3, AAC (lossy). Further details about compression would be case specific.
Input streams
- Gunnar: input streams no one yet investigated
- Gunnar: Unknown User (niskandar): investigate input steams as well from the page Android and System Level Audio
Presentation on the status of the POC from Piotr (presentation is attached and will be updated)
- Piotr Krawczyk : please update the presentation attached
- Piotr:Stage 1 of the POC should be finished next week
- Gunnar Andersson: provide a place to add the code of the POC
DONE → 2020-04-20: android-external-audio-mixing repository
- Gunnar Andersson: provide a place to add the code of the POC
- Piotr: the concept is that we can either
- Create a separate framework parallel to the android framework that takes care of accessing devices. Any application that is aware of such a framwork can use it in order to better control the streams.
- Or use the framework to create many devices and connect them as needed.
- Gunnar: this is also nice for systems with different OS
- Piotr: there is still one point of investigation
- the Android Audio API, the one created to allow low latency streams. we have to think how we would handle this.
Gathered Todo's:
- Unknown User (bartoszbialek) please add your comments in the page Android and System Level Audio
- Unknown User (wassim.filali) : please write down the decision about supporing the comrpession
- Unknown User (niskandar): investigate input steams as well from the page Android and System Level Audio
- Piotr Krawczyk : please update the presentation attached
- Gunnar Andersson: provide a place to add the code of the POC → android-external-audio-mixing repository
- @All: please check the topics of investigation to see if you would like to add more topics to the page Android and System Level Audio
- @All check your emails to see if you have received the invitation about the summit
Anchor AUDIO_HAL_CW15 AUDIO_HAL_CW15
Thursday 09 April - 11:30am CET (AUDIO_HAL_CW15)
| AUDIO_HAL_CW15 | |
| AUDIO_HAL_CW15 |
Participants
Philippe Robin Gunnar Andersson Piotr Krawczyk Unknown User (bartoszbialek)Andrii Chepurnyi@Henric Suhasini Raghuram Stephen Lawrence
apologies: Wassim, Nadim
Agenda
- from last week's minutes
- Discuss the gathered topics, who can work on which topics and which strategy
- Status update on the "access raw streams"
- AOB
- abstract of the GENIVI virtual tech summit AASIG Audio HAL workshop
Minutes
list of gathered topics on multi-zone audio management
Philippe: asked whether the participants started to think about the gathering of detailed topics for multi zone audio management analysis
- Philippe: reminds about the project timeline until the GENIVI virtual tech summit workshop scheduled on Thursday 14 My 9:00-12:00am CET
Gunnar: I started thinking about the definition of what a raw stream is
Gunnar: I would like the team to write down the topics or questions or features first and then check whether those fit in one strategy or the other or both
Suhasini: there are some termonilogy questions and concepts to align on
Gunnar: agreed, raw stream is one of them, are there others ?
Gunnar the list of prioritized topics is too high level, raw stream is at a lower level of details
Gunnar: I will create new headings on investigation topics and definitions in the wiki page created by Wassim
investigation topics added: raw streams from Android, input streams to Android
definitions added : raw stream, raw stream metadata
- /TODO/ all participants to populate the above sections with their proposals
hint: reading through the List of prioritized topics for the Audio HAL again and copy/paste whatever seems to you relevant for the work on multizone management
abstract of the GENIVI virtual tech summit AASIG Audio HAL workshop
"AASIG AAL abstract Audio management is one important aspect and constraint for the utilization of Android Automotive for the IVI unit. During Q1, 2020, the AASIG AUDIO HAL project team has finalized the list and priorities of audio management related topics. The team has decided to investigate the software architectural design for audio multi-zone management. The following design options are investigated, (a) all audio management is inside Android Automotive, and AA manages all sinks & sources and (b) all audio management is outside Android Automotive. The team is implementing proof-of-concepts to experiment the design options. The AASIG AUDIO HAL project team will present a status report of the proof-of-concept implementation and explain the design choices made. The following next steps will be then comprehensively debated:
- definition of proof-of-concept roadmap, versioning of proof-of-concept content w.r.t. roadmap milestones
- early assessment of implementation TRL (Technical Readiness Level) and discussion on when and how to reaching out Google
- identification of other proof-of-concepts implementation based of the list of prioritarized topics for audio management"
Thursday 02 April - 11:30am CET (AUDIO_HAL_CW14)
Participants
Philippe Robin Gunnar Andersson Piotr Krawczyk Unknown User (bartoszbialek) Unknown User (wassim.filali) Andrii Chepurnyi Unknown User (niskandar) @Henric Suhasini Raghuram
Agenda
- Slides from Wassim about the two models inside and outside of AA
- discussion of the cases/features that we are trying to solve
- checking the Audio Manager to decide whether or not it needs an update
Minutes
- 2 strategies or options:
- android provides sources and sinks
- Android controls the complete systems
- We will not discuss the need to have such options
- Basically some functions cannot be integrated to AA
- Audio sources coming from android or from external (android not aware of them)
- Each strategy has limitations and can be criticized
- Idea is to get each model and apply to it questions/cases
- Questions like safety sources, raw streams, etc.
Piotr: what do you mean by vendor partition
- Wassim: I didn't want to specify the actual parition because it can be vendor specific
- Gunnar: android concept "vendor partition" is booked, maybe we can find another name like virtual machines?
Piotr: audio sink with headphones, what did you mean?
- Wassim: a sink, it's meant as an Android sink, a sink that is connected to Android
- Gunnar: But is it that straightforward?
- Piotr: it's not, the BT for example is a special case, we would add it to the open questions
Piotr: this is actually something we can work on (refering to the BT case)
- Wassim: I'll add it to the wiki and make it possible for people to add topics
Wassim: we can also think about it that: what questions should be done where
- The output of this project would be a recommendation: we as GENIVI, recommend to use this feature outside of AA and this feature inside of AA, etc.
- Gunnar: this is a nice idea actually to make a hybrid solution because we cannot recommend one solution as GENIVI
- Gunnar: we learn from both approaches and use it to build the hybrid solution
- Piotr: it's also not sure if we can achieve one of these extreme cases completly
Discussion about the page followed to define the new page in the wiki
- Page will describe each topic and we will add comments if we can do it in Android or outside of Android and what areadvantages/disadvantages
- The page will help us reach our recommendation
Piotr: Major problem for common HAL is how to we get raw streams
- Henrik: how to get raw streams, what do you mean by this?
- Piotr: current solution, audio data -> tiny ALSA -> hardware
- To get more flexibility -> not directly to tiny ALSA, direct it to whatever we want
- If we can manipulate this behavior we can manipulate the audio stream and direct it to output
- Gunnar: raw stream is the stream that the source is creating, if we have this stream without volume change, without mixing, without effects, etc.
- Raw Stream: as little change as possible to the stream
- Piotr: I would start by extracting these stream from Android, see if this is feasible
- Gunnar: before implementing or starting we would just need to analyze the current system
- Wassim: we should know if we can extract raw stream from Android or not. Without this point we cannot start the second extreme
- Wassim: hybrid also needs the raw streams
Nadim: What about the point Audio manager update?
- Gunnar: There are updates going on but we are not yet concerned about them
- Gunnar: If we start with outside of android strategy we will come to this subject and check if we need to add to it or update it etc.
Decisions
To proceed in our project of Android and System Level Audio we need several steps:
- Gather topics or questions or features that we would like to check whether they fit in the two strategies
- @all Gather topics in the wiki page created by Wassim
- Analyze the topics in both strategies to see if they are feasible, if they are easy to implement, etc.
- POC and code for each topic in the different strategies would be needed
- Conclude GENIVI recommendation from the POCs made: form a hybrid solution.
- Which topic do we recommend to be implemented in AA and which outside of AA
Starting point would be to check if we can access the raw streams from AA
- Piotr Krawczyk: Tieto suggested to take responsivbility of this task, expect 2-3 weeks for a POC
Next Agenda (for AUDIO_HAL_CW15)
- Discusss the gathered topics, who can work on which topics and which strategy
- Status update on the "access raw streams"
Thursday 26 March - 11:30am CET
Participants
- nadim, henric, bartosz, piotr, lukasz, suhasini, wassim, philippe, gunnar
Agenda
- how to control multizone from the system service
- reference system design
- AOB
Minutes of AUDIO_HAL_CW13
how to control multizone from the system service
- tietoEVRY shows slide deck presenting the results of the PoC experimentation they made
- under the hood slide: this a long away to go to achieve a simple functionality
- useful links for understanding audio zone management in Android Automotive
- discussion on the slides follows
- Nadim: you cannot change zone w/o stopping the playback, that's a negative or missing point
- the use case discussed in the previous call (push a button and change the audio focus) is discussed again
- Piotr: the approach was not to change the AA framework to implement this PoC
- Piotr: I would recommend to build a new functionality based on existing functionalities, i.e. like the development of a get UID of active tracks, create such a service would allow the driver to manipulate the UI for redirecting applications to a given audio zone
- Wassim: are these zones like "play in the front or in the back", or "play on the main speakers or on the headphones
- Piotr: for each zone, the context is defined
- Wassim: how does the balance function interact with the multizone management ?
- Suhasini: are the zones more logical than physical ?
- Piotr: Google says a zone is a collection of devices (look at the definition at the beginning of the slide deck)
- Wassim: the device can have a mapping of the different speakers, this is different from a setting like the balance
- Wassim; do we have a limit of the number of apps that can be mixed ?
- Piotr: there is no limitation AFAIK
- Nadim: there is a table (as Piotr said it's static) that determines what audio can be mixed https://source.android.com/devices/automotive/audio/multi_zone/audio-focus
- Wassim: let us assume the actual hw have a limitation on the number of audio channels that can be mixed at the same time
- Bartozs: when the limit is encountered, the audio HAL should report an error which is communicated back to AA
- Wassim; the limitation does not come from AA, but the error can be reported (general error handling in AA)
- Piotr: the audio focus is separate for each audio zone
- Nadim: what is the A2B in the presentation?
- Piotr: AA is missing support for A2B
- Suhasini: yes we know about this.
- random thoughts
- Gunnar: let's think of a multizone concept outside of AA because:
- from history many systems (like MOST based for example) are outside of the Infotainment Head Unit, getting the raw data streams out and they would rely on local DSP or amplifier to tune or do an effect
- (alternatively, if you wish) think of it as just a thought experiment to see what problems we might encounter to understand better the concept
- → is there any limitation in the AA model? let's discover it
- (Question: But why not just assume AA to be in charge of all the audio management?)
- Gunnar: Another reason is the complexity of real car development. Consider car models with different (Infotainment) variants where the one variant might not have AA but are based on a different OS? (You may then be forced to use solutions that work across all those variations of systems)
- Henric: in Audio HAL there are options to access input/output streams, this can be useful
- Gunnar: if we are under the Audio HAL we have the possibility to do whatever we want with the streams
- Wassim: so this gets us back to the main topic of thinking about managing the audio with AA in control and with AA not in control
- Wassim: we need to think more about which model is better suited for our cases
- Nadim: I think it's better to follow the AA in control model because it might be cheaper in the long run and because it includes a lot flexibility
- Wassim: how much flexibility though? does it cover all of our cases? what are our cases actually? maybe we can gather the cases for next week
- Piotr: what direction should we continue with the POC?
- Wassim: I think we should determine what model is better: AA in or not in control
- Piotr: so the AA not in control would be some sort of a Framework (FW) service that is injecting audio data to AA
- Bartosz: here are some inputs from my side:
- AA is expecting that some sources are not handled by Android, this is how it was designed
- Specifically in Android10 they dropped Auto ducking because they reached the conclusion that they cannot control all sources
- Audio zones is limited to AA but we can still use external audio management (https://source.android.com/devices/automotive/audio#external-streams)
- Philippe: how do we manage Safety critical applications (ADAS for example) ?
- Piotr: this can be handled by the external management that Bartosz mentioned
- Gunnar: let's think of a multizone concept outside of AA because:
GENIVI Spring Virtual Technical Summit
- the virtual tech summit will be organized as half-a-day workshops scheduled to be either at US-friendly time or Asia-friendly depending the expected geo-location of the main participants (all times are CET)
- Tuesday 12 May afternoon (US friendly): president's keynote and state of the union followed by a couple of sessions about "looking forward" topics
- Wednesday 13 May afternoon (US friendly): AASIG VHAL engineering workshop (3 hours)
- Thursday 14 May morning (Asia friendly): AASIG Audio HAL engineering workshop (3 hours)
- Thusday 14 May afternoon (US friendly) : Cloud & Connected Services engineering workshop (3 hours)
- this is still TBC but please make a note in your calendar
Decisions taken in AUDIO_HAL_CW13
- Agenda for AUDIO_HAL_CW14:
- Slides from Wassim about the two models inside and outside of AA
- discussion of the cases/features that we are trying to solve
- checking the Audio Manager to decide whether or not it needs an update
- @all: Think and prepare diagrams of how you see the models of AA in control of the audio management and AA not in control of the audio management
- @all: Check what features and what cases do we have in order to validate both models
- @all: Prepare any questions about the Genivi Audio manager, mainly think if it's worth it to update it. Is AA audio manager missing some features?
- Unknown User (niskandar)focus on taking minutes of meeting
Thursday 19 March - 11:30am CET
Participants
- nadim, henric, piotr, wassim, suhasini, gunnar, philippe, stephen
- apologies: bartosz bialek
Agenda
- GENIVI Audio Manager Q&A
- how to control multizone from the system service (skipped, scheduled for next week)
- AOB
Minutes
GENIVI Audio Manager Q&A
Henric: I looked at the document, the AA framework also does audio management, how AM & AA both fit together ?
Gunnar: the main idea was to learn from the AM which solved this issue in existing production systems, the goal is not likley to reuse the AM code directly
Wassim: we have already systems using AM, if AA were used because it can meet the AM specifications, this would be fine
Nadim: I looked also at the document, what the AM is all about ? why is the rationale for using the AM ?
Gunnar: we said that reading the AM specifications is to learn about the concept and possibly reuse the concept
Wassim: we are doing automotive, does everyone have an automotive focus in the team ?
Gunnar: yes
Wassim: we need to look at system level, we can have a head unit, rsa, headphones for passengers,
Wassim: while each vendor develops their own ECU, the problem for the OEM to set/control/configure all the parameters in a centralized way, we need to control things in a centralized way,
Wassim: AM makes only the connection between sources and sinks and relies on other libraries to support the audio stream communications
AA offers the connection to Alsa at a device level, but fails to handle the connections at system level
this is the interested part of the discussion we should have: the handling of multiple connections is to be handle in the vendor partition (Audio HAL)
Wassim: I am interested in vendor agnostic (and OEM agnostic) solutions to solve the handle of multiple connections
Nadim: we would recommend to reuse the AM then
Wassim: you do not need an AM everywhere in the system, you need however a routing adapter in every ECU of the system
discussion on the architectural design of the AM (AM daemon, plugins)
Gunnar; the AM daemon can be used by everyone *unchanged*, the plugins have to be adapted to a given project details
Wassim: if you have multiple (audio) domains, you might also vendor specific buses for each domain, so you will need gateways to convert from one domain to another
Wassim: you informed the AM that you have two domains for instance and the AM will know about the routing to provide
Gunnar: if we look at the AA system, do I simplify too much if I say we want the AA system to produce all streams, controlling the mixing and priorities ? (to be re-captured, Gunnar Andersson please update)
Wassim: let me give an example use case
I have one button somewhere and I want to select the AA radio and route the audio stream to the device where I touched the button
Wassim: our goal is to give the capability of having all devices on the same SoC and having the external amplifiers in another location
Wassim: it would be nice also to have classes of sources (entertainement, announcements, etc.)
Gunnar: what is Tieto thought on this ?
Piotr: there are classes in AA, there is a hardcoded matrix to describe them since AOSP 10
AA exposes those to 3rd party applications, there is a framework layer and the HAL layer, are we fine with the features supported by the framework layer ?
discussion on how to use the framework features (audio flinger for instance)
is the framework functionality enough for implement what we need in the car ?
Gunnar: IMHO there are too much features in the AA framework (to be re-captured, Gunnar Andersson please update)
discussion on what we should standardized outside of AA
Wassim: could we iidentify and name the 2 following scenarii ?
1 android manages all sinks and sources, i.e. all audio management is inside AA
2 all audio management is outside AA, this is the so-called simple scenario where the audio management is done externally
Nadim: what is the expectation for the poc ?
Philippe: the objective of the poc is to validate our design and check it supports the required flexibilty
Wassim: scenario 2 control can be in a different software partition or SoC or an external ECU
Philippe: we need to agree on a reference system design to implement the poc use cases
/TODO/ all participants to produce their own system design from which we will identify commonalities and agree on a reference system design
Gunnar: Piotr was already assigned a presentation on multi-zone management (for next week)
Nadim needs to understand better the problem
Gunnar: how to manage an external amplifier with AA is a use case by itself
- discussion on how to jump start on the reference system design
/TODO/ Wassim to initiate a wiki page describing the block-diagram of a possible reference system design
Thursday 12 March - 11:30am CET
Participants
- bartosz bialek (tieto), nadim, henric, piotr, wassim, ruslan, suhasini, gunnar, philippe
Agenda
- GENIVI Audio Manager - Q&A after review of the docs as discussed in last week's call
- PoCs design
- Project workplan
- AOB
Minutes
GENIVI Audio Manager
Nadim: stiil need more time
Piotr : still on my todo list
Suhasini: went to the architecture overview
Henric : on my todo list
Gunnar: if this is the right way of doing ?
Nadim: Philippe is right, it should be on the top of the list
Gunnar: the AM is how to design an audio system which is flexible
Wassim: do we agree we all need a vendor partition distinct and beyond from Android Automotive ?
PoC design
- Wassim: we need to define simple use cases and demos in order to work out the PoC design and derive how it could look like w.r.t. devboard, dev kit, etc.
- Philippe: reminds the team about the GENIVI test farm
- Gunnar: test farm is actually hosted by Renesas and uses R-Car boards, NXP board is with Gunnar only obviously
- Gunnar: explains some details of the lava test farm operation
- Wassim: would be interested to have links to the test farm
- link: lava.genivi.org
- Gunnar: the test farm it is not documented yet in the wiki, this topic might be for a later time though
- /TODO/ Gunnar invite Wassim to the BIT call where the test farm is discussed
Project workplan
- Philippe: reminds his concern from a management perspective !
- Piotr: we need to dig Android Multi-Zone concept -- specifically, how you could control multi-zone from a system service such a use case is missing currently
- /TODO/ Pior to prepare a material on how to control multizone for next week or following week call
Thursday 5 March - 11:30am CET
Participants
- Wassim, Ruslan (EPAM, automotive systems) Gunnar, Philippe
- apologies: Nadim, Bartosz, Piotr
Agenda
- GENIVI Audio Manager
- Audio HAL project workplan
Minutes
Roundtable
- Ruslan represents Andrii in today's call
GENIVI Audio Manager
- Q&A after review of the GENIVI Audio Manager docs is shitfted to next week's call
recap on the decision made by the team last week to get awareness on the GENIVI Audio Manager
Gunnar: the code might not be reusable as such in an Android Autmotive context, but it is worth understanding the archictectural design of the Audio Manager
Gunnar: Android is lacking the flexibilty audio systems have in a car, this project intends to enhance this
Wassim: there are deployed systems using the linux based GENIVI Audio Manager or using Android Automotive, who should take care of the audio management in the Android context ?
Gunnar: there 2 tracks one is to find out the audio management requirements in the Android Automotive context, track #2 is to achieve this with the appropriate technologies, we need to decide first what we want
Wassim: we need to define the common problem statement the Audio HAL wants to solve
Discussion on the Audio HAL project workplan
- Wassim: Audio effects are critical because they fall into the domain of custom interfaces in Android Automotive
- Wassim: the networked audio systems exemplify why a car is different from a smart phone
- Wassim: my vision is that with the networked audio systems Android Automotive will become an OS for distributed systems
- Gunnar: GENIVI is now focusing on system-level integration
- Gunnar: for the VHAL our objective is not only to provide more vehicle data (than the Google vehicle properties), it is also to dig into the Android Automotive based head unit and integrate with the rest of the vehicle systems
- Wassim: Android Automotive uses a wrapup around Alsa, we do not need to change this, what we should look at rather is how to solve the custom interfaces in a general way as the VHAL is doing for the access to vehicle data
- Wassim: what about graphics ?
- Gunnar: graphics came as a side track in the GSHA project, we look for instance at the combination of the Wayland (linux) stack and the Android Automotive stack, this was not in the AASIG actually, there is one company working on an implementation for this
- browsing through the collaborative tools for AASIG: wiki structure and jira project
- Philippe: explains the way we structure the work in the SIG from the identification of problems to solve, the brainstorming on architectural design options and possible prood-of-concepts to validate the architecture concepts and the definition of work breakdown structure for the PoC implementation
Thursday 27 February - 11:30am CET
Participants
- Nadim, Bartosz Cichosz, Gunnar, Andrii, Bartosz Bialek, Piotr, Stephen, Wassim, Suhasini, Henric
- apologies: Philippe
Agenda
- Networked audio devices
- PoCs planning
Minutes
Networked audio devices
Gunnar: Le'ts start with the networked audio devices overview. It can be good input information before discussing PoC details
Suhasini presents her slide deck (link)
Wassim: When you say streams, (and the red lines in the picture), what does it mean?
Suhasini: It represents a network connection of some type (any type). Streams, I mean an audio stream logically from one device to another here. On the next slide there is a mapping shown.
... mapping slide
... Automotive Audio Bus (A2B) slide.
A2B is an example of single master, multiple slaves setup.
It is routed according to your configuration.
You can for example configure that mic input must go to head unit. Protocol takes care of it according to configuration.
Problem: Physically speaking the head unit is connected to only one device. The system views it as a single device.
On the logical picture, one stream is connecting to more than one receiver. Viewing as a separate device is useful but...
Android 10 features has Audio Device Out but, but it's not clear if we can address (receiving nodes) as we want.
Wassim: One device visible but many logical channels...
Suhasini: Problem 2 could be diagnostics support. Checking if the node is in an OK state...
Control path should be able to support more than volume. E.g. a series of diagnostic routines that can be initiated.
ALSA has minimal support also for this.
Wassim: ALSA supports well the (audio) data flow but not control. GENIVI audio manager has the primary purpose to define such interfaces. (It is not handling the audio stream directly).
The drivers (from silicon vendor) might still be implement the control function itself.
Wassim: The vendor needs to provide a HAL for those control functions. If Android defined the interface, this might be implemented.
Summary: View each device stream separately inside Android.
Discussion yields that we should perhaps handle each stream separately, in addition to devices. (sink and source abstraction)
Agree that each single stream could have types (mono, stereo, multi-channel) but we don't expect to need to handle reconfigure channels.
Gunnar: In addition to differentiating the individual receiving devices I expect we need to consider that one or several streams per device?
... how to implement additional features ...
Wassim: GAM for example has a generic support for "Properties" from user to sink, and "Notifications" from sink to user.
Gunnar: Note: "user" here means the controlling software, either the application or the main system HMI that is defining the behavior we want to have from the audio system.
Wassim agrees.
Discussion on other needed control interfaces... example: firmware update?
More discussion..,
Suhasini: In order to transfer a certain stream e.g. phone input to some output, this in the end needs to be configured in hardware...
Initial configuration will be created by the vendor, but some kind of interaction with the driver is needed.
Wassim: Look at open documentation to AudioManager. The routing plugin abstracts the logic of this. The implementation is doing the connection between hardware specifics and the generic interface.
I have seen that Android also allows some kind of custom bus implementation, but have not yet seen if this solves it.
Bartosz (Harman) - short introduction. Android engineer, not specifically focused on audio however. Based in Poland.
Action (all): Read up on the architecture and interfaces of the GENIVI Audio Manager, since it was designed to make a framework for exactly this. Docs: https://genivi.github.io/AudioManager/
The ideas / design / APIs should be reusable. Possibly some code also, eventually.
Stephen: Are those docs also covering plugins?
Gunnar: To further understand plugins, try looking at code of the plugins: https://github.com/GENIVI/AudioManagerPlugins (The code of the AM daemon is at https://github.com/GENIVI/AudioManager/ )
Stephen finds another slide deck describing routing-adapter.
PoC planning
We had 2 main topics: Global Effects API and more flexible audio channel routing.
Gunnar: OK so the discussion we had before led forward in the audio channel routing topic. What about Global Effects?
Bartosz (Tieto) reports that the plans for the next Android version include new interfaces that seem likely to solve the global-effects problem.
It is possible therefore that we do not need to work out solutions for global effects: LINK 1
, LINK 2, LINK 3
Adjourned at 12:25.
Thursday 13 February - 11:30am CET
Participants
- Suhasini (Analog Devices), Henric (Bosch), Nadim (Mobis), Bartoz (Tieto), Philippe
Agenda
- Overview of last week's F2F outcome
- Call for participation
- AOB
Minutes
Overview of last week's F2F outcome
- Philippe presents the outcome of the meeting using this slide deck
- Meeting was productive and the team had a very good collaborative spirit
- Minutes of the F2F meeting are here
- Nadim: minutes are very helpful
- Philippe: asks participants for their feedback and comments in the wiki
- Suhasini: I would like us to refine further the scope of the PoCs
- Suhasini: I would like to understand better what was discussed on AVB in the meeting
Nadim: Bartoz, Gunnar and Piotr wanted to know about which technologies Analog Devices has in mind for the networking of audio devices, this is why AVB popped up
Call for participation
- Philippe calls for participation for refining further the PoCs scope and initiating architectural design work
Suhasini: /TODO/ in 2-week time, will give an overview and a problem statement for the networking of audio devices
Audi HAL call schedule
- Next call will be scheduled on Thursday 27 February at 11:30am CET
Audio HAL F2F Meeting 4-5 February
23 January 2020
Participants
- Andrey, Piotr, Wassim, Henric, Philippe
Minutes
upcoming F2F meeting
- agenda is here, Audio HAL part is scheduled on Day 2 morning
- review of high priority topics selected for discussion at the F2F: Common HAL & Source Management
- assignment of preparation work done (although one is TBC)
AOB
- CES debriefing: GENIVI showcase at CES went out well, GENIVI hosted 1400 people, feedback about the event was very positive
- AMM: the Spring AMM will be scheduled in May, dates and location should be announced soon
19 December 2019
Participants
- Andrey, Suhasini, Piotr, Stephen Lawrence, Gunnar, Philippe
Minutes
upcoming F2F meeting for the Vehicle HAL project
- decision on the dates was made => 4-5 February, at BMW, Munich, Germany
- Philippe: we have an opportunity to coschedule (and colocate) a meeting for the Audio HAL project
- /TODO/ all fill in the participation table and indicate whether you would be available and interested by joining a meeting either F2F or via telco
Review of the list of prioritarized topics (continuation)
13- BT handsfree
Suhasini: it is rather a question of multisource management
Andrey: would agree it is related to multisource management
Gunnar: on the mobile phone, the BT role is the server role, the car is more the handsfree role
Andrey: we need to clarify the automotive use cases
the group should work on use cases but there are more fundamental questions to be solved on multi source management
14- Android does not implement all features required by customer.
Andrey: this is about multisource management, support of external devices, limited number of priorities, etc. the various bullet points can be distributed over the table
/TODO/ Andrey distribute the bullet points among the table
Piotr: what are the priorities you are talking about ?
Andrey: these are the priorities related to phone calls, I will clarify this when I do the split (phone calls vs. not phone calls)
15- Android Audio subsystem is developed only for infotainment purposes. Safety-related features need to be implemented in another RTOS
Andrey: thinks the safety-related audio features are not in the scope of this group
Piotr: we need to keep in mind that there is a higher priority audio functionnality in the system
Gunnar: it might be worth discussing how AA can use external components that manage the higher priority audio functionnality, these are system-level considerations
/TODO/ Gunnar add HERE what was discussed in this call about the system level architecture Thanks
Gunnar: do we look for solutions on AA side first or do we look also at the general system design for audio ? my recommendation is to start something concrete at the design level
16- Android way of extending its functionality is developing vendor extensions without modification of the framework
Piotr: we need to avoid changing the framework because there is a lot of manual work necessary to track the changes that need to be taken into account
Andrey: agreed
Andrey: we have a mechanism in Harman to follow up the changes in AOSP
Gunnar: we need to start the design work asap and as a general principle we do not change the framework
17- Google is still very “hand wavy” about solution
Gunnar: WR has a lot of work to implement the audio HAL by themselves
Piotr: people look at the code of android directly, I would like to get some tooling to cope with the fact the doc will not be provided by Google anyway
Piotr: tool can take care of the configuration setting, helping with work we do currently manually
Andrey: we need to adapt to the Google way, we cannot change this
Piotr: we can translate HIDL to Franca and provide a more flexible way of implementing things
Andrey: we are considering similar ideas within Harman
Gunnar: agreed, it is part of the discussion on how to use a common interface in multiple systems (with the remote interfaces, not only on AA), something like CommonAPI, there is a generic discussion to have there
18- Genivi ?
Gunnar: in my opinion this is a question about the role of GENIVI, we need to think about what could be the outcome of the group, as I said before, the question is rather to find solutions, it can be anything from a formal specification, to guidelines and even code
- category changed to "Deliverables"
- Philippe: IMHO this is also a question about the recycling of earlier work done by GENIVI, e.g. the audiio manager, we said we would organize a presentation of the GENIVI audio manager and the automotive use cases it supports
- /TODO/ Gunnar plan a presentation of the GENIVI audio manager in Q1, 2020
list of prioritarized topics
- /TODO/ all review the minutes of 12 & 19 December together with the list of prioritarized topics and identify any gap / missing points in the list
Next call
- Thursday 23 January, 15:00 CET / 20:30 India / 6am US Pacific
- agenda
- review comments for the list of prioritarized topics
12 December 2019
Participants
- Andrey, Suhasini, Piotr, Jimhyuk Jung (mobis), Patrick Carlisle (mobis), Stephen Lawrence, Giovanni, Gunnar, Philippe
- apologies: Wassim
Minutes
Roundtable
- everyone delivers a quick personal introduction
- Jimhyuk works for mobis technical center Europe (MTCE), he is an IVI expert
- Patrick works for mobis technical center America (MTCA)
- Piotr indicates he would like to have a less hardware aware Android
Review of the list of prioritarized topics
- list upated online, each contributor explains the rationale for setting the priority he chose
- 1- Networked Audio Devices: Support for the configuration of networked audio devices. Currently Android can view it only as a single device/sound card. An audio network in car might have multiple devices that needs to be controlled by the Android OS
- 2- Overall configuration management
- Piotr: there is no way to automatically validate the configuration
- Andrey: IMHO there is more important things to solve than the configuration
- 3- Common Audio HAL
Piotr: audio HAL is not so HW dependent because of tinyalsa, the porting the common audion HAL should be limited to tinyalsa
Andrey: I do not think whether we can make a common audio HAL, it would be great to have such a common HAL for using eAVB ?
Piotr: some hw vendors are working on it, must be feasible
Gunnar: this is about finding the common part, this is about avoiding duplicated effort to provide the same functionalities, this about reducing the amount of effort for each vendor to implement the HAL
- 4- Audio Data Transfer
Suhasini (not captured)
Piotr; utilizing a DSP is very hw specific, this is why I put it a low priority, however it is important, it should be high priority
Gunnar: this is about finding the right abstraction
Piotr: we need to talk about interfaces / abstraction to create audio effects
- 5- Equalization
- all agree it is high priority to have a way to control the equalization
- 6- Audio calibration
- Suhasini; audio calibration sounds to me like controlling global effects, not as big a problem as others
- Piotr: agreed, could be covered by the same interfaces, there is a need for the a generic interface
- 7- Controlling audio effects
- the above 3 items could be covered by the same interfaces
8- Latencies
Suhasini: for Analog Devices, latencies are important because we have a lot of time sensitive applications, currently we are doing these on the DSP, we would be interested in moving some of them on the Android system
Piotr: IMHO the latencies question is pretty hw specific, it cannot be solved in a general way, this is rather Google responsibility with their framework
Piotr: another problem is shared audio on a virtualized platform, not sure we can solve this problem in a joint effort
Gunnar: in the GENIVI Hypervisors (HV) project, Opensynergy is behind the proposal for a set of standard interfaces for the audio platform, we could dig into it if there is an interest in the Audio HAL group
/TODO/ Gunnar organize a presentation of the work done by the HV project on the standard interfaces for the audio platform
Andrey: we encountered a problem with latencies when we switched to hypervisord, actually we had problems with latencies AND scheduling
Gunnar: it is a difficult challenge, agreed
- 9- Multiple audio channels
- Bosch indicated their challenge is to adapt the AA framework to their HW I/O, e.g. they have 4 audio channels while they need to present them as 2 audio channels to AA
Piotr: it is hard for me to judge because it is very specific, my claim is that it can be solved with AA
Andrey: I do not know it was feasible
Philippe: this needs further investigation
10- Early audio for RVC
Andrey: early audio RVC will be rather handled by the HV or RTOS, because there are safety requirements which cannot be achieved by AA
Suhasini: what is the audio application for RVC ?
Andrey: (not captured)
Stephen: there might be one opportunity for standardization, some people manage the early audio on RTOS and then switch over to AA framework. Can the methods on Android side by standardised? This also applies to other 'early' features such as video for reversing camera.
Piotr: one possibility is to have 2 audio HALs, one "simple" audio HAL for early audio and one full feature audio HAL
Philippe: iy would be good to look at a standardized way of switching modes
Andrey: we still have the safety requirements for this
- 11- Source Management
- Multi‐source multi‐sink, etc.
- Piotr: this is in the framework, it maybe not be worth looking at it as first priority, this is why I put a low
- 12- Audio focus
- Andrey: it would be great to get applications on AA to request audio focus (not optional audio focus), usually automotive use cases have a limited
number of sound applications
Patrick: at Mobis we have our own sound manager and pass it to AA, adding a little man between the sound applications and AA works well
Patrick : I do not know much effort we put in this tool though
Gunnar: IMHO the concepts supported by the GENIVI audio manager component might be relevant for this issue
/TODO/ Gunnar contact the "right" person (i.e. who knows the GENIVI audio manager component) and organize a presentation of the concepts supported
- review of other topics will continue next week
F2F opportunity in early Q1/2020
- Philippe: the Vehicle HAL project intends to have a F2F meeting in early Q1/2020, BMW will host the meeting in Munich, Germany
- Philippe: it might be an opportunity for the Audio HAL project to meet F2F at the same date & location
- /TODO/ all check about your availabilities and the clearance of travel expenses
Next call
- it will be scheduled on Thu 19 December at 11:30am CET (usual time)
4 December 2019
Participants
- Andrey, Suhasini, Wassim, Stefan, Maria, Pontus, Gunnar, Philippe
Minutes
Introduction
- Philippe reminds all participants to identify additional extensions to Audio HAL that would not change too many things and identify a part of the Audio HAL that could common for all SoC vendors
- this analysis needs more work and consensus reaching within each participant's organization
- Suhasini asks whether the list of topics is available in the wiki
/TODO/ Philippe merge all inputs from minutes and slide decks into a single wiki pageDONE- page is List of prioritized topics for the Audio HAL
- tentative categories have been added: configuration, common audio HAL, performance, multi-source management, multi-OS management, extensions. All categories are To Be Confirmed (TBC) and the mapping of categories to topics are all TBC.
- /TODO/ all participants review the list of topics, categories, mapping and add priority (high, medium, low)
Configuration & Common Audio HAL
- Piotr shows the slide deck he presented at the tech summit in order to exemplify what our workplan should consist of
Next call
- Thursday 12 December at 16:00 CET / 7am PST (unusual time)
F2F meeting opportunity
- Vehicle HAL project will meet F2F at the end of January / beginning of February, this is an opportunity to colocate a F2F meeting for the Audio HAL project
- please look at Vehicle HAL F2F - 4-5 February 2020 - Organization and fill it in (you may have to ask for credentials)
21 November 2019 - Project Scoping - Second Call
Participants
- Andrey (Harman) Sw architect
- Suhasini (Analog Devices) SW engineer for the car infotainment
- Wassim (BMW) Audio team SW architect, works on cross-ECUs function, audio manager, specific implementation, needs to clarify his position w.r.t. the maintenance role for GENIVI audio manager
- Stefan (Bosch) component maintainer for Audio at Bosch Car Multimedia, Hisldesheim, Germany
- H. Carlsson (Bosch) Bosch Car Multimedia, Lund, Sweden
- Bartosz + Piotr (Tieto)
- Gunnar
- Philippe
Minutes
Introduction
Philippe welcomes participants and details the agenda
Philippe recommends the review of the presentations delivered by Tieto and Windriver at the tech summit in Detroit where they presented their return of experience on Android Automotive, in particular on Audio HAL
please look here, search for "return of experience"
Introduction to Audio HAL
- Bartosz presents this slide deck
- We have more than 1 Audio HAL module in the system. There is always primary audio HAL that handles primary output (car speakers) and primary input (mic, aux, tuner, etc.) and in Android Automotive usually we also have USB audio HAL (dedicated to USB audio) and A2DP audio HAL (dedicated to Bluetooth A2DP). Different Audio HAL modules don't talk directly to each other, but the routing/bridging between them is handled via AudioFlinger/AudioPolicy based on audio policy configuration (where routing topology is defined).
- slide 6: grey boxes are external to AA
- Wassim: question on slide 14 about configuration
- Bartosz: configuration files are in the system partition or in the vendor partition, we lack a good concept for configuration, and for the persistence of configuration as well, support tools would greatly help
Gathering of list of topics for the Audio HAL project
- Bosch: we have issues with the BT audio stream / BT handsfree but I am just starting digging into it
- Gunnar: I am surprised that the BT handsfree is not part of the Google AA code
- Bosch: there is some support in the framework to support handsfree
- Gunnar: is the basic hansfree profile implementation part of the BT support in AA ?
- Bosch: pairing the phone with the device works but there is a need to connect the audio
- Andrey: shows a list of questions / problems, look at the list below
- Android does not implement all features required by customer.
- Audio focus can be lost at any time
- Limited number of sources
- Limited number of priorities -2
- No easy way to support external amplifiers
- No priorities of phone call types.
- Android Audio subsystem is developed only for infotainment purposes. Safety-related features need to be implemented in another RTOS
- Need to share the same hardware between 2 OSes
- Running Android as a virtual machine inside RTOS leads to problems with scheduling of audio processes
- Android way of extending its functionality is developing vendor extensions without modification of the framework.
- But in some cases you need to modify framework. There is a process for this, but slow. And changes needed to one customer / developer is not needed for another. Even if you do this, you have to wait until Tier 1 merges this change to its drop
- You have to retain compatibility for third-party apps
- Android does not implement all features required by customer.
- Gunnar: what are the most important topics to tackle on ? maybe there is a particular problem to look at first
- Wassim: there is some topics that are low-hanging fruit
- Stefan: IMHO some of the topics are not related to Audio HAL but rather system-level architectural issues
- Gunnar: Audio HAL is much more complex than the Vehicle HAL
- Gunnar: in the Vehicle HAL we are finding solutions that can sit aside from AA, we do not intend to make a fork which would provoke a lot of
integration problems - Stefan: priority of sources varies among OEMs, we would need a better configuration capability from Google to handle this better
- Bartorz: extending the framework is fine provided you do not change the APIs, the HAL is actually something where we can put additional stuff "for free"
- Bartoz: this is a much bigger challenge to work out a common Audio Hal and port it to different SoCs but it is interesting to do
- Gunnar: agreed
- /TODO/ all participants identify additional extensions to Audio HAL that would not change too many things
- /TODO/ all participants identify a part of the Audio HAL that could common for all SoC vendors
- /TODO/ Gunnar and Philippe create the wiki page to gather the priorities of topics of interest listed so far (from minutes of first meeting and presentations delivered by Tieto and Windriver at the tech summit)
Adjourned: 18:20 CET
7 November 2019 - Project Scoping - First Call
Participants
- Andrey (Harman) principal architect, Android expert, based in Nijni-Novgorod, Russia
- Denis (Harman) manager of Android COE team (Center Of Excellence)
- Alexander (Harman) workis in the COE for low level Android
- Suhasini (Analog Devices), automotive infotainment software engineer, based in Bangalore, India
- Piotr + 1 (Tieto) mobile devices, automotive HW layer of Android, based in Poland
- Maria + 2 (Bosch Car Multimedia) manager and two SW engineers
- Gunnar (GENIVI)
- Philippe (GENIVI)
Minutes
- Philippe reminds the objective of the call (or of the first series of calls rather) which is to define the scope of a Audio HAL project
- Suhasini: we have not much experience with Android Automotive (AA), we are interested in AA because AA is now in the head unit
- we have a portfolio of audio processors and an automotive audio bus to distribute audio within the car, we want to configure our proprietary audio bus SW stack to use it with AA as we did for Linux
- we want to determine whether we have a problem of bandwith when using our proprietary network with AA, one approach is to use shared memory transfer
in the audio HAL
- Piotr: we identified the following list of problems / tasks with AA when integrating the HAL for silicon vendors and OEMs
- configuration of Automotive overlay, there is not much tool to configure the AA system
- there is no way to control the equalization, i.e. no simple way to control global effects for output streams (by default Android application controls its own tracks)
- there is no Audio calibration interface
- there is a need for a generic interface for controlling audio effects on HAL level, global effects are designed for input streams but control over them is limited by interface
- early audio for RVC (Rear View Camera) or other services
- configuration management component for TinyALSA
- Audio Focus doesn't forbid to interrupt Audio, Android 10 provides additional interface for Automotive to solve this problem
- Suhasini: we would like the following topics to be investigated
- Audio data transfer / streaming to a co-processor for post processing of audio from the HAL/other Android layers
- Support for the configuration of networked audio devices. Currently Android can view it only as a single device/sound card. An audio network in car might have multiple devices that needs to be controlled by the Android OS.
- Bosch engineer: we are working on two things, with the audio HAL we received from our silicon vendor
- 1- patches for handsfree management, combination of BT stream with other streams, how to enable the correct audio routing, documentation on the Audio HALis missing, it is difficult to find examples on line
- 2- I/O for our Bosch HW, the challenge is to adapt the AA framework to our HW, which has for instance 4 audio channels while we need to present them as 2 audio channels to AA
- Gunnar: how many of you are familiar with the Audio Manager of GENIVI ? I would recommend you to look at the features supported by GENIVI Audio Manager because it has been designed for automotive audio specifically
- Gunnar: introduces shortly the Audio Manager daemon and plug-ins
- Harman engineer: using Audio Manager am implementation designed for linux would raise an issue because of the Google CTS
- Andrey: safety related audio features can likely not be implementted in AA because AA is not a real-time OS
Next steps
- Gunnar: @Harman - would you consider making a list similar to what Tieto presented ?
- Suhasini: having an overview of the AA audio HAL would help ? can someone give such a presentation ?
- /TODO/ Harman & Bosch provide a list of topics that could be worked on jointly
- /TODO/ Philippe create a wiki page DONE
- /TODO/ Piotr prepare a short presentation on the AA audio HAL so that we share the same understanding of Audio HAL features
Next call
- we will have calls every other week, on Thursdays at 11:30 CET
- next call is scheduled on Thursday 21 November, a calendar invite has been sent