Hypervisor Workshop Schedule at Munich AMM 2018

NOTE: This is for 2018. The schedule for the following AMM is here.

Schedule used at GENIVI 18th All Member Meeting ~ Munich

Complete minutes – further down the page!

Schedule
(Topic name & Responsible & Timeslot)

Presentations can be found here – look for the Hypervisor Session

Workshop introduction and intention 9:00-9:05 5mn
- presented by: Gunnar (GENIVI), Sang-Bum (Perseus)
History of HVs 20mn presentation 9:05-9:25
- presented by: Sang-Bum
Introduction to XEN project – 9.20-->
- by: Lars Kurth (Xen project)
Market Overview 20mn presentation 9:40-10:00 (->9.50-10.15)
- presented by:Franz Walkembach (SysGo)
- includes an intro to certification
HV design and implementation 10mn + 15mn presentation + discussion 10:20-10:45
- Introduced by: Ralph (Open Synergy)
>>>> short break 15mn 10:45-11:00 CET
Requirements gathering 10mn intro + 10mn discussion 11:00-11:20
- HV vendors asking OEMs/adopters/customers/etc to clarify technical requirements
- Introduced by: Matti (Open Synergy)
Virtualization for Multi-core, SoC peripheral hardware and special- purpose CPUs 10mn intro + 10mn discussion 11:20-11:40
- Introduced by:Artem (EPAM)

Audio system design with HVs 10mn intro + 10mn discussion 11:40-12:00
- Introduced by: Artem (EPAM)
Standardization of hypervisor APIs (virtio and friends) 10mn intro + 10mn discussion 12:00-12:20
- Introduced by: Matti (Open Synergy) (virtio intro + what's needed?) + ?
*morning wrapup* 12:20-12:30
- Agree on outcome, actions, topics for future project, etc.

>>>> lunch break 12:30-14:00

Graphics/GPU Sharing (in relation to GSHA project) 10mn intro + 15mn discussion 14:30-14:55
- Introduced by: looking for volunteer to introduce the topic
- Artem? Alex? (EPAM)
(Cyber-)Security enhancements based on virtualization 10mn intro + 20mn discussion
14:55-15:25
- Introduced by: Sang-Bum
15:30 - 15:45 Break
Health/Debugging/Analysis/Logging (in relation to SHDA project) 5mn intro + 10mn discussion
15:55-16:15
- Introduced by: Gunnar Andersson (GENIVI)
*afternoon wrapup* 16:15-16:30
- Agree on outcome, actions, topics for future project, etc.

BACKUP TOPICS

Terms / Nomenclature?
Hypervisor project organization (post-AMM)
How can reliability & safety be quantified?
System design to optimize Boot Time,
Boot requirements, e.g. secure boot, integrity check,
System design and working with HVs from users/system integrator's perspective.

Defining next steps + Hypervisor Project.
Performance comparison between open source software based hypervisors on ARM SoC 10mn intro + 10mn discussion
- Introduced by: Sang-Bum

Minutes from Hypervisor Workshop at AMM Munich April 2018

The presented material (slides) can be found here – look for the Hypervisor Session

____________________

GENIVI

Jeremiah C. Foster
Philippe Robin
____________________

Table of Contents
_________________

2 Minutes from 18th GENIVI AMM April, 2018
.. 2.1 Hypervisor Workshop Introduction and Intention
..... 2.1.1 History of the Hypervisor
..... 2.1.2 Xen Project Automotive and embedded overview
..... 2.1.3 Sysgo AG -- Hypervisor Market Overview
..... 2.1.4 Hypervisor Design(s)
..... 2.1.5 Requirements gathering
..... 2.1.6 EPAM Virtualization for Multi-core, SoC peripheral hardware
..... 2.1.7 Security enhancement on Xen ARM
..... 2.1.8 Standardization of hypervisor APIs (virtio)
..... 2.1.9 Health/Debugging/Analysis/Logging (in relation to SHDA project)

Almost full room

2 Minutes from 18th GENIVI AMM April, 2018
==========================================

2.1 Hypervisor Workshop Introduction and Intention
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Gunnar Andersson, GENIVI
- Dr. Sang-Bum Suh, CEO Perseus Co., Ltd

One of the driving forces of the Hypervisor work in GENIVI is the
Domain Interaction work being done in GENIVI. The Hypervisor Working
Group has really kicked off after Seoul AMM in 2018. The hope is that
after this workshop there is some clearly defined goals to work on.

2.1.1 History of the Hypervisor
-------------------------------

Virtual Machine Monitor == Type 1 virtualization. Runs one or more
virtual machines. Each virtual machine is called a guest. Type 1 is
bare metal, type 2 is with operating OS inbetween. Main use case is
reduction in space, weight, and power. HVs are a scheduling operating
systems.

Implementation of an HV is as difficult as the linux kernel although the concept seems simple

In automotive we see RFQs on digital cluster and IVI on single
hardware or SoC Reuse of legacy code or using existing ECUs for new
projects Fastboot and secureboot, boot via RTOS for example.

Another use case is combining Android / OS software in a secure
environment.

Slide: shared showing example of space separation in automotive.
Slide: HV vs Linux containers

IBM started hypervisors for migration of bank transactions to an
uninterruptable services, done in the 1970s.

Slide showing theory vs. practice. The theory is that HVs are simple,
but in fact are quite complex. Design of a type 1 is HV is comparable
to the linux kernel in complexity. Two scenarios nowadays; SoC BOM
cost savings through consolidation and zero downtime. Without support
of hardware you got a heavy hypervisor and low performance. Wit SoC
redesign you get a thinner HV and better performance.

First comes CPU/MMU Virtualization, the I/O, then PV. GPU
virtualization, embedded in SoC design, ought to provide significant
performance increases.

History of Xen ARM hypervisor Xen on ARM was released in 2008. CPU
overhead 3% after optimizations.

[Video demonstration of HV on cell phone for the purpose of security improvement]

2.1.2 Xen Project Automotive and embedded overview
--------------------------------------------------

- Lars Kurth, Xen Community Manager

Ecosystem overview Xen capabilities and Challenges Safety
certification

Xen moved heavily in cloud computing, but then saw that Xen was being
used in safety cases and started to be adopted for security in end computing devices

Project: OpenXT = xen + open embedded, www.openxt.org, mostly military solutions. Forked Xen
and OE but the hope is that there will be a return of code back to Xen.

Project: Virtuosity from Dornerworks, avionics certified DO-178, IEC
62304, ISO 26262. Went already through the ceritifcation process for a first iteration
Added support for VxWorks, RTEMs, etc.

Project: EPAM Fusion Project: GlobalLogic Nautilus

Automotive vendors add to the community like Renesas, Bosch, LG, Adit,
Samsung.

- Automotive requirements vs. Xen Project

Automotive requirements come from AGL white paper
Eupam running a project on memory/IO bus bandwith allocation and rebalancing
GPU sharing is being generalized to co-processors

- Security requirements, Root of trust and secure boot needs to be
supported (WIP). Hardware isolation, core functionality is there, except firewalls.

- Safety requirements
Fastboot is available Power management is available, more work being
done EPAM, XILINX, Aggios, discussion on accessing memory

- Safety certification
The goal is to make it easier for Xen users to Safety Certify KSLOC
about 70 thousand lines of code The future target is to be around 40K
to 50K

Q: How do we track progress of this work? A: Some things are
happening on the mailing list, but hope is to setup a proper project
infrastructure

Q: What is the cost on boot time of using containers? A: While it
doesn't do everything the cost is nearly zero.

2.1.3 Sysgo AG -- Hypervisor Market Overview
--------------------------------------------

- Franz Walgenbach - Sysgo

Market trends, main vendors, which HV types are visible with the
focus on MMU vs. MPU

Sysgo goal is to be the leading European OS provider for embedded.
Cluster ADAS/Gateway IVI, Virtual ECU, ECU consolidation are the
spaces 500 million dollar market by 2019, so relatively small

2.1.4 Hypervisor Design(s)
--------------------------

- Dr. Ralphe Sasse OpenSynergy

Slide: Xen design Dom0 DomU DomU Slide: KVM is part of the kernel via
a kernel module.

Slide: Possible VM allocation patterns
- 1 core == 1 VM
N cores `= 1 VM aka SMP distribution requires inter-core communication
1 VM =' 1 Supplier 1 VM `= 1 functionality 1 VM =' 1 criticality 1 VM
== 1 timing (real time / best effort)

Slide: KVM
major difference with XEN is that there is no tool stack, otherwise design very similar to xen
Sang bum: there is a large difference in booting approaches and time between kvm and xen

WATCH OUT!!
for an instrument cluster mmi, we are talking about a mmu

for a controlsystem, we are talking about mpu

2.1.5 Requirements gathering
----------------------------

- Matti Möll Opensynergy

OpenSynergy All hypervisors have formal requirements that govern the
design and implementation. Android compatibility Definition Document
(link in slides) Example requirements shown.

Q: Is there any discussion of diagnostics and diagnostic responses?
A: Discussed but perhaps not in detail. Diagnostics will depend on
design, like depending on CAN design

We can use the AGL white paper as a starting point, but we have a less
specific goal than AGL.

2.1.6 Virtualization for Multi-core, SoC peripheral hardware
-----------------------------------------------------------------

- Artem Mygaev - EPAM

Intro on peripherals sharing - design options

example: USBs - how do we handle dynamic devices ?
Ralf: who is the owner of a usb device that pops up ? is it one VM ? is it all VMs ?
Artem: how do you create a policy of what goes where ?

How do you add dynamic devices, adding devices with different
capabilities? Likely better to use para-virtualization. Still there is
a performance hit so not so good for data intensive scenarious.
Discussion on how to add a peripheral and into which domain. How do
you create policies for what goes where and how it goes. Many
co-processors (DSPs, GPUs, IPUs) do not have native virtualization
support. There is a technology called "mediated pass-through" (HMP)
where you have partial pass-through and partial emulation. Can be
emulated in an HV. Emulation and mediation does the context switch.

Q: Franz - GPU sharing - OpenGL virtualization can be done in the GPU or higher in the
stack, do you know the performance cost?
A: With mediated pass-through there is a about a 5% overhead
A: artem; we think we could do better with Vulkan but we have not started on this yet - https://www.khronos.org/vulkan/

Heterogeneous multicore
an example is big.LITTLE from ARM which has different cores
and presents a question on what type of virtualization to use.
It can be difficult to communicate in the VM which core a
thread show run on. You can do a brute force attempt but that might
create performance issues. Also, energy aware scheduling might be
disrupted.

Q: Is there are trend towards an asymmetrical approach? A: The trend
is to make things more complicated

On ARM side we have an enhancement called 'ARM Dynamic' which is fully
scalable but still a cluster architecture. Multiple mixed cores and
much more diverse.

Q: Surely the system architect would be better positioned to know what
the VM gets for a core A: But what about autonomous driving and other
asymmetrical use cases?

note: slack reclamation is a good approach but it is not scheduling
discussion on multiple scheduling layers

2.1.6.1 Audio system design with HVs
--------------------------------------
- Artem Mygaev - EPAM

There needs to be a control logic because there are policies and modes
of operation around sound in the vehicle. (Analysis of pulse vs. Alsa)
We should follow up on Xen changes to GENIVI Audio Manager to have a
discussion on their patches and design.

Artem: there are some scenarii available in GENIVI audio manager that
would be worth being investigated in the context of domain interaction

there are pros & cons in using GENIVI audio manager

/TODO/ Jeremiah (as Community Manager) ask GENIVI audio manager people to answer Xen request on pros and cons

Q: How to hand different DSP features? A: Various DSPs are a 'final
stages' and go into a pass-through mode to the domain that controls
the mixing.

a little bit of latency is acceptable for general purpose audio,
no latency is acceptable for echo cancellation

Matti: linux alsa stack does not support very well the offloading
Matti: recommends the reading of Android Auto CDD to know how to measure latency (not an obvious task)

2.1.7 Standardization of hypervisor APIs (virtio)
-------------------------------------------

- Matti Möll Opensynergy

UEFI Unified Extensible Firmware Interface
Platform standardization BIOS PXE x86 only UEFI Combo of PXE, BIOS and
boot loader, also has a runtime environment Challenges of
virtualization in automotive, they come down to disc and network
because the systems tend to be headless. There is a low amount of
reusable devices. The question for standardization is where do you
cut? COTS or domain specific? Strict requirements or rough consensus?

vendor specific device models: eg Virtual Box drivers

cloud virtualisation/ amazon uses xen modified and nvidia drivers
because most cloud providers use nvidiA

device virualization technologies

virtual gpu - question on status:
distinction between device (server) and guest (client)

Lars: virtio will be brought by security people in the xen community

Kevin / Virtual Open Systems (www.virtualopensystems.com) commented on virtio performance and some teams going to PCI

Q. are the device modes available somewhere ?
A. no but a device model is something simple to develop

automotive use case: DRM

QA OPTEE will have kernel awareness soon
opensynergy, greenhills are using wirtual io

Q. who is using QNX with HV ?
A. no one answers

GHS has jumped into the discussion

Epam & Opensynergy are ok to support the standard
windriver and ghs are not willing to answer
Nikola will try to forward the topic back to GHS management
no commitment from WR though

2.1.8 Security enhancement on Xen ARM
-------------------------------------

Features for secure smartphones
- Service isolation
- Secure boot
- Secure storage
- Access control

How often was a DomU switched away due to interupt handling.
- Profiling data, statistical profiling where that system was at a
given time. Linux users can use perf.
- Information from Dom0 on scheduling is also available
Q: What about IO load? A:

overview of denial of service
discussion on how to access HSM

Ralf Opensynergy
how do we make an IC safe ?
Ralf: we add a monitoring system to monitor the ic, the hv based architecture enables the deployment of counter measures

2.1.9 Health/Debugging/Analysis/Logging (in relation to SHDA project)
--------------------------------------------------------------------

Gunnar: what do you want to measure on an hypervisor
cpu load ?
memory usage ?
io ?

Matti
time spent in interrupt
profiling data: statistics on data, linux is pretty great for that, hypervisor should not block the gathering of statistics

Sang bum
if we instrument the system for debugging, we modified the behavior
it depends on the kind of systems you are debugging
the work done for assessing security might not be relevant for performance profiling

Artem
in xen we have the profiling tools, i.e. special tools for RT to plan budget, deadline
debug is trying to analyze what happens ar run time (disk for instance)
how long a context switch takes in the gpu ? there is some jitter

PVRtune
https://www.imgtec.com/blog/introducing-the-groundbreaking-new-pvrstudio-and-pvrtune-complete/

IMG have PVR tooling, Renesas have tooling. How well does that tooling
stand-up in a HV situation? PVR Tune is a graphical monitoring tool
for Imagination GPUs System health monitoring Q: Does the HV
understand the process modeling in the guest? A: The only feedback
you can get from the kernel if it is in the HV is the ticks scheduler

Windriver
it is not only about gpu, the tools depend on the context
PVRtune is rather for gpu level analysis

There exist Xen tools and there is a rich toolset on the developer side for debugging.

Q: Is it possible to get a demonstration of the tooling? A: It is
kind of use case bound but we'll release some information shortly

No one tool can cover all areas, no magic wand. PS5 is the hyper
debugger from ARM

guest introspection
Matti: the interns of the kernel change every day and you can get very little info from the kernel
hw debugging environment can do it: trace32 lauterbach, arm psi tools
you can do real introspection with such a tool

Ralf: what is at hand for an operating system that could help bringing up an HV system ?
Matti: only lauterbach can help

Homologation -- granting approval by certifying authority
Type certification -- certification of a component

Page tree

Schedule
(Topic name & Responsible & Timeslot)

Minutes from Hypervisor Workshop at AMM Munich April 2018

Archives

Topic Introduction

AMM Workshop Agenda (Topic name & Responsible)

AMM Workshop Agenda Topics (brainstorm and more details)

Page tree

Hypervisor Workshop Schedule at Munich AMM 2018

Schedule (Topic name & Responsible & Timeslot)

Minutes from Hypervisor Workshop at AMM Munich April 2018

Archives

Topic Introduction

AMM Workshop Agenda (Topic name & Responsible)

AMM Workshop Agenda Topics (brainstorm and more details)

Schedule
(Topic name & Responsible & Timeslot)