-
Notifications
You must be signed in to change notification settings - Fork 294
System Device Tree Meeting Notes 2021
Nathalie Chan King Choy edited this page May 12, 2021
·
8 revisions
- Bruce Ashfield
- Christopher Clark
- Daniel P Smith
- Stefano Stabellini
- Nathalie Chan King Choy
- Krzysztof Kepa GE Research
- Loic Pallardy ST
- Mark Dapoz
- Ed Mooring
- Tomas Evensen
- Joakim Bech Linaro
- Maarten Koning
- Grant Likely Arm
- Christopher and Daniel will talk about their Device Tree bindings for HyperLaunch, which has very close similarities to System Device Tree domains.
- We'll also discuss the topic of hierarchical domains
- Bruce will present the latest Lopper updates.
- Stefano: Send slides
- Bruce: Send out the Lopper YAML grammar
- Recording: (I am not sure how long before the recordings expire or I will hit my storage limit, so if you need to catch up by watching the recording, please download it in the next couple weeks) Recording link System DT call
- Slides: Stefano sent out to the mailing list
- Hyperlaunch
- Stefano: Intro
- System DT and Xen Dom0less
- This meeting will focus more on execution domains
- System DT domains were inspired on older spec for Xen Dom0less, which was DT-based spec for Xen for describing Xen VMs to start at boot time. Similar concepts & very much aligned.
- In next few months, would like to work more on hypervisor side
- Daniel & Christopher worked on Hyperlaunch (Xen Dom0less spec)
- Christopher, Daniel
- Started with several years of OpenXT work
- Building for both X86 & Arm systems
- Dom0less does some dynamic generation of DT for Arm guests for VM config
- Flow we've seen in System DT doc w/ Lopper suggests static system config
- Hyperlaunch looking at enabling bootloader to do some dynamic generation of the DT that gets passed to Xen to describe VMs, so can do more at runtime.
- Stefano: To describe domains on multiple execution levels, incl. trusted firmware domain on Arm
- Christopher: Moving away from coarse grain privileged Dom0 & non-privileged DomU. Will send finer grained description, based on DT: e.g. domain to
- ___.
- Deprivileged guest VMs
- Isolation VMs to protect other VMs on the system from adversarial activity from that network device.
- Stub domains: Device emulation in isolated environment.
- Christopher: One use case in mind: Virtual TPM enables isolation between VMs that are holding secrets on behalf of guest. No all-powerful Dom0 that has access to HW & all the confidential info. Requires sequenced launch to provide that env.
- Showed example:
- Older version, bit more x86 specific, but newer version will come out that is more generic
- Hypervisor node would go under chosen
- Would allow us to crop this node if we wanted to remove this info from DT
- Defined some containers
- Hyperlaunch would like to be able to use SystemDT if possible
- Stefano: Think we could align them well enough so there would be just 1 spec
- Stefano: Intro
- Hierarchical domains
- Higher privilege domain on top, lower privilege below
- Xen is an obvious example of more privileged domain (hypervisor at EL2) with children (EL1)
- Beyond Xen, other examples:
- Xilinx firmware has concept of subsystems
- Collection of HW resources that power up together & have same lifecycle -> belong to same "domain", which are so close to system DT domains, but instead of hypervisor-defined they are firmware-defined
- Xilinx subsystem can include cortex-R or cortex-A in 1 subsystem
- Looks like clean example as top level domain of hierarchy
- If other vendors have similar firmware-defined domains, System DT can work
- Subsystem: ID is really necessary. Want to add to other domains.
- ATF is running at higher privilege than Linux, so makes sense as parent of Linux
- Tomas: By expressing it this way, you inherit what's in the inner loop, so you don't have to repeat b/c ATF has access to what's in the children. Helps you see shared devices & memory.
- Daniel: Interesting w/ nested virtualization on Arm platform
- Stefano: Challenge we haven't solved yet for virtualization: How to distinguish physical devices from virtual devices. Will take some iteration.
- Resource group: shared between multiple domains (e.g. SRAM)
- What about memory that is not main memory?
- There are a number of memory types on SoC that are not DDR
- e.g. MMIO SRAM: how to specify these notes in System DT under domain?
- Adding in access list like other device: But it's memory, so may want to carve up into subsets - need address & range.
- What if we consider it as normal memory? "Memory" property is really meant for main memory
- Took inspiration from openamp remoteproc binding: Uses property called "sram"
- Joakim: Like tree idea. But subnodes on exception level?
- Stefano:
- Higher privileged domain is parent & lower is child. Typically tied to EL. ATF is secure EL3, so is parent & Linux A72 is normal level EL1.
- In scenario with no EL, top level specify as secure EL3, but it's not really informative in this case
- Stefano:
- Joakim: If you have list of different memory regions, it can be helpful to see who has been taking which memory are. Hard to see in Linux what memory you're not allowed to touch.
- Stefano: One reason why can be helpful. Can describe what's present but not meant for the OS.
- Tomas: Lopper will warn if you use the same memory in different domains. But, if it's child using the memory, it's OK unless you say otherwise. Static config w/ Lopper can give you these checks.
- Stefano: Memory & access works bit differently with hierarchy: You keep carving out for 1 domain or another - top of set at top & then further down is smaller set. Device access list: Uniquely assigned.
- Mark: Wouldn't you want to do that we domains & device access list? Can get checks.
- Stefano: Yes, we thought so too. But it became cumbersome & didn't give a lot of benefit & made it harder to read.
- Stefano: Virtual CPU is still an open question
- Bruce: Lopper update
- WIP, adding more utility & functionality
- Can take
- DTS, DTB
- YAML representation of the domains: Easier for tools & humans to edits. Have defaults & assumed values.
- More stages in the pipeline, which allows us to do different things on the tree
- Working w/ Zephyr tam to bring in their dtlib/edtlib processing for pure Python, so won't need dtc, cpp
- Making Lopper on pypi to make it easier to install once the Zephyr work is done
- Will have a reference pipeline in Lopper repo to show
- Example slide: Custom back-end. In lopper repo now.
- Mark: Do you have a spec for the YAML grammar?
- Stefano: Yes. Knew we'll run out of time
- Grant: Are you suing same YAML schema as rob?
- Stefano:
- Looking at the schema & starting to write the spec to convert to more & more schema
- Lopper YAML is not meant to describe the spec: More a 1:1 correspondence to DT
- Grant: There is a YAML representation of DT that's being used for the schema checker. Can the 2 representations be unified.
- Stefano:
- Next time:
- Stefano & Bruce present YAML
- Tomas - Xilinx
- Stefano - Xilinx
- Frank Rowand - Sony
- Dan Milea - Wind River
- Dan Driscoll - MGC
- Krzysztof Kepa - GE Research
- Bruce Ashfield - Xilinx
- Kumar G - Linaro
- Loic Pallardy - ST
- Ben Levinsky - Xilinx
- Hannes Tschofenig - Arm
- Rob Herring - Arm
- Mark Dapoz - Wind River
- Ed Mooring - Xilinx
- Arnaud Pouliquen - ST
- I would also like to cover again the remoteproc bindings and they get generated from system device tree.
- I would like to continue the discussion on the bus firewall bindings. I sent an email to the list with the summary of the past discussions.
- If we have time, we can discuss how system device tree aligns with ffa partitions definitions.
- Loic: Check HW lock bindings
- Stefano: Will try to combine the 2 properties into 1 & come up with some examples.
- Recording: (I am not sure how long before the recordings expire or I will hit my storage limit, so if you need to catch up by watching the recording, please download it in the next couple weeks) Recording link System DT call
- Slides: Stefano sent out to the mailing list
- Announcement: Linaro Connect session accepted: DTE state of the union from 4 perspectives: Bill Mills, Kumar, Stefano, Wind River
- Stefano: Remoteproc bindings have made good progress in the past year
- Showed original Xilinx remoteproc binding & Ben recently posted to upstream revised Xilinx binding based on feedback (thanks to the reviewers)
- Showed TI K3 binding & is much more aligned. Stefano has slide listing the commonalities.
- Kumar: What is Linux doing with sram property? Can be helpful to specify the property outside of Linux.
- Ben: Application specific, or IPC
- Rob: R5 is common from Arm & have 2 different bindings for it between Xilinx & TI
- Loic: Agree we can find common foundation between the drivers for the co-processors
- Rob: There are common parts. Focused on R5
- Kumar:
- There's value in defining a common binding.
- Remote proc state machine could be common too
- Rob: There is a start on work on that. Have acked.
- Showed System DT representation for Xilinx bindings. Interesting part is in domains node.
- Access: Give unique access to something
- Include: Include shared resources
- Pass to Lopper as input. Ben created Xilinx-specific plug-in in Lopper repo, assists.
- Stefano: Bus firewall bindings
- Another way to call it is is system-wide MPU
- Doesn't fit interconnect bindings well
- But maybe very similar to IOMMU bindings
- What's missing: How to specify the slave side connection (e.g. firewall property)
- Currently assume IOMMU covers everything in address space
- Rob: Another issue: What if you have an IOMMU & a firewall? Probably makes sense for these both to be separate.
- Rob: Why can't bus master ID & firewall be same property?
- Stefano: Could, if we had a way to distinguish them
- Rob: That node defines what each one is.
- Stefano: Could define positioning
- Loic: Driver in Linux must be able to check what it's allowed to access IP register before probing.
- Loic: Need to investigate if can do at kernel level, if we can't add this new property.
- Loic: If have several drives, have to know know which reg value
- Stefano: Makes sense to have slave side info there, to know what's protected. Shouldn't have to encode in drivers or Lopper - can. Xilinx lopper plug in to pre-generate configuration of bus firewall @ low-level details. Could be hard-coded in Lopper plug-in to cover slave side, but makes System-DT description incomplete.
- Kumar: Makes sense to describe b/c non-Linux SW cases. TEE or boot firmware might be configuring the firewall, and needs the info.
- Kumar: What is firewall property in CAN case?
- Stefano: MMIO region starting at 0xff06000 is protected by system-wide MPU. So, if you want to block everyone except 1 master to make DMA txn for that region, you have to go to lpd_xppu. Unlike w/ IOMMU, not all areas are protected.
- Loic: Need additional ID for ST b/c not based on memory region, but rather peripheral identifier. Have hard-coded entries in peripheral firewall, similar to clock & reset. -> firewall-cells after phandle in firewall property. Other SOC vendors have HW semaphores to get access to peripherals, so SW that wants to access peripherals should 1st do something at firewall level to get access & then do something.
- Rob: HW lock binding exists for that, but don't have to use here.
- Loic: Will check HW lock binding. Think it's complement. Firewall adds check of access right + lock.
- Kumar: 2 ways in DT to describe directionality. 1 way: memory & DMA ranges for in & out view that's CPU-centric. Interrupts: Interrupt nexuses w/ graph-based view, where we don't separate direction based on property name. Should we move more to interrupt way of describing things?
- Rob: Should be able to combine into 1 property & say there's always at least 1 cell & that cell says master or slave & subsequent cells are ID values.
- Stefano: that would make it easier
- Rob: Not sure if you need master & slave ID. Could walk the list to find firewall properties & firewall property entries. Based on cell value could know if slave or master on that system.
- Stefano: That might work too
- Rob: Assuming address range might break if >1 reg entry, generically
- Stefano: Different slave ports on different reg entries?
- Stefano: Assumption: Each node, even if it has multiple regs, are all slave the same way & master the same way. No different master & slave config for each reg entry of 1 node. Think it holds true on our boards.
- Rob: Cell values could be address, even if duplicated. May be a subset of reg entries.
- Stefano: Good idea. Could put it as a 1st ID after lpd_xppu
- Rob: Generally cells only completely meaningful to phandle they are associated w/. If common, would be convenient.
- Stefano: Will try to combine the 2 properties into 1 & come up with some examples.