3. Execution Domains
An execution domain is a node which describes a software or firmware image running on a CPU cluster, along with a collection of configuration values that enable an operating system or an application to run on CPU cores in the cluster.
For example, execution domains can be defined for:
software or firmware images running at different execution levels on an Arm v8-A architecture CPU
firmware images running in the secure and non-secure CPU states on an Arm v8-M architecture CPU with TrustZone
other trusted and untrusted environments
software or firmware running on individual CPUs that are not part of SMP clusters
a kernel or RTOS application running on a subset of CPUs within an SMP cluster
Each CPU cluster node can have one or more associated execution domains. Multiple execution domains associated with the same cluster can result, for example:
from virtualization or non-lockstep execution on CPU clusters that support it
from a multi-stage boot on a microcontroller, where an execution domain for a bootloader permanently yields control of a CPU to an execution domain running a later stage firmware image
when partitioning the CPUs in an SMP cluster into subsets, each of which has its own distinct software or firwmare
3.1. Example
Execution domains are expressed by a new binding for the
“openamp,domain-v1” value for the compatible property. Domains are
placed under a new top-level node within a system devicetree:
/domains
. Here is an example:
domains {
openamp_r5 {
compatible = "openamp,domain-v1";
cpus = <&cpus_r5 0x2 0x80000000>;
#memory-flags-cells = <0>;
memory = <0x0 0x0 0x0 0x8000000>;
#access-flags-cells = <1>;
access = <&can0 0x3 ðernet0 0x7>;
id = <0x1>;
};
};
As shown above, openamp_r5
contains information about:
the physical CPUs the software is running on, via cpus
memories assigned to the domain, via memory
devices that should only be accessible by the domain, via access
Domains can be nested recursively within other nodes under /domains
.
3.2. Execution Domain Binding, v1
Property Name |
Usage |
Value Type |
Definition |
---|---|---|---|
|
R |
<string list> |
Value shall include “openamp,domain-v1”. See [DTSpec] §2.3.1. |
|
SD |
|
See Section 3.2.1. Specifies the CPU cluster on which the domain software runs. |
|
O |
|
Specifies the number of |
|
SD |
|
See Section 3.2.2. Specifies devices configured to only be accessible by this domain (the node in which the access property appears). |
|
O |
|
Specifies the number of |
|
SD |
|
See Section 3.2.3. Specifies the memory assigned to the domain. |
|
O |
|
Specifies the number of |
|
SD |
|
See Section 3.2.4. Specifies the MMIO SRAM assigned to the domain. |
|
R |
|
A 32-bit integer that uniquely identifies the domain |
|
SD |
|
See Section 3.2.5 |
|
SD |
|
See Section 3.2.6 |
|
SD |
array |
See Section 3.2.6 |
|
SD |
|
See Section 3.2.6 |
|
SD |
array |
See Section 3.2.6 |
|
SD |
|
See Section 3.2.6 |
|
SD |
array |
See Section 3.2.6 |
Usage legend: R=Required, O=Optional, OR=Optional but Recommended, SD=See Definition |
Note
The system devicetree bindings which define execution domains are separate from the bindings used for hardware description (see Section 2) for two main reasons:
A different persona will add and edit the information
configuration should be separated from hardware description, since it has a different rate of change
3.2.1. cpus Property
Property |
|
---|---|
Value type |
|
Description |
Required; defines the physical CPUs this domain (the domain in which the cpus property appears) runs on. |
Example |
|
Within the triplet:
cpu-cluster is a phandle to a CPU cluster node
cpu-mask is a bitfield indicating the subset of CPUs in the cluster which the domain runs on
execution-level is a cluster-specific execution level for the domain
The execution level is the most privileged level that the domain can make use of. The permissible values for the execution-level cell in a cpus property depend on the CPU cluster hardware. The following permissible values are provided for some CPU architectures. To add other CPU architectures, this specification should be amended.
For Arm Cortex-R5 CPUs, execution-level is a bit map where:
bit 31: secure (1) / non-secure (0)
bit 30: lockstep (1) / split (0)
bits 1 through 29: reserved, must be zero
For Arm Cortex-A53 and -A72 CPUs, execution-level is a bit map where:
bit 31: secure (1) / non-secure (0)
bits 2 through 30: reserved, must be zero
bits 0-1: EL0 (0x0), EL1 (0x1), or EL2 (0x2)
3.2.2. access Property
Property |
|
---|---|
Value type |
Optional |
Description |
A list of devices the domain shall have exclusive access to, using bus firewalls or other similar technologies. |
Example |
|
Within each pair:
device is a phandle to the device node
flags contains domain-specific flags. The number of cells in each flag is defined by the #access-flags-cells property of this domain (the domain in which the access property appears).
3.2.3. memory Property
Property |
|
---|---|
Value type |
Optional |
Description |
An array of memory ranges assigned to the execution domain (the node in which the memory property appears). This must be a subset of the physical memory present in the system. |
Example |
|
Within each triplet:
start is the physical address of the start of the memory range. The number of cells used to represent the start address is determined by the #address-cells property.
size is the size of the memory range, in bytes. The number of cells used to represent the size is determined by the #size-cells property.
flags contains domain-specific flags. The number of cells in each flag is defined by the #memory-flags-cells property of the execution domain.
Note that the memory property can also be used to express memory sharing between domains. For example:
domains {
openamp_r5 {
compatible = "openamp,domain-v1";
memory = <0x0 0x0 0x0 0x8000000 0x8 0x0 0x0 0x10000 0x0>;
id = <0x2>;
};
openamp_a72 {
compatible = "openamp,domain-v1";
memory = <0x0 0x8000000 0x0 0x80000000 0x8 0x0 0x0 0x10000 0x0>;
id = <0x3>;
};
};
In this example, a 16 pages range starting at 0x800000000 is shared between two domains.
3.2.4. sram Property
Property |
|
---|---|
Value type |
Optional |
Description |
An array of sram ranges assigned to the execution domain (the node in which the sram property appears). This must be a subset of the physical SRAM memory present in the system. |
Example |
|
Within each triplet:
start is the physical address of the start of the memory range. The number of cells used to represent the start address is determined by the #address-cells property.
size is the size of the memory range, in bytes. The number of cells used to represent the size is determined by the #size-cells property.
flags contains domain-specific flags. The number of cells in each flag is defined by the #sram-flags-cells property of the execution domain.
3.2.5. os,type Property
Execution domains can have an optional “os,type” property, which describes one or more operating systems that may run on the domain.
The field may be used by automated tooling for activities such as verifying that the domain is capable of running the operating system, configuring a build system to produce the proper operating system, configuring a storage mechanism to include the specified operating system, or other purposes.
The value of os,type is a string defined in the format:
OS_TYPE[,TYPE_ID[,TYPE_ID_VERSION]]
OS_TYPE
is mandatory. It defines the operating system’s type. Its
value must match one of the following:
OS_TYPE:
baremetal
linux
freertos
zephyr
custom
x-<vendor>[-os]
This specification should be updated if additional types are required.
baremetal
refers to a direct application that executes on the system with no conventional operating system. Examples of this may include a first stage boot loader, a second stage boot loader, U-Boot [U-Boot], Trusted Firmware-A [TF-A], etc.linux
refers to a Linux based operating system. Examples of this may include Yocto Project [Yocto] derived distributions, Red Hat Enterprise Linux [RHEL], Ubuntu [Ubuntu] distributions, etc.freertos
refers to the FreeRTOS [FreeRTOS] real-time operating systemzephyr
refers to the Zephyr [Zephyr] real-time operating systemcustom
refers to a user specific operating system. Custom must only be used by the group providing the operating system implementation. Each usage ofcustom
will be different.*x-<vendor>[-os]
refers to an extension of a non-registered vendor specific operating system. The ‘x’ refers to extension, which is attempts to avoid namespace collisions by convention. The mandatory<vendor>
component identifies the operating system vendor, for examplex-xilinx
. However, the vendor name may not be a specific enough namespace to avoid collision, so an optional-os
is allowed as well. The<vendor>
controls the namespace of-os
values, if they are used. For instance, Wind River VxWorks could be specified usingx-windriver-vxworks
.It is recommended that a vendor register their operating system in the official named list, only using this extension format until it is official.
TYPE_ID
is specific to each OS_TYPE
, but is not currently
formalized. The purpose of this is to further clarify details on the
OS_TYPE
if desired. For instance, to specify Ubuntu Linux, use:
“linux,ubuntu”.
As TYPE_ID
is not yet formalized, it is open for different usages by
different parties. It is recommended that groups work together to define
common values where appropriate.
TYPE_ID_VERSION
is optional parameter which may appear after a
TYPE_ID
value. Its purpose is to specify the version of the
operating system identified by TYPE_ID
. Extending the prior example
of “linux,ubuntu”, version 18.04 of that operating system may be
specified using “linux,ubuntu,18.04”.
As with TYPE_ID
, this may be open to namespace collisions, and it is
again recommended that groups work together to define common values
where appropriate.
Here are some example os,type values:
os,type = "linux"
os,type = "linux,ubuntu,18.04"
os,type = "linux,ubuntu,18.04.01"
os,type = "linux,yocto"
os,type = "linux,yocto,gatesgarth"
os,type = "baremetal"
os,type = "baremetal,fsbl"
os,type = "baremetal,newlib,3.3.0"
3.2.6. Implicit Flags Properties
It is possible to specify default flags values at the domain level using the following properties:
#access-implicit-default-cells
access-implicit-default
#memory-implicit-default-cells
memory-implicit-default
#sram-implicit-default-cells
sram-implicit-default
Each property specifies the default value for the access, memory and sram flags for the execution domain (the node in which the implicit flags properties appear).
The number of cells to use in each case is provided by the #access-implicit-default-cells, #memory-implicit-default-cells, and #sram-implicit-default-cells properties.
Here is an example:
#access-implicit-default-cells = <1>;
access-implicit-default = <0xff00ff>;
#access-flags-cells = <0x0>;
access = <&mmc0>;
3.3. Default Execution Domain
There is a concept of a default execution domain in system devicetree.
This corresponds to an execution domain running on the default CPU
cluster, /cpus
(see Section 2.4). This default
domain is compatible with the current base specification.
Here are some use cases for this domain:
As a way to specify the default place to assign added hardware (see usage environment #1 in Section 1.4)
The default domain does not have to list the all the hardware resources allocated to it. It gets everything not explicitly allocated to other domains.
This minimizes the amount of information needed in
/domains
.This can also be useful for managing dynamic hardware, such as add-on boards and FPGA images that add new devices.
The default domain can be used to specify what a master environment sees (see usage environment #2)
For example, the default domain can be the entity configuring a master environment like Linux or Xen, while the other domains are to be managed by the master.
In a system device tree without a default CPU cluster, the memory assignment for each domain is specified using the memory property in each “openamp,domain-v1” node. In a devicetree with a default domain and software running on it that is not aware of the system devicetree’s semantics, it may be convenient to “hide” the memory assignments for non-default execution domains from that software.
This is possible using /reserved-memory
. Here is an example:
reserved-memory {
#address-cells = <0x2>;
#size-cells = <0x2>;
ranges;
memory_r5@0 {
compatible = "openamp,domain-memory-v1";
reg = <0x0 0x0 0x0 0x8000000>;
};
};
The purpose of memory_r5@0
is to let the default execution domain
know that it shouldn’t use the 0x0-0x8000000 memory range, because it is
reserved for use by other domains.
3.4. Per-Domain Reserved Memory and Chosen Nodes
/reserved-memory
and /chosen
are top-level nodes defined in the
base specification which are dedicated to configuration of the default
execution domain, rather than hardware description of that domain.
Each execution domain in a system devicetree might need similar
configuration. To enable this, domain nodes may have chosen
and
reserved-memory
child nodes with the same semantics, but which apply
to this domain. The top-level /reserved-memory
and /chosen
nodes
remain in place for the default execution domain.
Here is an example:
/ {
/* chosen settings for /cpus */
chosen {
};
/* reserved memory for /cpus */
reserved-memory {
};
domains {
openamp_r5 {
compatible = "openamp,domain-v1";
/* chosen for "openamp_r5" */
chosen {
};
/* reserved memory for "openamp_r5" */
reserved-memory {
};
};
};
};