Booting a twin CPU system

Updated 2,105 Days AgoPublic

Actions

System Architecture

There are many possibly ways to design a twin CPU system. Some important considerations from a boot perspective are:

• Which CPU has access to which areas of Flash?
  • It is possible that the secure CPU has no access to the Flash from which the non-secure CPU will boot, in which case the non-secure CPU will presumably have a separate root of trust and perform its own integrity checks on boot.
• How does the non-secure CPU behave on power-up? Is it held in reset, does it jump to a set address, …?
• What are the performance characteristics of the two CPUs?
  • There could be a great disparity in performance

TF-M Twin CPU Booting

In an effort to make the problem manageable, as well as to provide a system with good performance, that is flexible enough to work for a variety of twin CPU systems, the following design decisions have been made:

• TF-M will (for now) only support systems where the secure CPU has full access to the Flash that the non-secure CPU will boot from
  • This keeps the boot flow as close as possible to the single CPU design, with the secure CPU responsible for maintaining the chain of trust for the entire system, and for upgrade of the entire system
• The secure code will make a platform-specific call immediately after setting up hardware protection to (potentially) start the non-secure CPU running
  • This is the earliest point at which it is safe to allow the non-secure code to start running, so starting it here ensures system integrity while also giving the non-secure code the maximum amount of time to perform its initialization
  • Note that this is after the bootloader has validated the non-secure image, which is the other key part to maintain security
  • This also means that only tfm_s and tfm_ns have to change, and not mcuboot
• Both the secure and non-secure code will make platform-specific calls to establish a synchronization point. This will be after both sides have done any initialization that is required, including setting up inter-CPU communications. On a single CPU system, this would be the point at which the secure code jumps to the non-secure code, and at the very start of the non-secure code.
• After completing initialization on the secure CPU (at the point where on a single CPU system, it would jump to the non-secure code), the main thread on the secure CPU will be allowed to die
  • The scheduler has been started at this point, and an idle thread exists. Any additional work that is only required in the twin CPU case will be interrupt-driven.
• Because both CPUs may be booting in parallel, executing different initialization code, at different speeds, the design must be resilient if either CPU attempts to communicate with the other before the latter is ready. For example, the client (non-secure) side of the IPC mechanism must be able to handle the situation where it has to wait for the server (secure) side to finish setting up the IPC mechanism.
  • This relates to the synchronization calls mentioned above. It means that those calls cannot utilise the IPC mechanism, but must instead use some platform-specific mechanism to establish this synchronization. This could be as simple as setting aside a small area of shared memory and having both sides set a “ready” flag, but may well also involve the use of interrupts.
  • This also means that the synchronization call must take place after the IPC mechanism has been set up but before any attempt (by either side) to use it.

API Additions

Three new HAL functions are required:

void tfm_spm_hal_boot_ns_cpu(uintptr_t start_addr);

• Called on the secure CPU from tfm_core_init() after hardware protections have been configured.
• Performs the necessary actions to start the non-secure CPU running the code at the specified address.

void tfm_spm_hal_wait_for_ns_cpu_ready(void);

• Called on the secure CPU from the end of tfm_core_init() where on a single CPU system the secure code calls into the non-secure code.
• Flags that the secure CPU has completed its initialization, including setting up the IPC mechanism.
• Waits, if necessary, for the non-secure CPU to flag that it has completed its initialisation

void tfm_spm_hal_wait_for_s_cpu_ready(void);

• Called on the non-secure CPU from main() after the twin-cpu-specific initialization (on a single CPU system, this would be the start of the non-secure code), before the first use of the IPC mechanism.
• Flags that the non-secure side has completed its initialization.
• Waits, if necessary, for the secure CPU to flag that it has completed its initialization.

For all three, an empty implementation will be provided with a weak symbol so that platforms only have to provide the new functions if they are required.