Phriction Trusted Firmware Trusted Firmware M Design Twin-cpu Mailbox Design for TF-M on Dual Core System History Version 1 vs 15
Version 1 vs 15
Version 1 vs 15
Edits
Edits
- Edit by davidhuziji, Version 15
- Jul 24 2019 10:37 AM
- Edit by davidhuziji, Version 1
- Apr 19 2019 2:45 AM
| Edit Older Version 1... | Edit Current Version 15... |
Content Changes
Content Changes
# 1. Introductions
This document proposes a generic design of the mailbox communication for Trusted Firmware-M (TF-M) on a dual core system. The mailbox functionalities transfer PSA client call request and return results between Non-secure Processing Environment (NSPE) and Secure Processing Environment (SPE)).
The dual core system should satisfy the following requirements
- NSPE and SPE are properly isolated and protected following PSA.
- An Arm M-profile core locates in SPE and acts as the secure core.
- TF-M runs on the Secure core with platform specific drivers support.
- An Inter-Processor Communication hardware module in system for communication between secure core and non-secure core. Mailbox communication is based on the Inter-Processor Communication.
- Non-secure memory shared by NSPE and SPE.
## 1.1. Scope
This design document focuses on the mailbox functionalities in NSPE and SPE on a dual core system. The mailbox functionalities include the initialization of mailbox in NSPE and SPE, and the transfer of PSA client call request and reply between NSPE and SPE. Data types and mailbox APIs are defined in this document.
Some details of interactions between mailbox and other modules are specified in other design documents. Those details are skipped in this document. [Communication Prototype Between NSPE And SPE In Dual Core System](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/communication_prototype_between_nspe_and_spe_in_dual_core_system/) defines a general communication prototype between NSPE and SPE. It describes how TF-M interacts with mailbox functionalities and the general requirements on mailbox. [Booting a twin CPU system](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/bootloader/) describes a synchronization step of mailbox between NSPE and SPE during system booting.
## 1.2. Organization of the document
- Section 2 provides an overview on the mailbox architecture.
- Section 3 discusses about the mailbox communication for PSA client call.
- Section 4 introduces the initialization of mailbox.
- Section 5 lists mailbox data types and APIs.
# 2. Mailbox architecture
The mailbox consists of two parts sitting in NSPE and SPE respectively. NSPE mailbox library provides mailbox functionalities in NSPE and SPE mailbox library provides mailbox functionalities in TF-M in SPE.
TF-M provides a reference implementation of NSPE mailbox library as a part of TF-M NSPE interface library. Vendor platform can also implement a specific NSPE mailbox library following this design document.
The NSPE mailbox library delivers the PSA client call requests to SPE mailbox library. After the PSA client call result is replied from SPE, NSPE mailbox library fetches the result and returns it to non-secure tasks.
NSPE mailbox objects are managed by NSPE mailbox library in non-secure memory to hold PSA client call parameters, return result and other mailbox data.
NSPE mailbox library relies on platform specific Inter-Process Communication to process mailbox notifications between NSPE and SPE.
The SPE mailbox library in TF-M receives PSA client call requests from NSPE mailbox library. It parses the requests and delivers them to TF-M core/SPM. After the PSA client call is completed, TF-M core/SPM invokes SPE mailbox library to return results to NSPE.
SPE mailbox objects are managed by SPE mailbox library in secure memory to hold the data copied from non-secure memory.
SPE mailbox library relies on platform specific Inter-Process Communication to process mailbox notifications between SPE and NSPE.
The architecture is showed in following figure.
{F33348}
# 3. Mailbox communication for PSA client call
This section describes the design of PSA client call request and reply transfer between NSPE and SPE via mailbox.
## 3.1. Mailbox objects
This section lists the mailbox objects required in NSPE and SPE.
### 3.1.1. NSPE mailbox objects
NSPE mailbox objects are managed by NSPE mailbox library. But NSPE mailbox objects can be accessed by both NSPE mailbox library and SPE mailbox library.
#### 3.1.1.1. Mailbox queue
NSPE mailbox library maintains a mailbox queue in non-secure memory. Please refer to the structure definition in section 5.1.5.
The mailbox queue contains one or more slots. The number of slots should be aligned with that in SPE mailbox queue.
Each slot in mailbox queue consists of a pair of a mailbox message structure and a mailbox reply structure. Each slot might contain additional fields, such as identification of non-secure task which initiates the PSA client call request. Each slot serves a PSA client call from non-secure task.
The parameters of PSA client call are hosted in the mailbox message inside the slot. Section 3.1.1.2 describes the details of mailbox message.
The mailbox reply is used to receive the PSA client call return result from SPE. Section 3.1.1.3 describes the details of mailbox reply.
#### 3.1.1.2. Mailbox messages
A mailbox message contains the parameters of a PSA client call request from a non-secure task. Please refer to the structure definition in section 5.1.2.
When a PSA client call comes from TF-M NSPE interface library, NSPE mailbox library selects an empty mailbox queue slot for the PSA client call. The parameters of that PSA client call is organized into the mailbox message inside the selected slot. SPE mailbox library copies the mailbox message from non-secure memory to secure memory for PSA client call handling in TF-M.
A mailbox message can contain the following fields
- PSA client call type
- Parameters required in PSA Client call. The parameters can include the following, according to PSA client call type
- Service ID (SID)
- Handle to established connection
- Input vectors and the lengths
- Output vectors and the lengths
- Requested version of secure service
- The non-secure memory address to be written with PSA client call return result.
- NSPE Client ID. Optional. The NSPE Client ID is required when NSPE RTOS enforces non-secure task isolation.
More members can be defined in mailbox message to transfer additional information from NSPE to SPE for PSA client call processing.
#### 3.1.1.3. Mailbox reply structures
A mailbox reply structure in non-secure memory receives the PSA client call return result replied from SPE mailbox library. Please refer to the structure definition in section 5.1.3.
The memory address of a mailbox reply structure in non-secure memory is set in the mailbox message in the same slot. After TF-M completes a PSA client call, SPE mailbox library writes the return result into the mailbox reply structure, according to the address set in the mailbox message dedicated to the PSA client call.
### 3.1.2. SPE mailbox objects
SPE mailbox objects are managed by SPE mailbox library. SPE mailbox objects are protected from NSPE accesses by system specific isolation.
#### 3.1.2.1. Mailbox queue
SPE mailbox library can maintain a mailbox queue to store mailbox message copied from non-secure memory. Please refer to the structure definition in section 5.1.7.
The mailbox queue contains one or more slots. The number of slots should be aligned with that in NSPE mailbox queue. After SPE is notified that a PSA client call request is pending, SPE mailbox library assigns an empty slot and copies PSA client call parameters from non-secure memory into this slot.
Each slot in mailbox queue can contain the following fields
- A filed to hold mailbox message content from non-secure memory.
More members can be defined in the slot structure to support mailbox processing in SPE.
## 3.2. Overall workflow
The overall workflow of transferring PSA client call request and return value between NSPE and SPE via mailbox is showed as below.
1. Non-secure task initiates a PSA client cal by calling the dedicated PSA client call API provided by TF-M NSPE interface library.
2. On a dual core system, TF-M NSPE interface library invokes NSPE mailbox library. The routine can vary in diverse NSPE OS or use case implementations. The details of this routine are out of scope.
3. NSPE mailbox library assigns an empty slot from NSPE mailbox queue for that PSA client call. If the NSPE mailbox queue is full, NSPE mailbox will return error code `MAILBOX_QUEUE_FULL` immediately.
4. If step 3 succeeds, NSPE mailbox library prepares the parameters of PSA client call in the dedicated mailbox message inside the assigned slot.
5. After the mailbox message is ready, NSPE mailbox library invokes platform specific Inter-Processor Communication driver to notify SPE. The notification mechanism of Inter-Processor Communication is platform specific.
6. After the notification is completed, non-secure task waits for the reply from SPE. The mechanism of waiting and waking may vary in different NSPE OS and on diverse platforms. Please check section 3.4 for more details.
7. Platform specific Inter-Processor Communication interrupt for mailbox is asserted in SPE. Platform specific Inter-Processor Communication interrupt handler should activate SPE mailbox library to process the request(s).
8. During mailbox processing in TF-M,
1. SPE mailbox library checks and validates NSPE mailbox queue status.
2. SPE mailbox library copies mailbox message(s) from non-secure memory, into SPE mailbox queue.
If mailbox supports multiple ongoing PSA client call requests, it is recommended to copy multiple mailbox message(s) together to save time consumption.
3. SPE mailbox library parses a mailbox message in SPE mailbox queue. Necessary checks are required.
4. SPE mailbox library invokes the APIs provided by TF-M SPM to deliver the PSA client call request to TF-M SPM.
5. The PSA client call is handled in TF-M SPM and target Secure Partition if necessary.
If there are multiple ongoing mailbox messages pending in the SPE message request queue, SPE mailbox library can process mailbox message one by one.
9. After the PSA client call is completed, TF-M SPM invokes SPE mailbox library to reply PSA client call return result to NSPE.
10. SPE mailbox library writes the PSA client call return result to the dedicated mailbox reply structure in non-secure memory.
11. SPE mailbox library notifies NSPE that a PSA client call completes. SPE mailbox library invokes Inter-Processor Communication driver to send notification to NSPE. The notification mechanism of Inter-Processor Communication is platform specific.
12. NSPE mailbox library is activated to handle the PSA client call return result in the mailbox reply structure. NSPE mailbox library cleans up the slot in NSPE mailbox queue after the return result is extracted out.
13. NSPE mailbox library returns the result to TF-M NSPE interface library. The related mailbox objects should be invalidated or cleaned by NSPE mailbox library.
14. TF-M NSPE interface library eventually returns the result to the non-secure task.
The following sections from 3.3 to 3.6 discuss about more details in key steps in above sequence.
## 3.3. Mailbox notifications between NSPE and SPE
As shown in the overall workflow in section 3.2, NSPE mailbox library asserts mailbox notification to trigger SPE to fetch PSA client call request. SPE mailbox library asserts mailbox notification to notify NSPE that PSA client call return result is written. The notification implementation is platform specific based on the Inter-Processor Communication.
It is recommended to assign one independent set of Inter-Processor Communication channel to each notification routine respectively, to implement a *full-duplex* notification mechanism between NSPE and SPE.
If both notification routines share the same Inter-Processor Communication channel, proper synchronization should be implemented to prevent conflicts between two notification routines.
The Inter-Processor Communication interrupt handler in TF-M should deal with the incoming notification from NSPE and activate the subsequent mailbox handling in SPE. [Communication Prototype Between NSPE And SPE In Dual Core System](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/communication_prototype_between_nspe_and_spe_in_dual_core_system/) defines the behavior of Inter-Processor Communication interrupt handler.
The handling in NSPE of the notification from SPE is IMPLEMENTATION DEFINED.
## 3.4. Mechanism of waiting for PSA client call reply in NSPE
After the notification of PSA client call request is sent to SPE, the non-secure task in NSPE can be blocked and wait for the PSA client call reply from SPE. After NSPE receives the notification of PSA client call completion from SPE, the waiting non-secure task is woken up to fetch the PSA client call return result.
If only one ongoing PSA client call is allowed in system, the non-secure task waiting for reply can spin in a busy waiting without being scheduled out.
If multiple ongoing PSA client calls are required, NSPE OS can switch another non-secure task in while current task is waiting for PSA client call reply. The sleeping and waking mechanism can be more complex and rely on specific NSPE OS thread management and use case. To keep the entire implementations flexible enough, this document doesn't define or implement a dedicated waiting/waking mechanism. The implementation of waiting/waking mechanism should be accomplished by NSPE OS and TF-M NSPE interface library.
A field `owner` is defined in NSPE mailbox queue slot structure to map a mailbox queue slot/mailbox message to the non-secure task, to support the feature of multiple ongoing PSA client calls.
The following pseudo-code shows an example of waiting and waking mechanism for supporting multiple ongoing PSA client calls. After `non_secure_task()` sends PSA client call, it falls into sleeping. After SPE notifies NSPE that PSA client call return result is replied, `notification_handler()` in NSPE OS gets the identification of waiting non-secure task by calling `mailbox_get_done_msg_owner()`. Then `notification_handler()` wakes `non_secure_task()` to fetch the return result, according to the task identification.
```lang=C
void non_secure_task(...)
{
mailbox_msg_handle_t handle;
...
/* Send PSA client call request to SPE */
handle = mailbox_tx_client_call_req(...);
/* Platform/NSPE OS specific waiting for PSA client call reply */
sleep_wait(...);
/* Platform/NSPE OS specific Woken up */
/* Fetch PSA client call return result */
mailbox_rx_client_call_reply(handle, ...);
...
}
void notification_handler(...)
{
/* Deal with notification from SPE */
...
/* Check if any mailbox is replied in the NSPE mailbox queue */
if (mailbox_queue_has_replied_msg()) {
/* Get the handle of non-secure task whose mailbox message is replied */
task = mailbox_get_replied_msg_owner();
/* Wake up the waiting non-secure task */
wake_task(task);
}
}
```
## 3.5. Critical section protection of NSPE mailbox queue
Proper protection should be implemented to protect the critical accesses to NSPE mailbox queue. The critical sections can include atomic reading and modifying slot status, and other critical operations on NSPE mailbox queue.
NSPE and SPE mailbox library define corresponding critical section APIs. The implementation of those APIs can be platform specific. Please see more details in section 5.2.1 and 5.2.2.
The implementation should protect a critical access to NSPE mailbox queue from corruptions by
- Other non-secure tasks or exception service routines in NSPE OS
- Accesses from the peer core. SPE mailbox library also accesses NSPE mailbox queue to fetch mailbox message. Therefore, it is essential to implement synchronization or protection on NSPE mailbox queue between secure core and non-secure core.
It is recommended to rely on both hardware and software to implement the synchronization and protection.
## 3.6. Mailbox handling in TF-M
[Communication Prototype Between NSPE And SPE In Dual Core System](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/communication_prototype_between_nspe_and_spe_in_dual_core_system/) defines a generic processing of mailbox messages in TF-M.
If multiple ongoing PSA client calls should be supported, additional fields can be added to PSA message structure in TF-M to build up the connection between a mailbox message and a PSA message. TF-M SPM can store the mailbox message handle in a specific filed in PSA message structure to identify the PSA client caller, while creating a PSA message. While replying the PSA client call return result, TF-M SPM can extract the mailbox message handle from PSA message and pass back to mailbox reply function. SPE mailbox library can identify which mailbox message is completed and write the return result to expected NSPE address.
The details related to changes of TF-M will be documented in further design documents.
# 4. Mailbox initialization
This section describes the initialization sequence of mailbox in NSPE and SPE.
It should be guaranteed that NSPE mailbox library should not initiate PSA client call request until SPE mailbox library initialization completes. A synchronization between NSPE and SPE is required to protect the sequence. [Booting a twin CPU system](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/bootloader/) provides more details on the synchronization.
In current design, the base address of NSPE mailbox queue should be pre-defined and shared between NSPE mailbox library and SPE mailbox library.
## 4.1. SPE mailbox initialization
The SPE mailbox queue memory should be allocated before calling SPE `tfm_mailbox_init()`. `tfm_mailbox_init()` initializes the memory and variables.
SPE mailbox dedicated Inter-Processor Communication initialization is also enabled during SPE mailbox initialization.
After SPE mailbox initialization completes, SPE notifies NSPE that SPE mailbox functionalities are ready.
## 4.2. NSPE mailbox initialization
The NSPE mailbox queue memory should be allocated before calling NSPE `mailbox_init()`. NSPE `mailbox_init()` initializes the memory and variables.
NSPE mailbox dedicated Inter-Processor Communication initialization is also enabled during NSPE mailbox initialization.
# 5. Mailbox APIs and data structures
## 5.1. Data types
### 5.1.1. Constants
**NUM_MAILBOX_QUEUE_SLOT**
`NUM_MAILBOX_QUEUE_SLOT` set the number of slots in NSPE and SPE mailbox queues.
Both NSPE and SPE mailbox libraries should refer to the same `NUM_MAILBOX_QUEUE_SLOT` definition.
The following example configures 4 slots in mailbox queues.
```lang=C
#define NUM_MAILBOX_QUEUE_SLOT (4)
```
**MAILBOX_MSG_NULL_HANDLE**
`MAILBOX_MSG_NULL_HANDLE` is a zero-value null handle of a mailbox message.
```lang=C
#define MAILBOX_MSG_NULL_HANDLE ((mailbox_msg_handle_t)0)
```
**MAILBOX_SUCCESS**
`MAILBOX_SUCCESS` is a generic return value to indicate success of mailbox operation.
```lang=C
#define MAILBOX_SUCCESS (0)
```
**MAILBOX_QUEUE_FULL**
`MAILBOX_QUEUE_FULL` is a return value from mailbox function if mailbox queue is full.
```lang=C
#define MAILBOX_QUEUE_FULL (INT32_MIN + 1)
```
**MAILBOX_INVAL_PARAMS**
`MAILBOX_INVAL_PARAMS` is a return value from mailbox function if any parameter is invalid.
```lang=C
#define MAILBOX_INVAL_PARAMS (INT32_MIN + 2)
```
**MAILBOX_NO_PERMS**
`MAILBOX_NO_PERMS` is a return value from mailbox function if the caller doesn't any proper permission to execute the operation.
```lang=C
#define MAILBOX_NO_PERMS (INT32_MIN + 3)
```
### 5.1.2. Mailbox message
The following structures describe a mailbox message and its members.
```lang=C
/* Structure to hold parameters of PSA client call */
typedef struct client_call_params_t {
uint32_t sid;
psa_handle_t handle;
const psa_invec *in_vec;
size_t in_len;
psa_outvec *out_vec;
size_t out_len;
uint32_t minor_version;
} client_call_params_t;
typedef struct mailbox_msg_t {
uint32_t call_type;
client_call_params_t params;
const void *reply_base;
int32_t client_id;
} mailbox_msg_t;
```
### 5.1.3. Mailbox reply structure
This structure describes a mailbox reply structure.
```lang=C
typedef struct mailbox_reply_t {
int32_t return_val;
} mailbox_reply_t;
```
### 5.1.4. Mailbox queue status bitmask
`mailbox_queue_status_t` defines a bitmask for indicating a status of slots in mailbox queues.
```lang=C
typedef uint32_t mailbox_queue_status_t;
```
### 5.1.5. NSPE mailbox queue
The following structures describe the NSPE mailbox queue and its members in non-secure memory.
```lang=C
/* A single slot structure in NSPE mailbox queue */
typedef struct ns_mailbox_slot_t {
mailbox_msg_t msg;
mailbox_reply_t reply;
/* Identification of the owner of this slot */
void *owner;
} ns_mailbox_slot_t;
typedef struct ns_mailbox_queue_t {
mailbox_queue_status_t empty_slots;
mailbox_queue_status_t replied_slots;
/* Base address of mailbox queue in non-secure memory */
ns_mailbox_slot_t queue[NUM_MAILBOX_QUEUE_SLOT];
} ns_mailbox_queue_t;
```
### 5.1.6. Mailbox message handle
This data type is a reference to an active mailbox message in use.
```lang=C
typedef int32_t mailbox_msg_handle_t;
```
### 5.1.7. SPE mailbox queue
The following structures describe the SPE mailbox queue and its structure members in secure memory.
```lang=C
/* A single slot structure in SPE mailbox queue */
typedef struct secure_mailbox_slot_t {
mailbox_msg_t msg;
} secure_mailbox_slot_t;
typedef struct secure_mailbox_queue_t {
mailbox_queue_status_t empty_slots;
/* Base address of mailbox queue in secure memory */
secure_mailbox_slot_t queue[NUM_MAILBOX_QUEUE_SLOT];
} secure_mailbox_queue_t;
```
## 5.2. Mailbox APIs
### 5.2.1. NSPE mailbox APIs
This section describes the NSPE mailbox APIs. SPE should not invoke these NSPE mailbox APIs.
The NSPE mailbox APIs should be flexible enough to meet requirements in diverse NSPE OS and use cases.
#### 5.2.1.1. mailbox_tx_client_call_req()
This function sends the PSA client call request to SPE.
```lang=C
mailbox_msg_handle_t mailbox_tx_client_call_req(uint32_t call_type,
const client_call_params_t *params,
int32_t client_id);
```
**Parameters**
|||
|------------------------------------|--------------------------------------------------------------------------|
|`uint32_t call_type` |Type of PSA client call |
|`const client_call_params_t *params`|Address of PSA client call parameters structure containing the parameters.|
|`int32_t client_id` |ID of non-secure task. |
**Return**
|||
|------------|--------------------------------------------------------|
|`>= 0` |The handle to the mailbox message successfully assigned.|
|`< 0` |Operation failed with an error code. |
**Usage**
`mailbox_tx_client_call_req()` executes the following tasks:
- receives the PSA client call parameters
- prepares the mailbox message
- notifies SPE via Inter-Processor Communication
In `mailbox_tx_client_call_req()`, it calls `mailbox_set_owner()` to set the owner of the mailbox message.
#### 5.2.1.2. mailbox_rx_client_call_reply()
This function fetches the PSA client call return value after it is replied by SPE.
```lang=C
int32_t mailbox_rx_client_call_reply(mailbox_msg_handle_t handle,
int32_t *reply);
```
**Parameters**
|||
|-----------------------------|---------------------------------------------------------------|
|`mailbox_msg_handle_t handle`|The handle to the mailbox message containing the return result.|
|`int32_t *reply` |The address to be written with return result. |
**Return**
|||
|------------------|-----------------------------------------------|
|`MAILBOX_SUCCESS` |Successfully get PSA client call return result.|
|Other return codes|Operation failed with an error code |
**Usage**
A correct `handle` should be passed to `mailbox_rx_client_call_reply()` to determine the target mailbox message which sent the PSA client call.
`mailbox_rx_client_call_reply()` invokes `mailbox_check_owner()` to check if current caller has enough permission to access the targeted mailbox message. If check passes, the return result will be extracted from mailbox reply structure and written to `reply`. Otherwise, an error code will be returned.
Before exiting `mailbox_rx_client_call_reply()`, the mailbox objects related to that completed PSA client call are invalidated or cleaned.
#### 5.2.1.3. mailbox_is_msg_replied()
This function checks if a specific mailbox message has been replied by SPE, which means the PSA client call return value is replied.
```lang=C
bool mailbox_is_msg_replied(mailbox_msg_handle_t handle);
```
**Parameters**
|||
|-----------------------------|-----------------------------------------|
|`mailbox_msg_handle_t handle`|The handle to the target mailbox message.|
**Return**
|||
|------------|----------------------------------------------------|
|`true` |The PSA client call return value is replied. |
|`false` |The PSA client call return value is not replied yet.|
#### 5.2.1.4. mailbox_queue_has_replied_msg()
This function checks if any mailbox message has been replied by SPE but the PSA client call return result is not reaped yet.
```lang=C
bool mailbox_queue_has_replied_msg(void);
```
**Return**
|||
|-------|-------------------------------------------------------------------------------------------|
|`true` |A mailbox message is replied and the return result is waiting for non-secure task to fetch.|
|`false`|No mailbox message is replied yet. |
#### 5.2.1.5. mailbox_set_owner()
This function sets the owner of an active mailbox message contained in a mailbox queue slot.
```lang=C
int32_t mailbox_set_owner(ns_mailbox_slot_t *slot);
```
**Parameters**
|||
|-------------------------|---------------------------------------------------|
|`ns_mailbox_slot_t *slot`|The base address of a mailbox queue slot structure containing the target mailbox message.|
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation failed with an error code. |
**Usage**
`mailbox_set_owner()` should be implemented according to NSPE OS and specific use case.
The owner can be current non-secure task. The identification of the owner should be got by calling NSPE OS APIs.
If only one ongoing PSA client call is allowed in system, `mailbox_set_owner()` can be implemented as always returning `MAILBOX_SUCCESS` to simplify the implementation.
**//`mailbox_set_owner()` should not exported outside NSPE mailbox library.//**
#### 5.2.1.6. mailbox_check_owner()
This function check if the caller has enough permission to access an active mailbox message contained in a mailbox queue slot.
```lang=C
int32_t mailbox_check_owner(const ns_mailbox_slot_t *slot);
```
**Parameters**
|||
|-------------------------|---------------------------------------------------|
|`ns_mailbox_slot_t *slot`|The base address of a mailbox queue slot structure containing the target mailbox message.|
**Return**
|||
|------------------|----------------------------------------------------|
|`MAILBOX_SUCCESS` |The caller is allowed to access the mailbox message.|
|Other return codes|The access is disallowed with an error code. |
**Usage**
`mailbox_check_owner()` should be implemented according to NSPE OS and specific use case. The implementation should be aligned with `mailbox_set_owner()`.
If only one ongoing PSA client call is allowed in system, `mailbox_check_owner()` can be implemented as always returning `MAILBOX_SUCCESS`.
**//`mailbox_check_owner()` should not be exported outside NSPE mailbox library.//**
#### 5.2.1.7. mailbox_get_replied_msg_owner()
This function returns the owner of the first mailbox message whose PSA client call return result is replied from SPE.
```lang=C
void *mailbox_get_replied_msg_owner(void);
```
**Return**
|||
|------------|--------------------------------------------------------------|
|`NULL` |No mailbox message is replied or owner information is not set.|
|Other values|The identification of owner of the replied mailbox message. |
**Usage**
The owner value is set by `mailbox_set_owner()` when the mailbox message is created. If no mailbox message is replied or the owner is not set, `mailbox_get_replied_msg_owner()` returns `NULL`.
If only one ongoing PSA client call is allowed in system, `mailbox_get_replied_msg_owner()` is optional.
If there are multiple mailbox messages already replied in NSPE mailbox queue, `mailbox_get_replied_msg_owner()` returns the owner value of the first replied mailbox message in the queue. NSPE OS can wake up the non-secure task waiting for PSA client call return result, according to the owner value.
If NSPE OS enforces non-secure tasks isolation, it is recommended to invoke `mailbox_get_replied_msg_owner()` in privileged mode to protect owner value from disclosure or tampering.
#### 5.2.1.8. NSPE mailbox_notify_peer()
This function invokes platform specific Inter-Processor Communication drivers to send notification to SPE.
```lang=C
int32_t mailbox_notify_peer(void);
```
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation fails with an error code. |
**Usage**
`mailbox_notify_peer()` should be implemented by platform specific Inter-Processor Communication drivers.
**//`mailbox_notify_peer()` should not be exported outside NSPE mailbox library.//**
#### 5.2.1.9. NSPE mailbox_init()
This function initializes NSPE mailbox.
```lang=C
int32_t mailbox_init(ns_mailbox_queue_t *queue);
```
**Parameters**
|||
|---------------------------|-------------------------------------------------|
|`ns_mailbox_queue_t *queue`|The base address of NSPE mailbox queue structure.|
**Return**
|||
|------------------|-----------------------------------------|
|`MAILBOX_SUCCESS` |Initialization succeeds. |
|Other return codes|Initialization fails with an error code.|
#### 5.2.1.10. NSPE mailbox_hal_ipc_init()
This function initializes platform specific Inter-Processor Communication in NSPE.
```lang=C
int32_t mailbox_hal_ipc_init(void);
```
**Return**
|||
|------------------|-----------------------------------------|
|`MAILBOX_SUCCESS` |Initialization succeeds. |
|Other return codes|Initialization fails with an error code.|
**Usage**
`mailbox_hal_ipc_init()` should be completed with platform specific Inter-Processor Communication drivers.
#### 5.2.1.11. NSPE mailbox_enter_critical()
This function enters the critical section of NSPE mailbox queue access.
```lang=C
void mailbox_enter_critical(void);
```
**Usage**
NSPE mailbox library invokes `mailbox_enter_critical()` before entering critical section of NSPE mailbox queue.
`mailbox_enter_critical()` doesn't specify the routines invoking critical section protection. NSPE OS can define and implement another function to enter critical section used in exception handling if necessary.
`mailbox_enter_critical()` implementation is platform specific.
#### 5.2.1.12. NSPE mailbox_exit_critical()
This function exits from the critical section of NSPE mailbox queue access.
```lang=C
void mailbox_exit_critical(void);
```
**Usage**
NSPE mailbox library invokes `mailbox_exit_critical()` when exiting from critical section of NSPE mailbox queue.
`mailbox_exit_critical()` doesn't specify the routines invoking critical section protection. NSPE OS can define and implement another function to exit critical section used in exception handling if necessary.
`mailbox_exit_critical()` implementation is platform specific.
### 5.2.2. SPE mailbox APIs
This section describes the SPE mailbox APIs. NSPE should not invoke these SPE mailbox APIs.
#### 5.2.2.1. tfm_mailbox_handle_msg()
This function completes the handling of mailbox messages from NSPE.
```lang=C
int32_t tfm_mailbox_handle_msg(void);
```
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation fails with an error code. |
**Usage**
`tfm_mailbox_handle_msg()` executes the following tasks:
- Check NSPE mailbox queue status
- Copy mailbox message(s) from NSPE
- Necessary checks and validations
- Parse mailbox message
- Call TF-M APIs to create PSA message(s).
#### 5.2.2.2. tfm_mailbox_reply_msg()
This function replies the PSA client call return result to NSPE.
```lang=C
int32_t tfm_mailbox_reply_msg(mailbox_msg_handle_t handle, int32_t reply);
```
**Parameters**
|||
|-----------------------------|---------------------------------------------------------|
|`mailbox_msg_handle_t handle`|The handle to mailbox message related to PSA client call.|
|`int32_t reply` |The return result of PSA client call to be replied. |
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation fails with an error code. |
**Usage**
`tfm_mailbox_reply_msg()` is invoked inside handler of `psa_reply()` to return the PSA client call return result to NSPE.
`handle` determines which mailbox message in SPE mailbox queue contains the PSA client call. `handle` can be set as `MAILBOX_MSG_NULL_HANDLE` if multiple ongoing PSA client calls are not supported in system.
#### 5.2.2.3. SPE tfm_mailbox_notify_peer()
This function invokes platform specific Inter-Processor Communication drivers to send notification to NSPE.
```lang=C
int32_t tfm_mailbox_notify_peer(void);
```
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation fails with an error code. |
**Usage**
`tfm_mailbox_notify_peer()` should be implemented by platform specific Inter-Processor Communication drivers.
**//`tfm_mailbox_notify_peer()` should not be exported outside SPE mailbox library.//**
#### 5.2.2.4. SPE tfm_mailbox_init()
This function initializes SPE mailbox.
```lang=C
int32_t tfm_mailbox_init(secure_mailbox_queue_t *s_queue,
ns_mailbox_queue_t *ns_queue);
```
**Parameters**
|||
|---------------------------------|-------------------------------------------------|
|`secure_mailbox_queue_t *s_queue`|The base address of SPE mailbox queue structure. |
|`ns_mailbox_queue_t *ns_queue` |The base address of NSPE mailbox queue structure.|
**Return**
|||
|------------------|-----------------------------------------|
|`MAILBOX_SUCCESS` |Initialization succeeds. |
|Other return codes|Initialization failed with an error code.|
#### 5.2.2.5. SPE tfm_mailbox_hal_ipc_init()
This function initializes platform specific Inter-Processor Communication in SPE.
```lang=C
int32_t tfm_mailbox_hal_ipc_init(void);
```
**Return**
|||
|------------------|-----------------------------------------|
|`MAILBOX_SUCCESS` |Initialization succeeds. |
|Other return codes|Initialization fails with an error code.|
**Usage**
`tfm_mailbox_hal_ipc_init()` should be completed with platform specific Inter-Processor Communication drivers.
#### 5.2.2.6. SPE tfm_mailbox_enter_critical()
This function enters the critical section of NSPE mailbox queue access in SPE.
```lang=C
void tfm_mailbox_enter_critical(void);
```
**Usage**
SPE mailbox library invokes `tfm_mailbox_enter_critical()` before entering critical section of NSPE mailbox queue.
`tfm_mailbox_enter_critical()` implementation is platform specific.
#### 5.2.2.7. SPE tfm_mailbox_exit_critical()
This function exits from the critical section of NSPE mailbox queue access in SPE.
```lang=C
void tfm_mailbox_exit_critical(void);
```
**Usage**
SPE mailbox library invokes `tfm_mailbox_exit_critical()` when exiting from critical section of NSPE mailbox queue.
`tfm_mailbox_exit_critical()` implementation is platform specific.
Copyright (c) 2019 Arm Limited. All Rights Reserved.
# 1. Introductions
This document proposes a generic design of the mailbox communication for Trusted Firmware-M (TF-M) on a dual core system. The mailbox functionalities transfer PSA client call request and return results between Non-secure Processing Environment (NSPE) and Secure Processing Environment (SPE).
The dual core system should satisfy the following requirements
- NSPE and SPE are properly isolated and protected following PSA.
- An Arm M-profile core locates in SPE and acts as the secure core.
- TF-M runs on the Secure core with platform specific drivers support.
- An Inter-Processor Communication hardware module in system for communication between secure core and non-secure core. Mailbox communication is based on the Inter-Processor Communication.
- Non-secure memory shared by NSPE and SPE.
## 1.1. Scope
This design document focuses on the mailbox functionalities in NSPE and SPE on a dual core system. The mailbox functionalities include the initialization of mailbox in NSPE and SPE, and the transfer of PSA client call request and reply between NSPE and SPE. Data types and mailbox APIs are defined in this document.
Some details of interactions between mailbox and other modules are specified in other design documents. Those details are skipped in this document. [Communication Prototype Between NSPE And SPE In Dual Core System](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/communication_prototype_between_nspe_and_spe_in_dual_core_system/) defines a general communication prototype between NSPE and SPE. It describes how TF-M interacts with mailbox functionalities and the general requirements on mailbox. [Booting a twin CPU system](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/bootloader/) describes a synchronization step of mailbox between NSPE and SPE during system booting.
## 1.2. Organization of the document
- Section 2 provides an overview on the mailbox architecture.
- Section 3 discusses about the mailbox communication for PSA client call.
- Section 4 introduces the initialization of mailbox.
- Section 5 lists mailbox data types and APIs.
# 2. Mailbox architecture
The mailbox consists of two parts sitting in NSPE and SPE respectively. NSPE mailbox library provides mailbox functionalities in NSPE and SPE mailbox library provides mailbox functionalities in TF-M in SPE.
TF-M provides a reference implementation of NSPE mailbox library as a part of TF-M NSPE interface library. Vendor platform can also implement a specific NSPE mailbox library following this design document.
The NSPE mailbox library delivers the PSA client call requests to SPE mailbox library. After the PSA client call result is replied from SPE, NSPE mailbox library fetches the result and returns it to non-secure tasks.
NSPE mailbox objects are managed by NSPE mailbox library in non-secure memory to hold PSA client call parameters, return result and other mailbox data.
NSPE mailbox library relies on platform specific Inter-Process Communication to process mailbox notifications between NSPE and SPE.
The SPE mailbox library in TF-M receives PSA client call requests from NSPE mailbox library. It parses the requests and invokes TF-M Remote Procedure Call (RPC) module. RPC module delivers the requests to TF-M core/Secure Partition Manager (SPM). After the PSA client call is completed, TF-M core/SPM invokes TF-M RPC module to return results to NSPE, via SPE mailbox library.
SPE mailbox objects are managed by SPE mailbox library in secure memory to hold the data copied from non-secure memory.
SPE mailbox library relies on platform specific Inter-Process Communication to process mailbox notifications between SPE and NSPE.
The architecture is showed in following figure.
{F34471}
# 3. Mailbox communication for PSA client call
This section describes the design of PSA client call request and reply transfer between NSPE and SPE via mailbox.
## 3.1. Mailbox objects
This section lists the mailbox objects required in NSPE and SPE.
NSPE mailbox objects are managed by NSPE mailbox library. But NSPE mailbox objects can be accessed by both NSPE mailbox library and SPE mailbox library.
SPE mailbox objects are managed by SPE mailbox library. SPE mailbox objects are protected from NSPE accesses by system specific isolation.
### 3.1.1. NSPE Mailbox queue
NSPE mailbox library maintains a mailbox queue in non-secure memory. Please refer to the structure definition in section 5.1.5.
NSPE mailbox queue contains one or more slots. The number of slots should be aligned with that in SPE mailbox queue.
Each slot in NSPE mailbox queue consists of a pair of a mailbox message structure and a mailbox reply structure. Each slot might contain additional fields, such as identification of non-secure task which initiates the PSA client call request. Each slot serves a PSA client call from non-secure task.
The parameters of PSA client call are hosted in the mailbox message inside the slot. Section 3.1.2 describes the details of mailbox message.
The mailbox reply is used to receive the PSA client call return result from SPE. Section 3.1.3 describes the details of mailbox reply.
### 3.1.2. Mailbox messages
A mailbox message contains the parameters of a PSA client call request from a non-secure task. Please refer to the structure definition in section 5.1.2.
When a PSA client call comes from TF-M NSPE interface library, NSPE mailbox library selects an empty mailbox queue slot for the PSA client call. The parameters of that PSA client call is organized into the mailbox message inside the selected slot. SPE mailbox library copies the mailbox message from non-secure memory to secure memory for PSA client call handling in TF-M.
A mailbox message should contains PSA client call parameters organized in structure `client_call_params_t` defined by TF-M RPC module.
More members can be defined in mailbox message to transfer additional information from NSPE to SPE for PSA client call processing.
### 3.1.3. Mailbox reply structures
A mailbox reply structure in non-secure memory receives the PSA client call return result replied from SPE mailbox library. Please refer to the structure definition in section 5.1.3.
### 3.1.4. SPE Mailbox queue
SPE mailbox library can maintain a mailbox queue to store mailbox message copied from non-secure memory. Please refer to the structure definition in section 5.1.7.
SPE mailbox queue contains one or more slots. The number of slots should be aligned with that in NSPE mailbox queue. After SPE is notified that a PSA client call request is pending, SPE mailbox library assigns an empty slot and copies PSA client call parameters from non-secure memory into this slot.
Each slot in SPE mailbox queue can contain the following fields
- A field to hold mailbox message content from non-secure memory.
- Index of NSPE mailbox queue slot containing the mailbox message.
- A handle to the mailbox message. Optional. Identify the owner slot of PSA client call return result when multiple mailbox messages are under processing.
More members can be defined in the slot structure to support mailbox processing in SPE.
## 3.2. Overall workflow
The overall workflow of transferring PSA client call request and return value between NSPE and SPE via mailbox is showed as below.
1. Non-secure task initiates a PSA client cal by calling the dedicated PSA client call API provided by TF-M NSPE interface library.
2. On a dual core system, TF-M NSPE interface library invokes NSPE mailbox library. The routine can vary in diverse NSPE OS or use case implementations. The details of this routine are out of scope.
3. NSPE mailbox library assigns an empty slot from NSPE mailbox queue for that PSA client call. If the NSPE mailbox queue is full, it should follow a waiting mechanism to wait until empty NSPE mailbox queue slot is available.
4. After step 3 succeeds, NSPE mailbox library prepares the parameters of PSA client call in the dedicated mailbox message inside the assigned slot.
5. After the mailbox message is ready, NSPE mailbox library invokes platform specific Inter-Processor Communication driver to notify SPE. The notification mechanism of Inter-Processor Communication is platform specific.
6. After the notification is completed, non-secure task waits for the reply from SPE. The mechanism of waiting and waking may vary in different NSPE OS and on diverse platforms. Please check section 3.5 for more details.
7. Platform specific Inter-Processor Communication interrupt for mailbox is asserted in SPE. Platform specific Inter-Processor Communication interrupt handler should activate SPE mailbox library to process the request(s).
8. During mailbox processing in TF-M, if there are multiple ongoing mailbox messages pending in the SPE message request queue, SPE mailbox library can process mailbox message one by one. The handling routine can include the following steps:
1. SPE mailbox library checks and validates NSPE mailbox queue status.
2. SPE mailbox library copies mailbox message(s) from non-secure memory, into SPE mailbox queue. If mailbox supports multiple ongoing PSA client call requests, it is recommended to copy multiple mailbox message(s) together to save time consumption.
3. SPE mailbox library parses a mailbox message in SPE mailbox queue. Necessary checks are required.
4. SPE mailbox library invokes the APIs provided by TF-M RPC module to deliver the PSA client call request to TF-M SPM.
5. The PSA client call is handled in TF-M SPM and target Secure Partition if necessary.
9. After the PSA client call is completed, TF-M RPC module invokes SPE mailbox library to reply PSA client call return result to NSPE.
10. SPE mailbox library writes the PSA client call return result to the dedicated mailbox reply structure in non-secure memory. The related SPE mailbox objects should be invalidated or cleaned.
11. SPE mailbox library notifies NSPE that a PSA client call completes. SPE mailbox library invokes Inter-Processor Communication driver to send notification to NSPE. The notification mechanism of Inter-Processor Communication is platform specific.
12. NSPE mailbox library is activated to handle the PSA client call return result in the mailbox reply structure. NSPE mailbox library cleans up the slot in NSPE mailbox queue after the return result is extracted out.
13. NSPE mailbox library returns the result to TF-M NSPE interface library. The related mailbox objects should be invalidated or cleaned by NSPE mailbox library.
14. TF-M NSPE interface library eventually returns the result to the non-secure task.
The following sections from 3.3 to 3.7 discuss about more details in key steps in above sequence.
## 3.3. Mailbox notifications between NSPE and SPE
As shown in the overall workflow in section 3.2, NSPE mailbox library asserts mailbox notification to trigger SPE to fetch PSA client call request. SPE mailbox library asserts mailbox notification to notify NSPE that PSA client call return result is written. The notification implementation is platform specific based on the Inter-Processor Communication.
It is recommended to assign one independent set of Inter-Processor Communication channel to each notification routine respectively, to implement a **full-duplex** notification mechanism between NSPE and SPE.
If both notification routines share the same Inter-Processor Communication channel, proper synchronization should be implemented to prevent conflicts between two notification routines.
The Inter-Processor Communication interrupt handler in TF-M should deal with the incoming notification from NSPE and activate the subsequent mailbox handling in SPE. [Communication Prototype Between NSPE And SPE In Dual Core System](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/communication_prototype_between_nspe_and_spe_in_dual_core_system/) defines the behavior of Inter-Processor Communication interrupt handler.
The handling in NSPE of the notification from SPE is IMPLEMENTATION DEFINED.
## 3.4 Implement PSA Client API with NS Mailbox (Informative)
PSA Client APIs in NSPE software are implemented with Non-secure mailbox functions with Non-secure software specific support. The implementation should be platform and NSPE OS specific.
The pseudo code of a reference implementation of a PSA Client API is shown below.
```lang=C
... psa_xxx(...)
{
mailbox_msg_handle_t handle;
...
/*
* Select an empty slot from NSPE mailbox queue and send PSA client call
* request to SPE.
* When NSPE mailbox queue is full, a proper waiting mechanism should block
* current Non-secure thread until an empty slot is available.
*/
handle = mailbox_tx_client_call_req(...);
/* Platform/NSPE OS specific waiting for PSA client call reply */
wait_for_event(mailbox_reply_event, ...);
/* Woken up to handle reply */
/* Fetch PSA client call return result */
mailbox_rx_client_call_reply(handle, ...);
...
}
```
A PSA Client function should invoke `mailbox_tx_client_call_req()` to send the request to SPE.
`mailbox_tx_client_call_req()` selects empty NSPE mailbox queue slot, creates the mailbox message and notifies SPE mailbox. The implementation is platform and NSPE software specific. The details will be described in mailbox API section.
As PSA Firmware Framework requests, a PSA Client function should be blocked until the result is return from SPE. To comply with PSA specs, the implementation of `mailbox_tx_client_call_req()` should include a proper mechanism to keep current PSA Client API caller thread waiting when NSPE mailbox queue is full. The caller thread should be blocked inside the PSA Client function until an empty NSPE mailbox slot is available. The caller thread can be switched out by NSPE OS scheduler to release CPU time to other threads.
The waiting mechanism inside `mailbox_tx_client_call_req()` can be platform and NSPE OS specific or a general implementation.
After `mailbox_tx_client_call_req()` completes, PSA Client function should invoke platform and NSPE OS specific APIs to wait for the event of client call result return. It is recommended to force PSA Client function to exclusively wait for the mailbox event. Other signals or events irrelevant to mailbox or PSA Client call should be masked during this waiting, unless the dual-core system has special requirements on thread management.
The mechanism to wait for return result and wake the waiting thread is discussed in section 3.5.
After woken up, the PSA Client function invokes `mailbox_rx_client_call_reply()` to fetch return result and release NSPE mailbox queue resource. `mailbox_rx_client_call_reply()` can optionally implement a mechanism to collaborate with the waiting mechanism in `mailbox_tx_client_call_req()`.
The details of the API will be described in mailbox API section.
## 3.5. Mechanism of waiting for PSA Client call reply in NSPE (Informative)
As discussed in section 3.4 above, PSA Client function should wait for the mailbox reply return event after the PSA Client request is submit. When the result is returned to NSPE mailbox, the waiting non-secure thread is woken up to handle the result.
To support multiple ongoing PSA Client calls feature on dual-core system, NSPE software should be able to determine which non-secure thread is the owner of the incoming PSA Client call return result. To enable NSPE to find out the non-secure thread owner, a field `owner` is defined in NSPE mailbox queue slot structure to hold the identification or handle to the non-secure thread owner. The `owner` filed maps a mailbox queue slot/mailbox message containing the PSA Client call result, to the non-secure task.
The following pseudo-code shows an example of waking . After `non_secure_task()` sends PSA client call in `psa_xxx()`, it falls into sleeping in `psa_xxx()`. After SPE notifies NSPE that PSA client call return result is replied, `notification_handler()` in NSPE OS gets the identification of waiting non-secure task by calling `mailbox_get_done_msg_owner()`. Then `notification_handler()` wakes `non_secure_task()` to fetch the return result, according to the task identification.
```lang=C
void non_secure_task(...)
{
...
/* Blocked and waiting in PSA Client API until the result is returned from SPE */
psa_xxx(...);
...
}
void notification_handler(...)
{
/* Deal with notification from SPE */
...
/* Check if any mailbox is replied in the NSPE mailbox queue */
if (mailbox_queue_has_replied_msg()) {
/* Get the handle of non-secure task whose mailbox message is replied */
task = mailbox_get_replied_msg_owner();
/* Wake up the waiting non-secure task */
wake_task(task);
}
}
```
## 3.6. Critical section protection of NSPE mailbox queue
Proper protection should be implemented to protect the critical accesses to NSPE mailbox queue. The critical sections can include atomic reading and modifying slot status, and other critical operations on NSPE mailbox queue.
NSPE and SPE mailbox library define corresponding critical section APIs. The implementation of those APIs can be platform specific. Please see more details in section 5.2.1 and 5.2.2.
The implementation should protect a critical access to NSPE mailbox queue from corruptions by
- Other non-secure tasks or exception service routines in NSPE OS
- Accesses from the peer core. SPE mailbox library also accesses NSPE mailbox queue to fetch mailbox message. Therefore, it is essential to implement synchronization or protection on NSPE mailbox queue between secure core and non-secure core.
It is recommended to rely on both hardware and software to implement the synchronization and protection.
## 3.7. Mailbox handling in TF-M
[Communication Prototype Between NSPE And SPE In Dual Core System](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/communication_prototype_between_nspe_and_spe_in_dual_core_system/) defines a generic processing of mailbox messages in TF-M.
According to the dual-core communication prototype design, mailbox implementation should invoke `tfm_rpc_register_ops()` to hook its operations to TF-M RPC module callbacks during initialization.
Mailbox message handling should call TF-M RPC PSA client call handlers to deliver PSA client call request to TF-M SPM.
If multiple ongoing PSA client calls should be supported, additional fields can be added to PSA message structure in TF-M to build up the connection between a mailbox message and a PSA message. TF-M SPM can store the mailbox message handle in a specific field in PSA message structure to identify the PSA client caller, while creating a PSA message. While replying the PSA client call return result, TF-M SPM can extract the mailbox message handle from PSA message and pass back to mailbox reply function. SPE mailbox library can identify which mailbox message is completed and write the return result to expected NSPE address.
The field `handle` in SPE mailbox queue slot structure can contain this mailbox message handle.
# 4. Mailbox initialization
This section describes the initialization sequence of mailbox in NSPE and SPE.
It should be guaranteed that NSPE mailbox library should not initiate PSA client call request until SPE mailbox library initialization completes. A synchronization between NSPE and SPE is required to protect the sequence. [Booting a twin CPU system](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/bootloader/) provides more details on the synchronization.
In current design, the base address of NSPE mailbox queue should be pre-defined and shared between NSPE mailbox library and SPE mailbox library.
## 4.1. SPE mailbox initialization
The SPE mailbox queue memory should be allocated before calling SPE `tfm_mailbox_init()`. `tfm_mailbox_init()` initializes the memory and variables. The base address of NSPE mailbox queue can be passed to SPE mailbox library via `tfm_mailbox_hal_init()` during SPE mailbox initialization.
SPE mailbox dedicated Inter-Processor Communication initialization is also enabled during SPE mailbox initialization.
After SPE mailbox initialization completes, SPE notifies NSPE that SPE mailbox functionalities are ready.
## 4.2. NSPE mailbox initialization
The NSPE mailbox queue memory should be allocated before calling NSPE `mailbox_init()`. NSPE `mailbox_init()` initializes the memory and variables.
NSPE mailbox dedicated Inter-Processor Communication initialization is also enabled during NSPE mailbox initialization.
# 5. Mailbox APIs and data structures
## 5.1. Data types
### 5.1.1. Constants
**NUM_MAILBOX_QUEUE_SLOT**
`NUM_MAILBOX_QUEUE_SLOT` set the number of slots in NSPE and SPE mailbox queues.
Both NSPE and SPE mailbox libraries should refer to the same `NUM_MAILBOX_QUEUE_SLOT` definition.
The following example configures 4 slots in mailbox queues.
```lang=C
#define NUM_MAILBOX_QUEUE_SLOT (4)
```
**MAILBOX_MSG_NULL_HANDLE**
`MAILBOX_MSG_NULL_HANDLE` is a zero-value null handle of a mailbox message.
```lang=C
#define MAILBOX_MSG_NULL_HANDLE ((mailbox_msg_handle_t)0)
```
**MAILBOX_SUCCESS**
`MAILBOX_SUCCESS` is a generic return value to indicate success of mailbox operation.
```lang=C
#define MAILBOX_SUCCESS (0)
```
**MAILBOX_QUEUE_FULL**
`MAILBOX_QUEUE_FULL` is a return value from mailbox function if mailbox queue is full.
```lang=C
#define MAILBOX_QUEUE_FULL (INT32_MIN + 1)
```
**MAILBOX_INVAL_PARAMS**
`MAILBOX_INVAL_PARAMS` is a return value from mailbox function if any parameter is invalid.
```lang=C
#define MAILBOX_INVAL_PARAMS (INT32_MIN + 2)
```
**MAILBOX_NO_PERMS**
`MAILBOX_NO_PERMS` is a return value from mailbox function if the caller doesn't any proper permission to execute the operation.
```lang=C
#define MAILBOX_NO_PERMS (INT32_MIN + 3)
```
**MAILBOX_NO_PEND_EVENT**
`MAILBOX_NO_PEND_EVENT` is a return value from mailbox function if the expected event doesn't occur yet.
```lang=C
#define MAILBOX_NO_PEND_EVENT (INT32_MIN + 4)
```
**MAILBOX_CHAN_BUSY**
`MAILBOX_CHAN_BUSY` is a return value from mailbox function if the underlying Inter-Processor Communication resource is busy.
```lang=C
#define MAILBOX_CHAN_BUSY (INT32_MIN + 5)
```
**MAILBOX_CALLBACK_REG_ERROR**
`MAILBOX_CALLBACK_REG_ERROR` is a return value from mailbox function if the registration of mailbox callback functions failed.
```lang=C
#define MAILBOX_CALLBACK_REG_ERROR (INT32_MIN + 6)
```
**PSA client call type**
The following constants define the PSA client call type value shared between NSPE and SPE
```lang=C
#define MAILBOX_PSA_FRAMEWORK_VERSION (0x1)
#define MAILBOX_PSA_VERSION (0x2)
#define MAILBOX_PSA_CONNECT (0x3)
#define MAILBOX_PSA_CALL (0x4)
#define MAILBOX_PSA_CLOSE (0x5)
```
### 5.1.2. Mailbox message
The following structures describe a mailbox message and its members.
`client_call_params_t params` are defined by TF-M RPC module to contain PSA client call parameters passed from NSPE to SPE.
```lang=C
struct mailbox_msg_t {
uint32_t call_type;
client_call_params_t params;
int32_t client_id;
};
```
### 5.1.3. Mailbox reply structure
This structure describes a mailbox reply structure, which is managed by NSPE mailbox library in non-secure memory.
```lang=C
struct mailbox_reply_t {
int32_t return_val;
};
```
### 5.1.4. Mailbox queue status bitmask
`mailbox_queue_status_t` defines a bitmask for indicating a status of slots in mailbox queues.
```lang=C
typedef uint32_t mailbox_queue_status_t;
```
### 5.1.5. NSPE mailbox queue
The following structures describe the NSPE mailbox queue and its members in non-secure memory.
```lang=C
/* A single slot structure in NSPE mailbox queue */
struct ns_mailbox_slot_t {
mailbox_msg_t msg;
mailbox_reply_t reply;
/* Identification of the owner of this slot */
void *owner;
};
struct ns_mailbox_queue_t {
mailbox_queue_status_t empty_slots;
mailbox_queue_status_t pend_slots;
mailbox_queue_status_t replied_slots;
ns_mailbox_slot_t queue[NUM_MAILBOX_QUEUE_SLOT];
};
```
### 5.1.6. Mailbox message handle
This data type is a reference to an active mailbox message in use.
```lang=C
typedef int32_t mailbox_msg_handle_t;
```
### 5.1.7. SPE mailbox queue
The following structure describe a slot in SPE mailbox queue.
The field `ns_slot_idx` records the index of NSPE mailbox slot containing the mailbox message under processing. SPE mailbox library determines the reply structure address according to this index.
The field `msg_handle` contains the handle to the mailbox message under processing. The handle can be delivered to TF-M SPM while creating PSA message. When TF-M SPM replies a PSA client call return result, the handle value is also returned to mailbox. SPE mailbox library relies on this handle to identify the owner slot of the return result, if multiple ongoing NSPE PSA client call request pending in SPE mailbox queue.
```lang=C
/* A single slot structure in SPE mailbox queue */
struct secure_mailbox_slot_t {
mailbox_msg_t msg;
uint8_t ns_slot_idx;
mailbox_msg_handle_t msg_handle;
};
```
The following structure describes the SPE mailbox queue in secure memory.
```lang=C
struct secure_mailbox_queue_t {
mailbox_queue_status_t empty_slots;
secure_mailbox_slot_t queue[NUM_MAILBOX_QUEUE_SLOT];
/* Base address of NSPE mailbox queue in non-secure memory */
ns_mailbox_queue_t *ns_queue;
};
```
## 5.2. Mailbox APIs
### 5.2.1. NSPE mailbox APIs
This section describes a **//reference design//** of NSPE mailbox APIs. SPE should not invoke these NSPE mailbox APIs.
Vendor can define and implement different NSPE mailbox APIs.
The NSPE mailbox APIs should be flexible enough to meet requirements in diverse NSPE OS and use cases.
#### 5.2.1.1. mailbox_tx_client_call_req()
This function sends the PSA client call request to SPE.
```lang=C
mailbox_msg_handle_t mailbox_tx_client_call_req(uint32_t call_type,
const struct client_call_params_t *params,
int32_t client_id);
```
**Parameters**
|||
|----------------------------------------------|--------------------------------------------------------------------------|
|`uint32_t call_type` |Type of PSA client call |
|`const struct client_call_params_t *params`|Address of PSA client call parameters structure containing the parameters.|
|`int32_t client_id` |ID of non-secure task. |
**Return**
|||
|------------|--------------------------------------------------------|
|`>= 0` |The handle to the mailbox message successfully assigned.|
|`< 0` |Operation failed with an error code. |
**Usage**
`mailbox_tx_client_call_req()` executes the following tasks:
- Select an empty NSPE mailbox queue slot. A proper mechanism is required to block current non-secure caller thread to wait for NSPE mailbox resource when NSPE mailbox queue is full. A platform/NSPE OS specific counting semaphore or other similar mechanisms can be used to implement this waiting behavior. Alternatively, it can implement a general waiting functionality independent to platform or NPSE OS.
- receives the PSA client call parameters
- prepares the mailbox message
- notifies SPE via Inter-Processor Communication
In `mailbox_tx_client_call_req()`, it calls `mailbox_set_owner()` to set the owner of the mailbox message.
#### 5.2.1.2. mailbox_rx_client_call_reply()
This function fetches the PSA client call return value after it is replied by SPE.
```lang=C
int32_t mailbox_rx_client_call_reply(mailbox_msg_handle_t handle,
int32_t *reply);
```
**Parameters**
|||
|-----------------------------|---------------------------------------------------------------|
|`mailbox_msg_handle_t handle`|The handle to the mailbox message containing the return result.|
|`int32_t *reply` |The address to be written with return result. |
**Return**
|||
|------------------|-----------------------------------------------|
|`MAILBOX_SUCCESS` |Successfully get PSA client call return result.|
|Other return codes|Operation failed with an error code |
**Usage**
A correct `handle` should be passed to `mailbox_rx_client_call_reply()` to determine the target mailbox message which sent the PSA client call.
`mailbox_rx_client_call_reply()` can invoke `mailbox_check_owner()` to check if current caller has enough permission to access the targeted mailbox message. If check passes, the return result will be extracted from mailbox reply structure and written to `reply`. Otherwise, an error code will be returned.
`mailbox_rx_client_call_reply()` can implement a corresponding mechanism to co-work with the waiting mechanism in `mailbox_tx_client_call_req()` to synchronize the NSPE mailbox queue resource with other threads. It can be implemented by platform/NSPE OS specific counting semaphore or other similar mechanisms, or a general implementation independent to platform or NSEP OS.
Before exiting `mailbox_rx_client_call_reply()`, the mailbox objects related to that completed PSA client call are invalidated or cleaned.
#### 5.2.1.3. mailbox_is_msg_replied()
This function checks if a specific mailbox message has been replied by SPE, which means the PSA client call return value is replied.
```lang=C
bool mailbox_is_msg_replied(mailbox_msg_handle_t handle);
```
**Parameters**
|||
|-----------------------------|-----------------------------------------|
|`mailbox_msg_handle_t handle`|The handle to the target mailbox message.|
**Return**
|||
|------------|----------------------------------------------------|
|`true` |The PSA client call return value is replied. |
|`false` |The PSA client call return value is not replied yet.|
#### 5.2.1.4. mailbox_queue_has_replied_msg()
This function checks if any mailbox message has been replied by SPE but the PSA client call return result is not reaped yet.
```lang=C
bool mailbox_queue_has_replied_msg(void);
```
**Return**
|||
|-------|-------------------------------------------------------------------------------------------|
|`true` |A mailbox message is replied and the return result is waiting for non-secure task to fetch.|
|`false`|No mailbox message is replied yet. |
#### 5.2.1.5. mailbox_set_owner()
This function sets the owner of an active mailbox message contained in a mailbox queue slot.
```lang=C
int32_t mailbox_set_owner(struct ns_mailbox_slot_t *slot);
```
**Parameters**
|||
|---------------------------------|---------------------------------------------------|
|`struct ns_mailbox_slot_t *slot`|The base address of a mailbox queue slot structure containing the target mailbox message.|
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation failed with an error code. |
**Usage**
`mailbox_set_owner()` should be implemented according to NSPE OS and specific use case.
The owner can be current non-secure task. The identification of the owner should be got by calling NSPE OS APIs.
If only one ongoing PSA client call is allowed in system, `mailbox_set_owner()` can be implemented as always returning `MAILBOX_SUCCESS` to simplify the implementation.
**//`mailbox_set_owner()` should not exported outside NSPE mailbox library.//**
#### 5.2.1.6. mailbox_check_owner()
This function check if the caller has enough permission to access an active mailbox message contained in a mailbox queue slot.
```lang=C
int32_t mailbox_check_owner(const struct ns_mailbox_slot_t *slot);
```
**Parameters**
|||
|---------------------------------------|---------------------------------------------------|
|`const struct ns_mailbox_slot_t *slot`|The base address of a mailbox queue slot structure containing the target mailbox message.|
**Return**
|||
|------------------|----------------------------------------------------|
|`MAILBOX_SUCCESS` |The caller is allowed to access the mailbox message.|
|Other return codes|The access is disallowed with an error code. |
**Usage**
`mailbox_check_owner()` should be implemented according to NSPE OS and specific use case. The implementation should be aligned with `mailbox_set_owner()`.
If only one ongoing PSA client call is allowed in system, `mailbox_check_owner()` can be implemented as always returning `MAILBOX_SUCCESS`.
**//`mailbox_check_owner()` should not be exported outside NSPE mailbox library.//**
#### 5.2.1.7. mailbox_get_replied_msg_owner()
This function returns the owner of the first mailbox message whose PSA client call return result is replied from SPE.
```lang=C
void *mailbox_get_replied_msg_owner(void);
```
**Return**
|||
|------------|--------------------------------------------------------------|
|`NULL` |No mailbox message is replied or owner information is not set.|
|Other values|The identification of owner of the replied mailbox message. |
**Usage**
The owner value is set by `mailbox_set_owner()` when the mailbox message is created. If no mailbox message is replied or the owner is not set, `mailbox_get_replied_msg_owner()` returns `NULL`.
If only one ongoing PSA client call is allowed in system, `mailbox_get_replied_msg_owner()` is optional.
If there are multiple mailbox messages already replied in NSPE mailbox queue, `mailbox_get_replied_msg_owner()` returns the owner value of the first replied mailbox message in the queue. NSPE OS can wake up the non-secure task waiting for PSA client call return result, according to the owner value.
If NSPE OS enforces non-secure tasks isolation, it is recommended to invoke `mailbox_get_replied_msg_owner()` in privileged mode to protect owner value from disclosure or tampering.
#### 5.2.1.8. NSPE mailbox_notify_peer()
This function invokes platform specific Inter-Processor Communication drivers to send notification to SPE.
```lang=C
int32_t mailbox_notify_peer(void);
```
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation fails with an error code. |
**Usage**
`mailbox_notify_peer()` should be implemented by platform specific Inter-Processor Communication drivers.
**//`mailbox_notify_peer()` should not be exported outside NSPE mailbox library.//**
#### 5.2.1.9. NSPE mailbox_init()
This function initializes NSPE mailbox.
```lang=C
int32_t mailbox_init(struct ns_mailbox_queue_t *queue);
```
**Parameters**
|||
|---------------------------------------|----------------------------------------------|
|`struct ns_mailbox_queue_t *queue`|The base address of NSPE mailbox queue.|
**Return**
|||
|------------------|-----------------------------------------|
|`MAILBOX_SUCCESS` |Initialization succeeds. |
|Other return codes|Initialization fails with an error code.|
`mailbox_init()` can invoke `mailbox_hal_init()` to complete platform specific mailbox and Inter-Processor Communication initialization.
The non-secure memory area for `queue` should be statically or dynamically pre-allocated before calling `mailbox_init()`.
#### 5.2.1.10. NSPE mailbox_hal_init()
This function executes platform-specific NSPE mailbox initialization.
```lang=C
int32_t mailbox_hal_init(struct ns_mailbox_queue_t *queue);
```
**Parameters**
|||
|--------------------------------|----------------------------------------------|
|`ns_mailbox_queue_t *queue`|The base address of NSPE mailbox queue.|
**Return**
|||
|------------------|-----------------------------------------|
|`MAILBOX_SUCCESS` |Initialization succeeds. |
|Other return codes|Initialization fails with an error code.|
`mailbox_init()` invokes `mailbox_hal_init()` to complete platform specific mailbox and Inter-Processor Communication initialization. `mailbox_hal_init()` can also share the address of NSPE mailbox queue with SPE mailbox library.
Platform support should implement `mailbox_hal_init()` according to specific platform and NSPE implementation.
#### 5.2.1.11. NSPE mailbox_enter_critical()
This function enters the critical section of NSPE mailbox queue access.
```lang=C
void mailbox_enter_critical(void);
```
**Usage**
NSPE mailbox library invokes `mailbox_enter_critical()` before entering critical section of NSPE mailbox queue.
`mailbox_enter_critical()` doesn't specify the routines invoking critical section protection. NSPE OS can define and implement another function to enter critical section used in exception handling if necessary.
`mailbox_enter_critical()` implementation is platform specific.
#### 5.2.1.12. NSPE mailbox_exit_critical()
This function exits from the critical section of NSPE mailbox queue access.
```lang=C
void mailbox_exit_critical(void);
```
**Usage**
NSPE mailbox library invokes `mailbox_exit_critical()` when exiting from critical section of NSPE mailbox queue.
`mailbox_exit_critical()` doesn't specify the routines invoking critical section protection. NSPE OS can define and implement another function to exit critical section used in exception handling if necessary.
`mailbox_exit_critical()` implementation is platform specific.
### 5.2.2. SPE mailbox APIs
This section describes a **//reference design//** of SPE mailbox APIs. NSPE should not invoke these SPE mailbox APIs.
#### 5.2.2.1. tfm_mailbox_handle_msg()
This function completes the handling of mailbox messages from NSPE.
```lang=C
int32_t tfm_mailbox_handle_msg(void);
```
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation fails with an error code. |
**Usage**
`tfm_mailbox_handle_msg()` executes the following tasks:
- Check NSPE mailbox queue status
- Copy mailbox message(s) from NSPE
- Necessary checks and validations
- Parse mailbox message
- Call TF-M APIs to create PSA message(s).
#### 5.2.2.2. tfm_mailbox_reply_msg()
This function replies the PSA client call return result to NSPE.
```lang=C
int32_t tfm_mailbox_reply_msg(mailbox_msg_handle_t handle, int32_t reply);
```
**Parameters**
|||
|-----------------------------|---------------------------------------------------------|
|`mailbox_msg_handle_t handle`|The handle to mailbox message related to PSA client call.|
|`int32_t reply` |The return result of PSA client call to be replied. |
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation fails with an error code. |
**Usage**
`tfm_mailbox_reply_msg()` is invoked inside handler of `psa_reply()` to return the PSA client call return result to NSPE.
`handle` determines which mailbox message in SPE mailbox queue contains the PSA client call. If `handle` is set as `MAILBOX_MSG_NULL_HANDLE`, the return result is replied to the message reply in the first SPE mailbox queue slot.
#### 5.2.2.3. SPE tfm_mailbox_notify_peer()
This function invokes platform specific Inter-Processor Communication drivers to send notification to NSPE.
```lang=C
int32_t tfm_mailbox_notify_peer(void);
```
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation fails with an error code. |
**Usage**
`tfm_mailbox_notify_peer()` should be implemented by platform specific Inter-Processor Communication drivers.
**//`tfm_mailbox_notify_peer()` should not be exported outside SPE mailbox library.//**
#### 5.2.2.4. SPE tfm_mailbox_init()
This function initializes SPE mailbox.
```lang=C
int32_t tfm_mailbox_init(void);
```
**Return**
|||
|------------------|-----------------------------------------|
|`MAILBOX_SUCCESS` |Initialization succeeds. |
|Other return codes|Initialization failed with an error code.|
`tfm_mailbox_init()` invokes `tfm_mailbox_hal_init()` to execute platform specific initialization.
#### 5.2.2.5 SPE tfm_mailbox_hal_init()
This function is implemented by platform support in TF-M. It completes platform specific mailbox initialization, including passing the the address of NSPE mailbox queue to SPE mailbox library and Inter-Processor Communication initializations and configurations.
```lang=C
int32_t tfm_mailbox_hal_init(struct secure_mailbox_queue_t *s_queue);
```
**Parameters**
|||
|----------------------------------------------|--------------------------------------------|
|`struct secure_mailbox_queue_t *s_queue` |The base address of SPE mailbox queue.|
**Return**
|||
|------------------|-----------------------------------------|
|`MAILBOX_SUCCESS` |Initialization succeeds. |
|Other return codes|Initialization failed with an error code.|
#### 5.2.2.6. SPE tfm_mailbox_enter_critical()
This function enters the critical section of NSPE mailbox queue access in SPE.
```lang=C
void tfm_mailbox_enter_critical(void);
```
**Usage**
SPE mailbox library invokes `tfm_mailbox_enter_critical()` before entering critical section of NSPE mailbox queue.
`tfm_mailbox_enter_critical()` implementation is platform specific.
#### 5.2.2.7. SPE tfm_mailbox_exit_critical()
This function exits from the critical section of NSPE mailbox queue access in SPE.
```lang=C
void tfm_mailbox_exit_critical(void);
```
**Usage**
SPE mailbox library invokes `tfm_mailbox_exit_critical()` when exiting from critical section of NSPE mailbox queue.
`tfm_mailbox_exit_critical()` implementation is platform specific.
Copyright (c) 2019 Arm Limited. All Rights Reserved.
# 1. Introductions
This document proposes a generic design of the mailbox communication for Trusted Firmware-M (TF-M) on a dual core system. The mailbox functionalities transfer PSA client call request and return results between Non-secure Processing Environment (NSPE) and Secure Processing Environment (SPE)).
The dual core system should satisfy the following requirements
- NSPE and SPE are properly isolated and protected following PSA.
- An Arm M-profile core locates in SPE and acts as the secure core.
- TF-M runs on the Secure core with platform specific drivers support.
- An Inter-Processor Communication hardware module in system for communication between secure core and non-secure core. Mailbox communication is based on the Inter-Processor Communication.
- Non-secure memory shared by NSPE and SPE.
## 1.1. Scope
This design document focuses on the mailbox functionalities in NSPE and SPE on a dual core system. The mailbox functionalities include the initialization of mailbox in NSPE and SPE, and the transfer of PSA client call request and reply between NSPE and SPE. Data types and mailbox APIs are defined in this document.
Some details of interactions between mailbox and other modules are specified in other design documents. Those details are skipped in this document. [Communication Prototype Between NSPE And SPE In Dual Core System](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/communication_prototype_between_nspe_and_spe_in_dual_core_system/) defines a general communication prototype between NSPE and SPE. It describes how TF-M interacts with mailbox functionalities and the general requirements on mailbox. [Booting a twin CPU system](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/bootloader/) describes a synchronization step of mailbox between NSPE and SPE during system booting.
## 1.2. Organization of the document
- Section 2 provides an overview on the mailbox architecture.
- Section 3 discusses about the mailbox communication for PSA client call.
- Section 4 introduces the initialization of mailbox.
- Section 5 lists mailbox data types and APIs.
# 2. Mailbox architecture
The mailbox consists of two parts sitting in NSPE and SPE respectively. NSPE mailbox library provides mailbox functionalities in NSPE and SPE mailbox library provides mailbox functionalities in TF-M in SPE.
TF-M provides a reference implementation of NSPE mailbox library as a part of TF-M NSPE interface library. Vendor platform can also implement a specific NSPE mailbox library following this design document.
The NSPE mailbox library delivers the PSA client call requests to SPE mailbox library. After the PSA client call result is replied from SPE, NSPE mailbox library fetches the result and returns it to non-secure tasks.
NSPE mailbox objects are managed by NSPE mailbox library in non-secure memory to hold PSA client call parameters, return result and other mailbox data.
NSPE mailbox library relies on platform specific Inter-Process Communication to process mailbox notifications between NSPE and SPE.
The SPE mailbox library in TF-M receives PSA client call requests from NSPE mailbox library. It parses the requests and invokes TF-M Remote Procedure Call (RPC) module. RPC module delivers theme requests to TF-M core/SPMSecure Partition Manager (SPM). After the PSA client call is completed, TF-M core/SPM invokes TF-M RPC module to return results to NSPE, TF-M core/SPM invokesvia SPE mailbox library to return results to NSPE.
SPE mailbox objects are managed by SPE mailbox library in secure memory to hold the data copied from non-secure memory.
SPE mailbox library relies on platform specific Inter-Process Communication to process mailbox notifications between SPE and NSPE.
The architecture is showed in following figure.
{F33348}{F34471}
# 3. Mailbox communication for PSA client call
This section describes the design of PSA client call request and reply transfer between NSPE and SPE via mailbox.
## 3.1. Mailbox objects
This section lists the mailbox objects required in NSPE and SPE.
### 3.1.1. NSPE mailbox objects
NSPE mailbox objects are managed by NSPE mailbox library. But NSPE mailbox objects can be accessed by both NSPE mailbox library and SPE mailbox library.
#### 3.1SPE mailbox objects are managed by SPE mailbox library. SPE mailbox objects are protected from NSPE accesses by system specific isolation.
### 3.1.1. NSPE Mailbox queue
NSPE mailbox library maintains a mailbox queue in non-secure memory. Please refer to the structure definition in section 5.1.5.
TheNSPE mailbox queue contains one or more slots. The number of slots should be aligned with that in SPE mailbox queue.
Each slot in NSPE mailbox queue consists of a pair of a mailbox message structure and a mailbox reply structure. Each slot might contain additional fields, such as identification of non-secure task which initiates the PSA client call request. Each slot serves a PSA client call from non-secure task.
The parameters of PSA client call are hosted in the mailbox message inside the slot. Section 3.1.1.2 describes the details of mailbox message.
The mailbox reply is used to receive the PSA client call return result from SPE. Section 3.1.1.3 describes the details of mailbox reply.
#### 3.1### 3.1.2. Mailbox messages
A mailbox message contains the parameters of a PSA client call request from a non-secure task. Please refer to the structure definition in section 5.1.2.
When a PSA client call comes from TF-M NSPE interface library, NSPE mailbox library selects an empty mailbox queue slot for the PSA client call. The parameters of that PSA client call is organized into the mailbox message inside the selected slot. SPE mailbox library copies the mailbox message from non-secure memory to secure memory for PSA client call handling in TF-M.
A mailbox message can contain the following fields
- PSA client call type
- Parameters required in PSA Client call. The parameters can include the following, according to PSA client call type
- Service ID (SID)
- Handle to established connection
- Input vectors and the lengths
- Output vectors and the lengths
should contains PSA client call parameters organized in structure `client_call_params_t` defined by TF-M RPC module. - Requested version of secure service
- The non-secure memory address to be written with PSA client call return result.
- NSPE Client ID. Optional. The NSPE Client ID is required when NSPE RTOS enforces non-secure task isolation.
More members can be defined in mailbox message to transfer additional information from NSPE to SPE for PSA client call processing.
#### 3.1### 3.1.3. Mailbox reply structures
A mailbox reply structure in non-secure memory receives the PSA client call return result replied from SPE mailbox library. Please refer to the structure definition in section 5.1.3.
The memory address of a mailbox reply structure in non-secure memory is set in the mailbox message in the same slot. After TF-M completes a PSA client call, SPE mailbox library writes the return result into the mailbox reply structure, according to the address set in the mailbox message dedicated to the PSA client call.
### 3.1.2. SPE mailbox objects
SPE mailbox objects are managed by SPE mailbox library. SPE mailbox objects are protected from NSPE accesses by system specific isolation.
#### 3.1.2.14. SPE Mailbox queue
SPE mailbox library can maintain a mailbox queue to store mailbox message copied from non-secure memory. Please refer to the structure definition in section 5.1.7.
TheSPE mailbox queue contains one or more slots. The number of slots should be aligned with that in NSPE mailbox queue. After SPE is notified that a PSA client call request is pending, SPE mailbox library assigns an empty slot and copies PSA client call parameters from non-secure memory into this slot.
Each slot in SPE mailbox queue can contain the following fields
- A field to hold mailbox message content from non-secure memory.
- Index of NSPE mailbox queue slot containing the mailbox message.
- A filed to hold mailbox message content from non-secure memoryhandle to the mailbox message. Optional. Identify the owner slot of PSA client call return result when multiple mailbox messages are under processing.
More members can be defined in the slot structure to support mailbox processing in SPE.
## 3.2. Overall workflow
The overall workflow of transferring PSA client call request and return value between NSPE and SPE via mailbox is showed as below.
1. Non-secure task initiates a PSA client cal by calling the dedicated PSA client call API provided by TF-M NSPE interface library.
2. On a dual core system, TF-M NSPE interface library invokes NSPE mailbox library. The routine can vary in diverse NSPE OS or use case implementations. The details of this routine are out of scope.
3. NSPE mailbox library assigns an empty slot from NSPE mailbox queue for that PSA client call. If the NSPE mailbox queue is full, NSPE mailbox will return error code `MAILBOX_QUEUE_FULL` immediatelyit should follow a waiting mechanism to wait until empty NSPE mailbox queue slot is available.
4. IfAfter step 3 succeeds, NSPE mailbox library prepares the parameters of PSA client call in the dedicated mailbox message inside the assigned slot.
5. After the mailbox message is ready, NSPE mailbox library invokes platform specific Inter-Processor Communication driver to notify SPE. The notification mechanism of Inter-Processor Communication is platform specific.
6. After the notification is completed, non-secure task waits for the reply from SPE. The mechanism of waiting and waking may vary in different NSPE OS and on diverse platforms. Please check section 3.45 for more details.
7. Platform specific Inter-Processor Communication interrupt for mailbox is asserted in SPE. Platform specific Inter-Processor Communication interrupt handler should activate SPE mailbox library to process the request(s).
8. During mailbox processing in TF-M, if there are multiple ongoing mailbox messages pending in the SPE message request queue, SPE mailbox library can process mailbox message one by one. The handling routine can include the following steps:
1. SPE mailbox library checks and validates NSPE mailbox queue status.
2. SPE mailbox library copies mailbox message(s) from non-secure memory, into SPE mailbox queue.
If mailbox supports multiple ongoing PSA client call requests, it is recommended to copy multiple mailbox message(s) together to save time consumption.
3. SPE mailbox library parses a mailbox message in SPE mailbox queue. Necessary checks are required.
4. SPE mailbox library invokes the APIs provided by TF-M SPMRPC module to deliver the PSA client call request to TF-M SPM.
5. The PSA client call is handled in TF-M SPM and target Secure Partition if necessary.
If there are multiple ongoing mailbox messages pending in the SPE message request queue9. After the PSA client call is completed, TF-M RPC module invokes SPE mailbox library can process mailbox message one by oneto reply PSA client call return result to NSPE.
9. After the PSA client call is completed,10. TF-M SPM invokes SPE mailbox library to replywrites the PSA client call return result to NSPE.
10the dedicated mailbox reply structure in non-secure memory. SPE mailbox library writes the PSA client call return result to the dedicThe related SPE mailbox objects should be invalidated mailbox reply structure in non-secure memoryor cleaned.
11. SPE mailbox library notifies NSPE that a PSA client call completes. SPE mailbox library invokes Inter-Processor Communication driver to send notification to NSPE. The notification mechanism of Inter-Processor Communication is platform specific.
12. NSPE mailbox library is activated to handle the PSA client call return result in the mailbox reply structure. NSPE mailbox library cleans up the slot in NSPE mailbox queue after the return result is extracted out.
13. NSPE mailbox library returns the result to TF-M NSPE interface library. The related mailbox objects should be invalidated or cleaned by NSPE mailbox library.
14. TF-M NSPE interface library eventually returns the result to the non-secure task.
The following sections from 3.3 to 3.67 discuss about more details in key steps in above sequence.
## 3.3. Mailbox notifications between NSPE and SPE
As shown in the overall workflow in section 3.2, NSPE mailbox library asserts mailbox notification to trigger SPE to fetch PSA client call request. SPE mailbox library asserts mailbox notification to notify NSPE that PSA client call return result is written. The notification implementation is platform specific based on the Inter-Processor Communication.
It is recommended to assign one independent set of Inter-Processor Communication channel to each notification routine respectively, to implement a **full-duplex*** notification mechanism between NSPE and SPE.
If both notification routines share the same Inter-Processor Communication channel, proper synchronization should be implemented to prevent conflicts between two notification routines.
The Inter-Processor Communication interrupt handler in TF-M should deal with the incoming notification from NSPE and activate the subsequent mailbox handling in SPE. [Communication Prototype Between NSPE And SPE In Dual Core System](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/communication_prototype_between_nspe_and_spe_in_dual_core_system/) defines the behavior of Inter-Processor Communication interrupt handler.
The handling in NSPE of the notification from SPE is IMPLEMENTATION DEFINED.
## 3.4. Mechanism of waiting for PSA client call reply in NSPE
After the notification of PSA client call request is sent to SPE, the non-secure task in NSPE can be blocked and wait for the PSA client call reply from SPE. After NSPE receives the notification of PSA client call completion from SPE, the waiting non-secure task is woken up to fetch the PSA client call return result. Implement PSA Client API with NS Mailbox (Informative)
If only one ongoing PSA client call is allowed in system,PSA Client APIs in NSPE software are implemented with Non-secure mailbox functions with Non-secure software specific support. the non-secure task waiting for reply can spin in a busy waiting without being scheduled outThe implementation should be platform and NSPE OS specific.
If multiple ongoing PSA client calls are required, NSPE OS can switch another non-secure task in while current task is waiting for PSA client call reply. The sleeping and waking mechanism can be more complex and rely on specific NSPE OS thread management and use case. To keep the entire implementations flexible enough, this document doesn't define or implement a dedicated waiting/waking mechanism. The implementation of waiting/waking mechanism should be accomplished by NSPE OS and TF-M NSPE interface libraryThe pseudo code of a reference implementation of a PSA Client API is shown below.
A field `owner` is defined in NSPE mailbox queue slot structure to map a mailbox queue slot/mailbox message to the non-secure task, to support the feature of multiple ongoing PSA client calls.
The following pseudo-code shows an example of waiting and waking mechanism for supporting multiple ongoing PSA client calls. After `non_secure_task()` sends PSA client call, it falls into sleeping. After SPE notifies NSPE that PSA client call return result is replied, `notification_handler()` in NSPE OS gets the identification of waiting non-secure task by calling `mailbox_get_done_msg_owner()`. Then `notification_handler()` wakes `non_secure_task()` to fetch the return result, according to the task identification.
```lang=C
void non_secure_task... psa_xxx(...)
{
mailbox_msg_handle_t handle;
...
/* Send PSA client call request to SPE
* Select an empty slot from NSPE mailbox queue and send PSA client call
* request to SPE.
* When NSPE mailbox queue is full, a proper waiting mechanism should block
* current Non-secure thread until an empty slot is available.
*/
handle = mailbox_tx_client_call_req(...);
/* Platform/NSPE OS specific waiting for PSA client call reply */
sleep_wait(wait_for_event(mailbox_reply_event, ...);
/* Platform/NSPE OS specific Woken up *//* Woken up to handle reply */
/* Fetch PSA client call return result */
mailbox_rx_client_call_reply(handle, ...);
...
}
```
A PSA Client function should invoke `mailbox_tx_client_call_req()` to send the request to SPE.
`mailbox_tx_client_call_req()` selects empty NSPE mailbox queue slot, creates the mailbox message and notifies SPE mailbox. The implementation is platform and NSPE software specific. The details will be described in mailbox API section.
As PSA Firmware Framework requests, a PSA Client function should be blocked until the result is return from SPE. To comply with PSA specs, the implementation of `mailbox_tx_client_call_req()` should include a proper mechanism to keep current PSA Client API caller thread waiting when NSPE mailbox queue is full. The caller thread should be blocked inside the PSA Client function until an empty NSPE mailbox slot is available. The caller thread can be switched out by NSPE OS scheduler to release CPU time to other threads.
The waiting mechanism inside `mailbox_tx_client_call_req()` can be platform and NSPE OS specific or a general implementation.
After `mailbox_tx_client_call_req()` completes, PSA Client function should invoke platform and NSPE OS specific APIs to wait for the event of client call result return. It is recommended to force PSA Client function to exclusively wait for the mailbox event. Other signals or events irrelevant to mailbox or PSA Client call should be masked during this waiting, unless the dual-core system has special requirements on thread management.
The mechanism to wait for return result and wake the waiting thread is discussed in section 3.5.
After woken up, the PSA Client function invokes `mailbox_rx_client_call_reply()` to fetch return result and release NSPE mailbox queue resource. `mailbox_rx_client_call_reply()` can optionally implement a mechanism to collaborate with the waiting mechanism in `mailbox_tx_client_call_req()`.
The details of the API will be described in mailbox API section.
## 3.5. Mechanism of waiting for PSA Client call reply in NSPE (Informative)
As discussed in section 3.4 above, PSA Client function should wait for the mailbox reply return event after the PSA Client request is submit. When the result is returned to NSPE mailbox, the waiting non-secure thread is woken up to handle the result.
To support multiple ongoing PSA Client calls feature on dual-core system, NSPE software should be able to determine which non-secure thread is the owner of the incoming PSA Client call return result. To enable NSPE to find out the non-secure thread owner, a field `owner` is defined in NSPE mailbox queue slot structure to hold the identification or handle to the non-secure thread owner. The `owner` filed maps a mailbox queue slot/mailbox message containing the PSA Client call result, to the non-secure task.
The following pseudo-code shows an example of waking . After `non_secure_task()` sends PSA client call in `psa_xxx()`, it falls into sleeping in `psa_xxx()`. After SPE notifies NSPE that PSA client call return result is replied, `notification_handler()` in NSPE OS gets the identification of waiting non-secure task by calling `mailbox_get_done_msg_owner()`. Then `notification_handler()` wakes `non_secure_task()` to fetch the return result, according to the task identification.
```lang=C
void non_secure_task(...)
{
...
/* Blocked and waiting in PSA Client API until the result is returned from SPE */
psa_xxx(...);
...
}
void notification_handler(...)
{
/* Deal with notification from SPE */
...
/* Check if any mailbox is replied in the NSPE mailbox queue */
if (mailbox_queue_has_replied_msg()) {
/* Get the handle of non-secure task whose mailbox message is replied */
task = mailbox_get_replied_msg_owner();
/* Wake up the waiting non-secure task */
wake_task(task);
}
}
```
## 3.56. Critical section protection of NSPE mailbox queue
Proper protection should be implemented to protect the critical accesses to NSPE mailbox queue. The critical sections can include atomic reading and modifying slot status, and other critical operations on NSPE mailbox queue.
NSPE and SPE mailbox library define corresponding critical section APIs. The implementation of those APIs can be platform specific. Please see more details in section 5.2.1 and 5.2.2.
The implementation should protect a critical access to NSPE mailbox queue from corruptions by
- Other non-secure tasks or exception service routines in NSPE OS
- Accesses from the peer core. SPE mailbox library also accesses NSPE mailbox queue to fetch mailbox message. Therefore, it is essential to implement synchronization or protection on NSPE mailbox queue between secure core and non-secure core.
It is recommended to rely on both hardware and software to implement the synchronization and protection.
## 3.67. Mailbox handling in TF-M
[Communication Prototype Between NSPE And SPE In Dual Core System](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/communication_prototype_between_nspe_and_spe_in_dual_core_system/) defines a generic processing of mailbox messages in TF-M.
If multiple ongoing PSA client calls should be supported, additional fields can be added to PSA message structure in TF-M to build up the connection between a mailbox message and a PSA message. TF-M SPM can store the mailbox message handle in a specific filed in PSA message structure to identify the PSA client caller, while creating a PSA message. While replying the PSA client call return resultAccording to the dual-core communication prototype design, TF-M SPM can extract the mailbox message handle from PSA message and pass back to mailbox reply function. SPE mailbox library can identify which mailbox message is completed and write the return result to expected NSPE addressmailbox implementation should invoke `tfm_rpc_register_ops()` to hook its operations to TF-M RPC module callbacks during initialization.
Mailbox message handling should call TF-M RPC PSA client call handlers to deliver PSA client call request to TF-M SPM.
The details related to changes of TF-M will be documented in further design documentsIf multiple ongoing PSA client calls should be supported, additional fields can be added to PSA message structure in TF-M to build up the connection between a mailbox message and a PSA message. TF-M SPM can store the mailbox message handle in a specific field in PSA message structure to identify the PSA client caller, while creating a PSA message. While replying the PSA client call return result, TF-M SPM can extract the mailbox message handle from PSA message and pass back to mailbox reply function. SPE mailbox library can identify which mailbox message is completed and write the return result to expected NSPE address.
The field `handle` in SPE mailbox queue slot structure can contain this mailbox message handle.
# 4. Mailbox initialization
This section describes the initialization sequence of mailbox in NSPE and SPE.
It should be guaranteed that NSPE mailbox library should not initiate PSA client call request until SPE mailbox library initialization completes. A synchronization between NSPE and SPE is required to protect the sequence. [Booting a twin CPU system](https://developer.trustedfirmware.org/w/tf_m/design/twin-cpu/bootloader/) provides more details on the synchronization.
In current design, the base address of NSPE mailbox queue should be pre-defined and shared between NSPE mailbox library and SPE mailbox library.
## 4.1. SPE mailbox initialization
The SPE mailbox queue memory should be allocated before calling SPE `tfm_mailbox_init()`. `tfm_mailbox_init()` initializes the memory and variables. The base address of NSPE mailbox queue can be passed to SPE mailbox library via `tfm_mailbox_hal_init()` during SPE mailbox initialization.
SPE mailbox dedicated Inter-Processor Communication initialization is also enabled during SPE mailbox initialization.
After SPE mailbox initialization completes, SPE notifies NSPE that SPE mailbox functionalities are ready.
## 4.2. NSPE mailbox initialization
The NSPE mailbox queue memory should be allocated before calling NSPE `mailbox_init()`. NSPE `mailbox_init()` initializes the memory and variables.
NSPE mailbox dedicated Inter-Processor Communication initialization is also enabled during NSPE mailbox initialization.
# 5. Mailbox APIs and data structures
## 5.1. Data types
### 5.1.1. Constants
**NUM_MAILBOX_QUEUE_SLOT**
`NUM_MAILBOX_QUEUE_SLOT` set the number of slots in NSPE and SPE mailbox queues.
Both NSPE and SPE mailbox libraries should refer to the same `NUM_MAILBOX_QUEUE_SLOT` definition.
The following example configures 4 slots in mailbox queues.
```lang=C
#define NUM_MAILBOX_QUEUE_SLOT (4)
```
**MAILBOX_MSG_NULL_HANDLE**
`MAILBOX_MSG_NULL_HANDLE` is a zero-value null handle of a mailbox message.
```lang=C
#define MAILBOX_MSG_NULL_HANDLE ((mailbox_msg_handle_t)0)
```
**MAILBOX_SUCCESS**
`MAILBOX_SUCCESS` is a generic return value to indicate success of mailbox operation.
```lang=C
#define MAILBOX_SUCCESS (0)
```
**MAILBOX_QUEUE_FULL**
`MAILBOX_QUEUE_FULL` is a return value from mailbox function if mailbox queue is full.
```lang=C
#define MAILBOX_QUEUE_FULL (INT32_MIN + 1)
```
**MAILBOX_INVAL_PARAMS**
`MAILBOX_INVAL_PARAMS` is a return value from mailbox function if any parameter is invalid.
```lang=C
#define MAILBOX_INVAL_PARAMS (INT32_MIN + 2)
```
**MAILBOX_NO_PERMS**
`MAILBOX_NO_PERMS` is a return value from mailbox function if the caller doesn't any proper permission to execute the operation.
```lang=C
#define MAILBOX_NO_PERMS (INT32_MIN + 3)
```
### 5.1.2. Mailbox message**MAILBOX_NO_PEND_EVENT**
The following structures describe a mailbox message and its members.`MAILBOX_NO_PEND_EVENT` is a return value from mailbox function if the expected event doesn't occur yet.
```lang=C
#define MAILBOX_NO_PEND_EVENT (INT32_MIN + 4)
```
**MAILBOX_CHAN_BUSY**
`MAILBOX_CHAN_BUSY` is a return value from mailbox function if the underlying Inter-Processor Communication resource is busy.
```lang=C
/* Structure to hold parameters of PSA client call */#define MAILBOX_CHAN_BUSY (INT32_MIN + 5)
typedef struct client_call_params_t {```
**MAILBOX_CALLBACK_REG_ERROR**
`MAILBOX_CALLBACK_REG_ERROR` is a return value from mailbox function if the registration of mailbox callback functions failed.
uint32_t sid;```lang=C
psa_handle_t#define MAILBOX_CALLBACK_REG_ERROR handle;(INT32_MIN + 6)
```
**PSA client call type**
The following const psa_invec *in_vec;ants define the PSA client call type value shared between NSPE and SPE
size_t in_len;```lang=C
psa_outvec *out_vec;#define MAILBOX_PSA_FRAMEWORK_VERSION (0x1)
size_t#define MAILBOX_PSA_VERSION out_len; (0x2)
#define MAILBOX_PSA_CONNECT uint32_t(0x3)
#define MAILBOX_PSA_CALL minor_version; (0x4)
} client_call_params_t;#define MAILBOX_PSA_CLOSE (0x5)
```
### 5.1.2. Mailbox message
The following structures describe a mailbox message and its members.
`client_call_params_t params` are defined by TF-M RPC module to contain PSA client call parameters passed from NSPE to SPE.
typedef ```lang=C
struct mailbox_msg_t {
uint32_t call_type;
client_call_params_t params;
const void *reply_base;
int32_t client_id;
} mailbox_msg_t;
```
### 5.1.3. Mailbox reply structure
This structure describes a mailbox reply structure, which is managed by NSPE mailbox library in non-secure memory.
```lang=C
typedef struct mailbox_reply_t {
int32_t return_val;
} mailbox_reply_t;
```
### 5.1.4. Mailbox queue status bitmask
`mailbox_queue_status_t` defines a bitmask for indicating a status of slots in mailbox queues.
```lang=C
typedef uint32_t mailbox_queue_status_t;
```
### 5.1.5. NSPE mailbox queue
The following structures describe the NSPE mailbox queue and its members in non-secure memory.
```lang=C
/* A single slot structure in NSPE mailbox queue */
typedef struct ns_mailbox_slot_t {
mailbox_msg_t msg;
mailbox_reply_t reply;
/* Identification of the owner of this slot */
void *owner;
} ns_mailbox_slot_t;
typedef struct ns_mailbox_queue_t {
mailbox_queue_status_t empty_slots;
mailbox_queue_status_t pend_slots;
mailbox_queue_status_t replied_slots;
/* Base address of mailbox queue in non-secure memory */
ns_mailbox_slot_t queue[NUM_MAILBOX_QUEUE_SLOT];
} ns_mailbox_queue_t;
```
### 5.1.6. Mailbox message handle
This data type is a reference to an active mailbox message in use.
```lang=C
typedef int32_t mailbox_msg_handle_t;
```
### 5.1.7. SPE mailbox queue
The following structures describe the SPE mailbox queue and its structure members in secure memory describe a slot in SPE mailbox queue.
The field `ns_slot_idx` records the index of NSPE mailbox slot containing the mailbox message under processing. SPE mailbox library determines the reply structure address according to this index.
The field `msg_handle` contains the handle to the mailbox message under processing. The handle can be delivered to TF-M SPM while creating PSA message. When TF-M SPM replies a PSA client call return result, the handle value is also returned to mailbox. SPE mailbox library relies on this handle to identify the owner slot of the return result, if multiple ongoing NSPE PSA client call request pending in SPE mailbox queue.
```lang=C
/* A single slot structure in SPE mailbox queue */
typedef struct secure_mailbox_slot_t {
mailbox_msg_t msg; msg;
uint8_t ns_slot_idx;
mailbox_msg_handle_t msg_handle;
};
} secure_```
The following structure describes the SPE mailbox_slot_t; queue in secure memory.
typedef ```lang=C
struct secure_mailbox_queue_t {
mailbox_queue_status_t empty_slots;
/* Base address of secure_mailbox_slot_t queue in secure memory */[NUM_MAILBOX_QUEUE_SLOT];
secure_/* Base address of NSPE mailbox_slot_t queue[NUM_MAILBOX_QUEUE_SLOT]; queue in non-secure memory */
ns_mailbox_queue_t *ns_queue;
} secure_mailbox_queue_t;
```
## 5.2. Mailbox APIs
### 5.2.1. NSPE mailbox APIs
This section describes thea **//reference design//** of NSPE mailbox APIs. SPE should not invoke these NSPE mailbox APIs.
Vendor can define and implement different NSPE mailbox APIs.
The NSPE mailbox APIs should be flexible enough to meet requirements in diverse NSPE OS and use cases.
#### 5.2.1.1. mailbox_tx_client_call_req()
This function sends the PSA client call request to SPE.
```lang=C
mailbox_msg_handle_t mailbox_tx_client_call_req(uint32_t call_type,
const struct client_call_params_t *params,
int32_t client_id);
```
**Parameters**
|||
|----------------------------------------------|--------------------------------------------------------------------------|
|`uint32_t call_type` |Type of PSA client call |
|`const struct client_call_params_t *params`|Address of PSA client call parameters structure containing the parameters.|
|`int32_t client_id` |ID of non-secure task. |
**Return**
|||
|------------|--------------------------------------------------------|
|`>= 0` |The handle to the mailbox message successfully assigned.|
|`< 0` |Operation failed with an error code. |
**Usage**
`mailbox_tx_client_call_req()` executes the following tasks:
- Select an empty NSPE mailbox queue slot. A proper mechanism is required to block current non-secure caller thread to wait for NSPE mailbox resource when NSPE mailbox queue is full. A platform/NSPE OS specific counting semaphore or other similar mechanisms can be used to implement this waiting behavior. Alternatively, it can implement a general waiting functionality independent to platform or NPSE OS.
- receives the PSA client call parameters
- prepares the mailbox message
- notifies SPE via Inter-Processor Communication
In `mailbox_tx_client_call_req()`, it calls `mailbox_set_owner()` to set the owner of the mailbox message.
#### 5.2.1.2. mailbox_rx_client_call_reply()
This function fetches the PSA client call return value after it is replied by SPE.
```lang=C
int32_t mailbox_rx_client_call_reply(mailbox_msg_handle_t handle,
int32_t *reply);
```
**Parameters**
|||
|-----------------------------|---------------------------------------------------------------|
|`mailbox_msg_handle_t handle`|The handle to the mailbox message containing the return result.|
|`int32_t *reply` |The address to be written with return result. |
**Return**
|||
|------------------|-----------------------------------------------|
|`MAILBOX_SUCCESS` |Successfully get PSA client call return result.|
|Other return codes|Operation failed with an error code |
**Usage**
A correct `handle` should be passed to `mailbox_rx_client_call_reply()` to determine the target mailbox message which sent the PSA client call.
`mailbox_rx_client_call_reply()` can invokes `mailbox_check_owner()` to check if current caller has enough permission to access the targeted mailbox message. If check passes, the return result will be extracted from mailbox reply structure and written to `reply`. Otherwise, an error code will be returned.
`mailbox_rx_client_call_reply()` can implement a corresponding mechanism to co-work with the waiting mechanism in `mailbox_tx_client_call_req()` to synchronize the NSPE mailbox queue resource with other threads. It can be implemented by platform/NSPE OS specific counting semaphore or other similar mechanisms, an error code will be returnedor a general implementation independent to platform or NSEP OS.
Before exiting `mailbox_rx_client_call_reply()`, the mailbox objects related to that completed PSA client call are invalidated or cleaned.
#### 5.2.1.3. mailbox_is_msg_replied()
This function checks if a specific mailbox message has been replied by SPE, which means the PSA client call return value is replied.
```lang=C
bool mailbox_is_msg_replied(mailbox_msg_handle_t handle);
```
**Parameters**
|||
|-----------------------------|-----------------------------------------|
|`mailbox_msg_handle_t handle`|The handle to the target mailbox message.|
**Return**
|||
|------------|----------------------------------------------------|
|`true` |The PSA client call return value is replied. |
|`false` |The PSA client call return value is not replied yet.|
#### 5.2.1.4. mailbox_queue_has_replied_msg()
This function checks if any mailbox message has been replied by SPE but the PSA client call return result is not reaped yet.
```lang=C
bool mailbox_queue_has_replied_msg(void);
```
**Return**
|||
|-------|-------------------------------------------------------------------------------------------|
|`true` |A mailbox message is replied and the return result is waiting for non-secure task to fetch.|
|`false`|No mailbox message is replied yet. |
#### 5.2.1.5. mailbox_set_owner()
This function sets the owner of an active mailbox message contained in a mailbox queue slot.
```lang=C
int32_t mailbox_set_owner(struct ns_mailbox_slot_t *slot);
```
**Parameters**
|||
|---------------------------------|---------------------------------------------------|
|`struct ns_mailbox_slot_t *slot`|The base address of a mailbox queue slot structure containing the target mailbox message.|
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation failed with an error code. |
**Usage**
`mailbox_set_owner()` should be implemented according to NSPE OS and specific use case.
The owner can be current non-secure task. The identification of the owner should be got by calling NSPE OS APIs.
If only one ongoing PSA client call is allowed in system, `mailbox_set_owner()` can be implemented as always returning `MAILBOX_SUCCESS` to simplify the implementation.
**//`mailbox_set_owner()` should not exported outside NSPE mailbox library.//**
#### 5.2.1.6. mailbox_check_owner()
This function check if the caller has enough permission to access an active mailbox message contained in a mailbox queue slot.
```lang=C
int32_t mailbox_check_owner(const struct ns_mailbox_slot_t *slot);
```
**Parameters**
|||
|---------------------------------------|---------------------------------------------------|
|`const struct ns_mailbox_slot_t *slot`|The base address of a mailbox queue slot structure containing the target mailbox message.|
**Return**
|||
|------------------|----------------------------------------------------|
|`MAILBOX_SUCCESS` |The caller is allowed to access the mailbox message.|
|Other return codes|The access is disallowed with an error code. |
**Usage**
`mailbox_check_owner()` should be implemented according to NSPE OS and specific use case. The implementation should be aligned with `mailbox_set_owner()`.
If only one ongoing PSA client call is allowed in system, `mailbox_check_owner()` can be implemented as always returning `MAILBOX_SUCCESS`.
**//`mailbox_check_owner()` should not be exported outside NSPE mailbox library.//**
#### 5.2.1.7. mailbox_get_replied_msg_owner()
This function returns the owner of the first mailbox message whose PSA client call return result is replied from SPE.
```lang=C
void *mailbox_get_replied_msg_owner(void);
```
**Return**
|||
|------------|--------------------------------------------------------------|
|`NULL` |No mailbox message is replied or owner information is not set.|
|Other values|The identification of owner of the replied mailbox message. |
**Usage**
The owner value is set by `mailbox_set_owner()` when the mailbox message is created. If no mailbox message is replied or the owner is not set, `mailbox_get_replied_msg_owner()` returns `NULL`.
If only one ongoing PSA client call is allowed in system, `mailbox_get_replied_msg_owner()` is optional.
If there are multiple mailbox messages already replied in NSPE mailbox queue, `mailbox_get_replied_msg_owner()` returns the owner value of the first replied mailbox message in the queue. NSPE OS can wake up the non-secure task waiting for PSA client call return result, according to the owner value.
If NSPE OS enforces non-secure tasks isolation, it is recommended to invoke `mailbox_get_replied_msg_owner()` in privileged mode to protect owner value from disclosure or tampering.
#### 5.2.1.8. NSPE mailbox_notify_peer()
This function invokes platform specific Inter-Processor Communication drivers to send notification to SPE.
```lang=C
int32_t mailbox_notify_peer(void);
```
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation fails with an error code. |
**Usage**
`mailbox_notify_peer()` should be implemented by platform specific Inter-Processor Communication drivers.
**//`mailbox_notify_peer()` should not be exported outside NSPE mailbox library.//**
#### 5.2.1.9. NSPE mailbox_init()
This function initializes NSPE mailbox.
```lang=C
int32_t mailbox_init(struct ns_mailbox_queue_t *queue);
```
**Parameters**
|||
|---------------------------|---------------|----------------------------------------------|
|`struct ns_mailbox_queue_t *queue`|The base address of NSPE mailbox queue structure.|
**Return**
|||
|------------------|-----------------------------------------|
|`MAILBOX_SUCCESS` |Initialization succeeds. |
|Other return codes|Initialization fails with an error code.|
`mailbox_init()` can invoke `mailbox_hal_init()` to complete platform specific mailbox and Inter-Processor Communication initialization.
The non-secure memory area for `queue` should be statically or dynamically pre-allocated before calling `mailbox_init()`.
#### 5.2.1.10. NSPE mailbox_hal_ipc_init()
This function initializexecutes platform -specific Inter-Processor Communication in NSPENSPE mailbox initialization.
```lang=C
int32_t mailbox_hal_ipc_init(void);nit(struct ns_mailbox_queue_t *queue);
```
**Parameters**
|||
|--------------------------------|----------------------------------------------|
|`ns_mailbox_queue_t *queue`|The base address of NSPE mailbox queue.|
**Return**
|||
|------------------|-----------------------------------------|
|`MAILBOX_SUCCESS` |Initialization succeeds. |
|Other return codes|Initialization fails with an error code.|
**Usage**
`mailbox_hal_ipc_init()` should be completed with platform specific Inter-Processor Communication drivers`mailbox_init()` invokes `mailbox_hal_init()` to complete platform specific mailbox and Inter-Processor Communication initialization. `mailbox_hal_init()` can also share the address of NSPE mailbox queue with SPE mailbox library.
Platform support should implement `mailbox_hal_init()` according to specific platform and NSPE implementation.
#### 5.2.1.11. NSPE mailbox_enter_critical()
This function enters the critical section of NSPE mailbox queue access.
```lang=C
void mailbox_enter_critical(void);
```
**Usage**
NSPE mailbox library invokes `mailbox_enter_critical()` before entering critical section of NSPE mailbox queue.
`mailbox_enter_critical()` doesn't specify the routines invoking critical section protection. NSPE OS can define and implement another function to enter critical section used in exception handling if necessary.
`mailbox_enter_critical()` implementation is platform specific.
#### 5.2.1.12. NSPE mailbox_exit_critical()
This function exits from the critical section of NSPE mailbox queue access.
```lang=C
void mailbox_exit_critical(void);
```
**Usage**
NSPE mailbox library invokes `mailbox_exit_critical()` when exiting from critical section of NSPE mailbox queue.
`mailbox_exit_critical()` doesn't specify the routines invoking critical section protection. NSPE OS can define and implement another function to exit critical section used in exception handling if necessary.
`mailbox_exit_critical()` implementation is platform specific.
### 5.2.2. SPE mailbox APIs
This section describes thea **//reference design//** of SPE mailbox APIs. NSPE should not invoke these SPE mailbox APIs.
#### 5.2.2.1. tfm_mailbox_handle_msg()
This function completes the handling of mailbox messages from NSPE.
```lang=C
int32_t tfm_mailbox_handle_msg(void);
```
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation fails with an error code. |
**Usage**
`tfm_mailbox_handle_msg()` executes the following tasks:
- Check NSPE mailbox queue status
- Copy mailbox message(s) from NSPE
- Necessary checks and validations
- Parse mailbox message
- Call TF-M APIs to create PSA message(s).
#### 5.2.2.2. tfm_mailbox_reply_msg()
This function replies the PSA client call return result to NSPE.
```lang=C
int32_t tfm_mailbox_reply_msg(mailbox_msg_handle_t handle, int32_t reply);
```
**Parameters**
|||
|-----------------------------|---------------------------------------------------------|
|`mailbox_msg_handle_t handle`|The handle to mailbox message related to PSA client call.|
|`int32_t reply` |The return result of PSA client call to be replied. |
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation fails with an error code. |
**Usage**
`tfm_mailbox_reply_msg()` is invoked inside handler of `psa_reply()` to return the PSA client call return result to NSPE.
`handle` determines which mailbox message in SPE mailbox queue contains the PSA client call. `handle` can be set as `MAILBOX_MSG_NULL_HANDLE` if multiple ongoing PSA client calls are not supported in systemIf `handle` is set as `MAILBOX_MSG_NULL_HANDLE`, the return result is replied to the message reply in the first SPE mailbox queue slot.
#### 5.2.2.3. SPE tfm_mailbox_notify_peer()
This function invokes platform specific Inter-Processor Communication drivers to send notification to NSPE.
```lang=C
int32_t tfm_mailbox_notify_peer(void);
```
**Return**
|||
|------------------|-------------------------------------|
|`MAILBOX_SUCCESS` |The operation completes successfully.|
|Other return codes|Operation fails with an error code. |
**Usage**
`tfm_mailbox_notify_peer()` should be implemented by platform specific Inter-Processor Communication drivers.
**//`tfm_mailbox_notify_peer()` should not be exported outside SPE mailbox library.//**
#### 5.2.2.4. SPE tfm_mailbox_init()
This function initializes SPE mailbox.
```lang=C
int32_t tfm_mailbox_init(secure_mailbox_queue_t *s_queue,
ns_mailbox_queue_t *ns_queue);void);
```
**Parameters**
|||
|---------------------------------|-------------------------------------------------|
|`secure_mailbox_queue_t *s_queue`|The base address of SPE mailbox queue structure. |
|`ns_mailbox_queue_t *ns_queue` |The base address of NSPE mailbox queue structure.|
**Return**
|||
|------------------|-----------------------------------------|
|`MAILBOX_SUCCESS` |Initialization succeeds. |
|Other return codes|Initialization failed with an error code.|
#### 5.2.2.5. SPE `tfm_mailbox_init()` invokes `tfm_mailbox_hal_ipc_init()init()` to execute platform specific initialization.
This function initializes platform specific Inter-Processor Communication in SPE#### 5.2.2.5 SPE tfm_mailbox_hal_init()
This function is implemented by platform support in TF-M. It completes platform specific mailbox initialization, including passing the the address of NSPE mailbox queue to SPE mailbox library and Inter-Processor Communication initializations and configurations.
```lang=C
int32_t tfm_mailbox_hal_ipc_init(void);nit(struct secure_mailbox_queue_t *s_queue);
```
**Parameters**
|||
|----------------------------------------------|--------------------------------------------|
|`struct secure_mailbox_queue_t *s_queue` |The base address of SPE mailbox queue.|
**Return**
|||
|------------------|-----------------------------------------|
|`MAILBOX_SUCCESS` |Initialization succeeds. |
|Other return codes|Initialization failsed with an error code.|
**Usage**
`tfm_mailbox_hal_ipc_init()` should be completed with platform specific Inter-Processor Communication drivers.
#### 5.2.2.6. SPE tfm_mailbox_enter_critical()
This function enters the critical section of NSPE mailbox queue access in SPE.
```lang=C
void tfm_mailbox_enter_critical(void);
```
**Usage**
SPE mailbox library invokes `tfm_mailbox_enter_critical()` before entering critical section of NSPE mailbox queue.
`tfm_mailbox_enter_critical()` implementation is platform specific.
#### 5.2.2.7. SPE tfm_mailbox_exit_critical()
This function exits from the critical section of NSPE mailbox queue access in SPE.
```lang=C
void tfm_mailbox_exit_critical(void);
```
**Usage**
SPE mailbox library invokes `tfm_mailbox_exit_critical()` when exiting from critical section of NSPE mailbox queue.
`tfm_mailbox_exit_critical()` implementation is platform specific.
Copyright (c) 2019 Arm Limited. All Rights Reserved.