This page is a brief overview of the coding for initial v8-R64 support, and to help with code reviews . More specifically the goals are:
- To briefly summarize the coding done, then
- clarify the nature of the files of the three necessarily large patches into more-manageable review slices, and
- to clarify some aspects of the patches that are not easy to see from the Gerrit reviews alone.
The Gerrit reviews created for this new platform are, necessarily, large. It’s not really feasible to break them down into much-smaller patches, with each independently-compilable pieces. For example, we originally wanted to split https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/10518 into one patch for No-EL3, and another for MPU. However, they had to be implemented simultaneously because:
- We couldn't do the MPU changes before No-EL3, because no platform that has both EL3 and an MPU exists, and
- we couldn't practically do the No-EL3 changes before MPU support, because many (perhaps most) of the EL3-system-register references are in the MMU-library code. We would have had to make many No-EL3 changes to the MMU-library code, only to abandon all of those changes when we switch over to MPU.
//**Overview of Implementation and Gerrit Reviews**//
The v8-R64 support was added in three distinct steps, each with its associated patch (Gerrit review).
- Creating the firmware framework for the new FVP_R platform.
- Changes to divert EL3-system-register accesses to corresponding EL2 registers, and creating `.../trusted-firmware-a/lib/xlat_mpu`, an MPU equivalent of the `.../trusted-firmware-a/lib/xlat_tables_v2` MMU library.
- Loading System Validation OS as a mockup of the Partner/Customer runtime environment, then transfering control to it.
//**Firmware Framework**//
This initial patchset is the platform creation of `.../trusted-firmware-a/plat/arm/board/fvp_r`, which includes the framework files adapted from `.../trusted-firmware-a/plat/arm/board/fvp` with some references removed such at BL2, BL31, etc. and different features removed that are not applicable for `fvp_r`. Most of this patch is new files that create the patch, with a few memory map changes needed for R.
Link to patch:
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/10512
//**No-EL3 and MPU**//
This second patch primarily consists of creating alternative functions to those that access EL3 system registers. The alternative functions similarly access the corresponding EL2 system registers. In many cases, new alternative functions were added in new `.../trusted-firmware-a/plat/arm/board/fvp_r/` files. In some cases though, the `#define`d label `NO_EL3` was used to choose between EL3 and EL2 code pieces within an existing function. (This label may be changed to `BL1_AT_EL2`.)
Sometimes we used macro parameterization to avoid duplicating too much code, such as with the `el3_common_macros.S` being changed to `el_max_common_macros.S` with a parameter to specify the max EL for the particular core. All references to that had to also change.
This patch also involved creating an MPU alternative to the existing `.../trusted-firmware-a/lib/xlat_tables_v2` MMU library. Because MPUs are inherently much-more-limited than MMUs, this new `.../trusted-firmware-a/lib/xlat_mpu` library is considerably simpler than the MMU libraries. Nevertheless, they are still setup via two NULL-terminated (in essence - zero length) arrays of `mmap_region_t`, called `bl_regions` and `plat_regions`.
Link to patch:
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/10518
//**Load and Hand-off to Customer/Partner Runtime System**//
This patchset includes the changes required for loading and jumping to the customer/partner runtime system, in other words jumping from BL1 directly to BL33. In this case, BL33 is intended to be an test-OS binary which we are using to represent the customer/partner runtime system.
- Creation of `bl1_load_bl33()` function, much like `bl1_load_bl2()` function in common code to specifically load BL33 that loads and authenticates the partner runtime image.
- Addition of `disable_mpu_icache_el2()` and `disable_mpu_el2()` helper functions to disable the MPU before jumping to the next image.
- Changes to `bl1_transfer_bl33()` function for preparing the context of the next BL33 image.
- Addition of a `bl1_run_next_image()` function where the exception return takes place to jump to the runtime image.
- General changes from hardcoded BL2 to BL33 as the next image for BL1.
- Move of FIP from NOR Flash0 to DRAM due to the size of the FIP required for our testing purposes for our test-OS.
- Instantiation of the BL33 image based on our test-OS.
Note: This patch allows BL1 to jump to the test-OS image.
Link to patch:
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/10519
//**Files to Review**//
How to use this table:
- Please see the first column below, "Review Types(s)," describing the high-level nature of – impetus for – that file's edits. It may make sense for a single person to review all of, say, the "MPU" changes.
- Each of the three patches has a column – 10512, 10518, and 10519. (Note that number that text functions as a link to the Gerrit review.)
- The entries in those three columns chronicle the history of the file for that particular row. For example, the eighth row, for file `.../trusted-firmware-a/plat/arm/board/fvp_r/fvp_r_common.c`...
- Was created in the first patch, 10512, thus the "Created" in that column, then
- was modifed in the second patch, 10518, thus the "Modified" in that column.
- Each of the "Created" or "Modified" markings in these three columns are working links to the particular files in that particular Gerrit review.
- Click on them to review that file's changes in that particular patch.
- In the case of new files (see the New File? column), please also see the link in the Review Information.
- As usual, the Gerrit review for new files shows that file "appearing out of thin air," so to speak.
- As you would expect, these were not really created from scratch.
- The links in the Review Information column take you to an `sdiff` of the final form of that file (prior to incorporating review comments) compared to the previously existing file from which it was originally made.
- Please make review comments in the Gerrit reviews based upon these `sdiff` outputs in the Gerrit reviews as usual. We'll respond and/or make modifications as requested.
| Review Type(s) | File Name: | 10512(Oldest Check-In) | 10518 | 10519 (Newest Check-In) | New file? | Review Information | Review Notes |
| Max EL = EL2 | plat/intel/soc/common/aarch64/plat_helpers.S | | Modified | | No | | v8-R64 cores do not support EL3, so el3_entrypoint_common changed to el_max_entrypoint_common with a parameter to specify the max EL for the particular core. |
| MPU | plat/arm/common/arm_common.mk | | Modified | | No | | |
| MPU | plat/arm/common/arm_bl1_setup.c | | Modified | Modified | No | | |
| Platform Setup, Max EL = EL2, MPU | plat/arm/board/fvp_r/platform.mk | Created | Modified | Modified | Yes | plat/arm/board/fvp/platform.mk vs. plat/arm/board/fvp_r/platform.mk | |
| Platform Setup | plat/arm/board/fvp_r/jmptbl.i | Created | | | Yes | plat/arm/board/fvp/jmptbl.i vs. plat/arm/board/fvp_r/jmptbl.i | |
| Platform Setup, Address Remap | plat/arm/board/fvp_r/include/platform_def.h | Created | Modified | Modified | Yes | plat/arm/board/fvp/include/platform_def.h vs. plat/arm/board/fvp_r/include/platform_def.h | Address map for addresses 0-0xffffffff on v8-R64 platform are mapped addresses same as v8-A FVP, except that bit 31 is toggled. |
| Platform Setup | plat/arm/board/fvp_r/include/plat.ld.S | Created | | | Yes | plat/arm/board/fvp/include/plat.ld.S vs. plat/arm/board/fvp_r/include/plat.ld.S | |
| Platform Setup | plat/arm/board/fvp_r/fvp_r_trusted_boot.c | Created | | | Yes | plat/arm/board/fvp/fvp_trusted_boot.c vs. plat/arm/board/fvp_r/fvp_r_trusted_boot.c | |
| Platform Setup | plat/arm/board/fvp_r/fvp_r_stack_protector.c | Created | | | Yes | plat/arm/board/fvp/fvp_stack_protector.c vs. plat/arm/board/fvp_r/fvp_r_stack_protector.c | |
| Platform Setup | plat/arm/board/fvp_r/fvp_r_security.c | Created | | | Yes | plat/arm/board/fvp/fvp_security.c vs. plat/arm/board/fvp_r/fvp_r_security.c | |
| Platform Setup | plat/arm/board/fvp_r/fvp_r_private.h | Created | | | Yes | plat/arm/board/fvp/fvp_private.h vs. plat/arm/board/fvp_r/fvp_r_private.h | |
| Max EL = EL2 | plat/arm/board/fvp_r/fvp_r_pauth_helpers.S | | Created | | Yes | ./lib/extensions/pauth/pauth_helpers.S vs. plat/arm/board/fvp_r/fvp_r_pauth_helpers.S | |
| Max EL = EL2 | plat/arm/board/fvp_r/fvp_r_misc_helpers.S | | Created | | Yes | lib/aarch64/misc_helpers.S vs. plat/arm/board/fvp_r/fvp_r_misc_helpers.S | |
| Platform Setup, Runtime Env. Handoff | plat/arm/board/fvp_r/fvp_r_io_storage.c | Created | | Modified | Yes | plat/arm/board/fvp/fvp_io_storage.c vs. plat/arm/board/fvp_r/fvp_r_io_storage.c | |
| Platform Setup | plat/arm/board/fvp_r/fvp_r_helpers.S | Created | | | Yes | plat/arm/board/fvp/aarch64/fvp_helpers.S vs. plat/arm/board/fvp_r/fvp_r_helpers.S | |
| Platform Setup | plat/arm/board/fvp_r/fvp_r_err.c | Created | | | Yes | plat/arm/board/fvp/fvp_err.c vs. plat/arm/board/fvp_r/fvp_r_err.c | |
| Max EL = EL2 | plat/arm/board/fvp_r/fvp_r_el_max_common_macros.S | | Created | | Yes | [[https://developer.trustedfirmware.org/w/tf_a/rclasssupportdocs/includearchaarch64el3_common_macrossvsplatarmboardel_max_common_macross/ | include/arch/aarch64/el3_common_macros.S vs. plat/arm/board/fvp_r/fvp_r_el_max_common_macros.S]] | |
| Platform Setup, Address Remap | plat/arm/board/fvp_r/fvp_r_def.h | Created | Modified | | Yes | plat/arm/board/fvp/fvp_def.h vs. plat/arm/board/fvp_r/fvp_r_def.h | Address map for addresses 0-0xffffffff on v8-R64 platform are mapped addresses same as v8-A FVP, except that bit 31 is toggled. |
| Max EL = EL2 | plat/arm/board/fvp_r/fvp_r_debug.S | | Created | | Yes | [[https://developer.trustedfirmware.org/w/tf_a/rclasssupportdocs/commonaarch64debugsvsplatarmboardfvp_rfvp_r_debugs/ | common/aarch64/debug.S vs. plat/arm/board/fvp_r/fvp_r_debug.S]] | |
| Max EL = EL2 | plat/arm/board/fvp_r/fvp_r_context_mgmt.c | | Created | | Yes | lib/el3_runtime/aarch64/context_mgmt.c vs. plat/arm/board/fvp_r/fvp_r_context_mgmt.c | |
| Max EL = EL2 | plat/arm/board/fvp_r/fvp_r_context.S | | Created | | Yes | lib/el3_runtime/aarch64/context.S vs. lib/el3_runtime/aarch64/context.S | No change from lib/el3_runtime/aarch64/context.S. |
| Max EL = EL2 | plat/arm/board/fvp_r/fvp_r_common.c | Created | Modified | | Yes | plat/arm/board/fvp/fvp_common.c vs. plat/arm/board/fvp_r/fvp_r_common.c | |
| Platform Setup, Runtime Env. Handoff | plat/arm/board/fvp_r/fvp_r_bl1_setup.c | Created | Modified | Modified | Yes | plat/arm/board/fvp/fvp_bl1_setup.c vs. plat/arm/board/fvp_r/fvp_r_bl1_setup.c | |
| Max EL = EL2, Runtime Env. Handoff | plat/arm/board/fvp_r/fvp_r_bl1_main.c | | Created | Modified | Yes | [[https://developer.trustedfirmware.org/w/tf_a/rclasssupportdocs/bl1bl1_maincvsplatarmboardfvp_rfvp_r_bl1_mainc/ | bl1/bl1_main.c vs. plat/arm/board/fvp_r/fvp_r_bl1_main.c]] | |
| Max EL = EL2 | plat/arm/board/fvp_r/fvp_r_bl1_exceptions.S | | Created | | Yes | [[https://developer.trustedfirmware.org/w/tf_a/rclasssupportdocs/bl1aarch64bl1_exceptionssvsplatarmboardfvp_rfvp_r_bl1_exceptionss/ | bl1/aarch64/bl1_exceptions.S vs. plat/arm/board/fvp_r/fvp_r_bl1_exceptions.S]] | |
| Max EL = EL2, Runtime Env. Handoff | plat/arm/board/fvp_r/fvp_r_bl1_entrypoint.S | | Created | Modified | Yes | [[https://developer.trustedfirmware.org/w/tf_a/rclasssupportdocs/bl1aarch64bl1_entrypointsvsplatarmboardfvp_rfvp_r_bl1_entrypoints/ | bl1/aarch64/bl1_entrypoint.S vs. plat/arm/board/fvp_r/fvp_r_bl1_entrypoint.S]] | |
| Max EL = EL2, Runtime Env. Handoff | plat/arm/board/fvp_r/fvp_r_bl1_context_mgmt.c | | Created | Modified | Yes | bl1/aarch64/bl1_context_mgmt.c vs. plat/arm/board/fvp_r/fvp_r_bl1_context_mgmt.c | |
| Max EL = EL2 | plat/arm/board/fvp_r/fvp_r_bl1_arch_setup.c | | Created | | Yes | [[https://developer.trustedfirmware.org/w/tf_a/rclasssupportdocs/bl1aarch64bl1_arch_setupcvsplatarmboardfvp_rfvp_r_bl1_arch_setupc/ | bl1/aarch64/bl1_arch_setup.c vs. plat/arm/board/fvp_r/fvp_r_bl1_arch_setup.c]] | |
| Platform Setup | plat/arm/board/fvp_r/fdts/fvp_r_tb_fw_config.dts | Created | | | Yes | plat/arm/board/fvp/fdts/fvp_tb_fw_config.dts vs. plat/arm/board/fvp_r/fdts/fvp_r_tb_fw_config.dts | |
| Platform Setup | plat/arm/board/fvp_r/fdts/fvp_r_soc_fw_config.dts | Created | | | Yes | plat/arm/board/fvp/fdts/fvp_soc_fw_config.dts vs. plat/arm/board/fvp_r/fdts/fvp_r_soc_fw_config.dts | |
| Platform Setup | plat/arm/board/fvp_r/fdts/fvp_r_nt_fw_config.dts | Created | | | Yes | plat/arm/board/fvp/fdts/fvp_nt_fw_config.dts vs. plat/arm/board/fvp_r/fdts/fvp_r_nt_fw_config.dts | |
| Platform Setup | plat/arm/board/fvp_r/fdts/fvp_r_fw_config.dts | Created | | | Yes | plat/arm/board/fvp/fdts/fvp_fw_config.dts vs. plat/arm/board/fvp_r/fdts/fvp_r_soc_fw_config.dts | |
| MPU | lib/xlat_tables/aarch64/xlat_tables.c | | Modified | | | | |
| MPU | lib/xlat_mpu/xlat_mpu_utils.c | | Created | | Yes | lib/xlat_tables_v2/xlat_tables_utils.c vs. lib/xlat_mpu/xlat_mpu_utils.c | |
| MPU | lib/xlat_mpu/xlat_mpu_private.h | | Created | | Yes | lib/xlat_tables_v2/xlat_tables_private.h vs. lib/xlat_mpu/xlat_mpu_private.h | |
| MPU | lib/xlat_mpu/xlat_mpu_core.c | | Created | | Yes | lib/xlat_tables_v2/xlat_tables_core.c vs. lib/xlat_mpu/xlat_mpu_core.c | |
| MPU | lib/xlat_mpu/xlat_mpu_context.c | | Created | | Yes | lib/xlat_tables_v2/xlat_tables_context.c vs. lib/xlat_mpu/xlat_mpu_context.c | |
| MPU | lib/xlat_mpu/xlat_mpu.mk | | Created | | Yes | lib/xlat_tables_v2/xlat_tables.mk vs. lib/xlat_mpu/xlat_mpu.mk | |
| MPU | lib/xlat_mpu/ro_xlat_mpu.mk | | Created | | Yes | lib/xlat_tables_v2/ro_xlat_tables.mk vs. lib/xlat_mpu/ro_xlat_mpu.mk | |
| MPU | lib/xlat_mpu/aarch64/xlat_mpu_arch.c | | Created | | Yes | lib/xlat_tables_v2/aarch64/xlat_tables_arch.c vs. lib/xlat_mpu/aarch64/xlat_mpu_arch.c | |
| MPU | lib/xlat_mpu/aarch64/enable_mmu.S | | Created | | Yes | lib/xlat_tables_v2/aarch64/enable_mmu.S vs. lib/xlat_mpu/aarch64/enable_mmu.S | |
| Max EL = EL2, Runtime Env. Handoff | lib/aarch64/misc_helpers.S | | | Modified | No | | |
| MPU | include/plat/arm/common/plat_arm.h | | Modified | No | | | |
| Platform Setup, Address Remap | include/plat/arm/common/arm_def.h | Modified | Modified | | No | | Address map for addresses 0-0xffffffff on v8-R64 platform are mapped addresses same as v8-A FVP, except that bit 31 is toggled. |
| Platform Setup, Address Remap | include/plat/arm/board/common/v2m_def.h | | Modified | | No | | Address map for addresses 0-0xffffffff on v8-R64 platform are mapped addresses same as v8-A FVP, except that bit 31 is toggled. |
| Max EL = EL2 | include/arch/aarch64/el3_common_macros.S | | Modified | | No | | |
| Max EL = EL2, Runtime Env. Handoff | include/arch/aarch64/arch_helpers.h | | Modified | Modified | No | | |
| Max EL = EL2 | include/arch/aarch64/arch.h | | Modified | | No | | |
| Max EL = EL2 | bl31/aarch64/bl31_entrypoint.S | | Modified | | No | | v8-R64 cores do not support EL3, so el3_entrypoint_common changed to el_max_entrypoint_common with a parameter to specify the max EL for the particular core. |
| Max EL = EL2 | bl2/aarch64/bl2_el3_entrypoint.S | | Modified | | No | | v8-R64 cores do not support EL3, so el3_entrypoint_common changed to el_max_entrypoint_common with a parameter to specify the max EL for the particular core. |
| Max EL = EL2 | bl1/bl1_main.c | | Modified | | No | | |
| Max EL = EL2 | bl1/aarch64/bl1_exceptions.S | | Modified | | No | | |
| Max EL = EL2 | bl1/aarch64/bl1_entrypoint.S | | Modified | | No | | v8-R64 cores do not support EL3, so el3_entrypoint_common changed to el_max_entrypoint_common with a parameter to specify the max EL for the particular core. |
| Max EL = EL2 | bl1/aarch64/bl1_context_mgmt.c | | Modified | | No | | |
| Max EL = EL2 | bl1/aarch64/bl1_arch_setup.c | | Modified | | No | | |