it seems there are two existing workarounds related to OpenSSL3 available in different downstream projects.

Hi Andre-ARM, to fix the problem, here's the pull request: https://github.com/ARM-software/arm-trusted-firmware/pull/1988

In T1003#11851, @Andre-ARM wrote:

Hi,
thanks for the info, I will have a look at this, though probably only later this week.
As for rebuilding: I assume you are using the firmware drops from the pftf github?
This is deeply hidden in the EDK2 build system, but it will effectively use a precompiled version of bl31.bin checked into the edk2-non-osi.git repository. This will be placed into the first 128KB of RPI_EFI.fd file, filled up with 0xff.
So to replace just bl31.bin, you simply overwrite the first part of that file, with your compiled version.
To get bl31.bin from source, you just need an aarch64 (cross-)compiler, then:
$ CROSS_COMPILE=aarch64-linux-gnu- make PLAT=rpi4 DEBUG=0
This should be described in docs/plat/rpi4.rst. If you find something missing, let me know, or even better: send a patch ;-)

Hi,
thanks for the info, I will have a look at this, though probably only later this week.
As for rebuilding: I assume you are using the firmware drops from the pftf github?
This is deeply hidden in the EDK2 build system, but it will effectively use a precompiled version of bl31.bin checked into the edk2-non-osi.git repository. This will be placed into the first 128KB of RPI_EFI.fd file, filled up with 0xff.
So to replace just bl31.bin, you simply overwrite the first part of that file, with your compiled version.
To get bl31.bin from source, you just need an aarch64 (cross-)compiler, then:
$ CROSS_COMPILE=aarch64-linux-gnu- make PLAT=rpi4 DEBUG=0
This should be described in docs/plat/rpi4.rst. If you find something missing, let me know, or even better: send a patch ;-)

In T1003#11849, @Andre-ARM wrote:

So that's a lot of details (thanks for that!), but what is the actual problem? That secondaries cannot be taken offline? Or that they don't came back online? And did that work before the commit you mentioned?

So that's a lot of details (thanks for that!), but what is the actual problem? That secondaries cannot be taken offline? Or that they don't came back online? And did that work before the commit you mentioned?

I can confirm this occurs with binutils 2.39. We (coreboot) are trying to update binutils from our toolchain and we are about to adjust our build system. --no-warn-rwx-segment fixes the issue. https://review.coreboot.org/c/coreboot/+/66920

Pinging @Andre-ARM RPi4 platform code owner for comment.

Hello Olivier,

Thanks for your reply. Looks like there is a fix under works:

Adding that the linking warns about both rwx-sections and execstack for bl2 too.
So both are needed or the linking needs to be fixed.
I think the no-warn flags are only available to newer tools, so defaulting to them will probably break things.

Hello Olivier,

Hi,
From the logs I understand BL31 is started by U-Boot SPL rather than TF-A's BL1/BL2, correct?
Would it be possible to gather more verbose logs (build with DEBUG=1 LOG_LEVEL=50) ?
Can you share the TF-A command line used to build this platform?
In particular what's the state of EL3_EXCEPTION_HANDLING, SPD, SPMD_SPM_AT_SEL2 toggles?
Thanks, Olivier.

Ping @raghuncstate for comment https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/commit/?id=7c2fe62f1347bb94d82e9fdd3bc5eaebedaf0bc7

Hey Chris, I may have raised the bug wrong we are tracking internally as its binutils-2.39, sorry!

Hi Heitbaum, could you tell me which toolchain you're using to build TF-A? The latest Arm GNU AArch64 toolchain is 11.3.Rel1, which packages binutils-2.38 and therefore compiles successfully, so I'm currently unable to reproduce this error.

I see. Thanks for the replies. Feel free to close this task then.

Regarding this web page reporting system its mainly now being used for Bug reporting. There is now a TF-A mailing list https://lists.trustedfirmware.org/mailman3/lists/tf-a.lists.trustedfirmware.org/ where many more people can help with questions.

Yes, this is a known issue. The DTs for the base FVP model were once imported from the Linux tree, but haven't been updated since. Meanwhile both DTC and the DT schema compliance tooling in the kernel tree got stricter, so the old files trigger warnings now.
One could go ahead and just fix each of those warnings, but I am actually working on rearranging the FVP DT files, so we can sync them from the kernel tree. That should fix those messages automatically.

I tried to follow the guidelines on https://github.com/ARM-software/tf-issues but I can't find where to add the "question" label. Sorry about this.

Hi Heitaum, Thanks for reporting this.

Sure, people are of course free to do what they want downstream. Especially as a temporary measure if this is ultimately needed to be upstreamed with fuller discussion once other stakeholders are available to facilitate that.

Thanks Joanna for letting know. This is actually blocking development so I'm thinking we go ahead with a local change that we think will be best and then we can discuss that change when Soby is back. What do you think?

Soby, himself is out until towards end of August. Maybe wait now until Beginning of Septif this is not urgent?

Apologies, holiday delays on our side too :)

HI Mirschkym,

Hi Mirschkym, just acknowledging your ticket. We are working on a response on how best to address your issue.

Hi Okash,
Olivier is on holiday and once he is back next week, we can arrange something to discuss.

Thanks. It seems like we are converging. Would it make sense to set up a meeting to thrash out details? Any time this week will be preferable. Let me know what works for you. Arve and Peter are in Pacific time zone. Rest of us are based in UK I guess?

That is not what I meant. The NS SVE/FP access trap is needed to avoid saving and restoring the state when the SVE/FP registers are actively used by the secure world.

In T989#11708, @soby-mathew wrote:

Hi Arve

S-EL2 cannot lazily save and restore the non secure register state though since it cannot trap accesses by the normal world. I have not thought much about how to optimize a lazy save and restore mechanism where a lower exception level also uses lazy save and restore, but I don't think the secure world is fundamentally different from the normal world here.

S-EL2 does not need to trap access by Normal world to do a lazy save and restore. The sequence that I have in mind would be something like below:

1. EL3 switches to the S-EL2 on receipt of a FF-A call from Non Secure. SPM schedules the right S-EL1 partition with the SVE and FP trap enabled. Note that the NS FPU/SVE is still present in the registers at this point in time.
2. S-EL1 SP now tries to access FP/SVE and takes a trap to SPM in S-EL2. SPM now saves the NS FPU/SVE context and restores the S-EL1 SP FP/SVE context and disables the trap. It reenters the S-EL1 partition.
3. The partition is now able to use SVE/FP and completes its work. Returns back to SPM.
4. SPM now saves the SP FP/SVE context and restores the NS SVE/FP context. Return back to NS caller via EL3.

As can be seen, The SPM does not need to trap NS SVE/FP accesses.

That is not what I meant. The NS SVE/FP access trap is needed to avoid saving and restoring the state when the SVE/FP registers are actively used by the secure world, but not by the normal world. In the sequence you describe the lazy save and restore is only lazy when the secure world does not use the SVE/FP registers.

Hi,

on systems that don't support S-EL2, SPMC functionality mostly, if not all, resides in EL3

This is an implementation choice. E.g. OP-TEE implements an S-EL1 SPMC without needing SPMC logic at EL3 (beyond the SPMD as FF-A relayer).
If you consider the EL3 FF-A SPMC just recently added, yes most of the SPMC logic resides at EL3.

Hey Soby and Olivier, on systems that don't support S-EL2, SPMC functionality mostly, if not all, resides in EL3 right? Going by that convention, would it make sense to have SVE save and restore in EL3? We can make it part of SPMC code in EL3. For additional space we can make use of DDR carveout as Soby mentioned above. Moreover, if we make that context save and restore part enablement configurable at compile time, then platform can choose whether they want the functionality. Would it then be acceptable?

Hi,
Today SEL2 unconditionally saves/restores FP/SIMD/SVE NS context on any SEL2 entry/exit.
I believe it could be optimized the way Soby is describing it by bullets 1,2,3,4.
It is worth noting that when SEL2 is not present (e.g. using the EL3 FF-A SPMC and a SEL1 TOS), the same lazy NS and TA contexts save/restore mechanism can be used by a SEL1 TOS and EL3 doesn't have to bother.

Hi Arve

In T989#11703, @soby-mathew wrote:

When you have a S-EL2 based system with possibly multiple S-EL1 partitions, it would be complex to implement a scheme where in EL3 will restore the right S-EL1 context on taking a trap during lazy save mechanism. In such systems, it is easier for S-EL2 to implement such a scheme since it is the manager for S-EL1 contexts.

S-EL2 cannot lazily save and restore the non secure register state though since it cannot trap accesses by the normal world. I have not thought much about how to optimize a lazy save and restore mechanism where a lower exception level also uses lazy save and restore, but I don't think the secure world is fundamentally different from the normal world here. You can have lazy save and restore in NS-EL1, NS-EL2, S-EL1, S-EL2 and EL3. I think it is worthwhile to see how this can be optimized to avoid saving and restoring register states that will not be used, but I would like to see a solution that does not leak data between execution environments that are supposed to be isolated.

TF-A works on a 6 month release cadence and we typically update the gcc toolchain to the latest released version along with the TF-A release. gcc versions we use are downloaded from https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/downloads.
Currently we don't update the toolchain to any versions newer than the one released here. We used the version 10.3-2021.07 with TF-A v2.6 release and will be updating to version 11.2-2022.02 with our upcoming v2.7 release which is planned for the end of May, 2022.

When you have a S-EL2 based system with possibly multiple S-EL1 partitions, it would be complex to implement a scheme where in EL3 will restore the right S-EL1 context on taking a trap during lazy save mechanism. In such systems, it is easier for S-EL2 to implement such a scheme since it is the manager for S-EL1 contexts.

Hey Olivier,