Soby, himself is out until towards end of August. Maybe wait now until Beginning of Septif this is not urgent?

Apologies, holiday delays on our side too :)

Hi Okash,
Olivier is on holiday and once he is back next week, we can arrange something to discuss.

Thanks. It seems like we are converging. Would it make sense to set up a meeting to thrash out details? Any time this week will be preferable. Let me know what works for you. Arve and Peter are in Pacific time zone. Rest of us are based in UK I guess?

That is not what I meant. The NS SVE/FP access trap is needed to avoid saving and restoring the state when the SVE/FP registers are actively used by the secure world.

In T989#11708, @soby-mathew wrote:

Hi Arve

S-EL2 cannot lazily save and restore the non secure register state though since it cannot trap accesses by the normal world. I have not thought much about how to optimize a lazy save and restore mechanism where a lower exception level also uses lazy save and restore, but I don't think the secure world is fundamentally different from the normal world here.

S-EL2 does not need to trap access by Normal world to do a lazy save and restore. The sequence that I have in mind would be something like below:

1. EL3 switches to the S-EL2 on receipt of a FF-A call from Non Secure. SPM schedules the right S-EL1 partition with the SVE and FP trap enabled. Note that the NS FPU/SVE is still present in the registers at this point in time.
2. S-EL1 SP now tries to access FP/SVE and takes a trap to SPM in S-EL2. SPM now saves the NS FPU/SVE context and restores the S-EL1 SP FP/SVE context and disables the trap. It reenters the S-EL1 partition.
3. The partition is now able to use SVE/FP and completes its work. Returns back to SPM.
4. SPM now saves the SP FP/SVE context and restores the NS SVE/FP context. Return back to NS caller via EL3.

As can be seen, The SPM does not need to trap NS SVE/FP accesses.

That is not what I meant. The NS SVE/FP access trap is needed to avoid saving and restoring the state when the SVE/FP registers are actively used by the secure world, but not by the normal world. In the sequence you describe the lazy save and restore is only lazy when the secure world does not use the SVE/FP registers.

Hi,

on systems that don't support S-EL2, SPMC functionality mostly, if not all, resides in EL3

This is an implementation choice. E.g. OP-TEE implements an S-EL1 SPMC without needing SPMC logic at EL3 (beyond the SPMD as FF-A relayer).
If you consider the EL3 FF-A SPMC just recently added, yes most of the SPMC logic resides at EL3.

Hey Soby and Olivier, on systems that don't support S-EL2, SPMC functionality mostly, if not all, resides in EL3 right? Going by that convention, would it make sense to have SVE save and restore in EL3? We can make it part of SPMC code in EL3. For additional space we can make use of DDR carveout as Soby mentioned above. Moreover, if we make that context save and restore part enablement configurable at compile time, then platform can choose whether they want the functionality. Would it then be acceptable?

Hi,
Today SEL2 unconditionally saves/restores FP/SIMD/SVE NS context on any SEL2 entry/exit.
I believe it could be optimized the way Soby is describing it by bullets 1,2,3,4.
It is worth noting that when SEL2 is not present (e.g. using the EL3 FF-A SPMC and a SEL1 TOS), the same lazy NS and TA contexts save/restore mechanism can be used by a SEL1 TOS and EL3 doesn't have to bother.

Hi Arve

In T989#11703, @soby-mathew wrote:

When you have a S-EL2 based system with possibly multiple S-EL1 partitions, it would be complex to implement a scheme where in EL3 will restore the right S-EL1 context on taking a trap during lazy save mechanism. In such systems, it is easier for S-EL2 to implement such a scheme since it is the manager for S-EL1 contexts.

S-EL2 cannot lazily save and restore the non secure register state though since it cannot trap accesses by the normal world. I have not thought much about how to optimize a lazy save and restore mechanism where a lower exception level also uses lazy save and restore, but I don't think the secure world is fundamentally different from the normal world here. You can have lazy save and restore in NS-EL1, NS-EL2, S-EL1, S-EL2 and EL3. I think it is worthwhile to see how this can be optimized to avoid saving and restoring register states that will not be used, but I would like to see a solution that does not leak data between execution environments that are supposed to be isolated.

TF-A works on a 6 month release cadence and we typically update the gcc toolchain to the latest released version along with the TF-A release. gcc versions we use are downloaded from https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/downloads.
Currently we don't update the toolchain to any versions newer than the one released here. We used the version 10.3-2021.07 with TF-A v2.6 release and will be updating to version 11.2-2022.02 with our upcoming v2.7 release which is planned for the end of May, 2022.

When you have a S-EL2 based system with possibly multiple S-EL1 partitions, it would be complex to implement a scheme where in EL3 will restore the right S-EL1 context on taking a trap during lazy save mechanism. In such systems, it is easier for S-EL2 to implement such a scheme since it is the manager for S-EL1 contexts.

Hey Olivier,

A similar issue was reported in the ticket https://developer.trustedfirmware.org/T984
Can you confirm if this issue is due to a bug in the toolchain itself?

Reported bug in gcc 12.1.0

• https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105523

The issue is related to this GCC ticket:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99578
Which is corrected in the official GCC 12.1 version.

Hi Peter,
So far this has been a deliberate design choice to avoid saving/restoring SVE state from EL3 mainly for BL31 footprint reasons (and performance if unconditionally done on each and every world switch). The vector register file ranges from 2KB to 8KB with 8 cores, and linearly scales to as many cores in the system (which can be hundreds in a server chipset). Apart from specific cases under discussion (SPM-MM or EL3 FF-A SPM), it is preferable to do this at lower EL e.g. a TOS at SEL1 (or Hafnium at SEL2). Do you have specific reasons why it cannot be done at lower EL?
You may also want to take a look at those options: ENABLE_SVE_FOR_NS and ENABLE_SVE_FOR_SWD
https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/docs/getting_started/build-options.rst#n409
Regards,
Olivier.

Hi Yuezhiran,
We are following this up internally. Could you please let us know what revision of Helios you have and whether you run linux? (Linux doesn't work around any Helios errata currently)

As I understand it from the white paper (v1.6) from developer.arm.com there is research ongoing for the mitigation sequence for the Cortex- A65. Once known I believe TF-A reference mitigation patches will be developed.

I notice that relevant patches have been merged into the mainline branch except A65. will it be uploaded recently ?

Reference implementations of mitigations in TF-A for vulnerabilities in various CPU's were initially made available for public review in Gerrit. These have now after the opportunity for feedback been merged into the mainline branch.

Hi @yuezhiran, I believe it's valid for this function to return 0. The bl1_plat_handle_post_image_load function is overridable by the platform, but we do not provide a mechanism for the platform to communicate the "acceptable" image IDs to the caller. We cannot therefore return an error if we receive an image ID that has no post-load process, because it may be called for any image (the FWU process does this here).

Ok, thanks a lot.

Have you tried a more recent release? v2.6 is our latest tagged release or try tip of tree?

I am using Avenger96 and Buildroot and ran into similar issue.

Hi
As you mentioned, req_states will be NULL only if pwrlvl is not valid. However, in my opinion, many checks have been done earlier to the invocation of the function to make sure pwrlvl is valid. See lines 246 and 428 in lib/psci/psci_common.c file. Can you tell if you have noticed any issue in your platform with PSCI library?

Thanks a lot.

A code coverage solution has been developed and is being tested before being rolled out in the production CI.

Thanks for your reply. So is there any way to calculate the coverage of TF-A(functions or codes)?

The TF-A project uses the OpenCI project https://www.trustedfirmware.org/projects/open-ci/ to provide "Daily" and "Gerrit patch review" testing through a Jenkins infrastructure system with jobs visible here https://ci.trustedfirmware.org and these make use of tests defined in the TF-A Tests repository and from other reposititories. Please refer to the OpenCI documentation for more information https://tf-ci-users-guide.readthedocs.io/en/latest/

Given that there's no activity to get the submission rule issues in 9990 resolved, I followed CJKay's and jwerner's instructions and made https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/10415, hopefully more ready for inclusion.

FYI I removed the previous SPAM comment.

Thanks, I submitted the following based on CJKay's patch above:
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/9990

The problem has been solved.

Yes, the __sramdata in the declaration is a mistake, the correct target section for that global needs to be .pmusram.data. This used to be in .sram.data once upon a time but then the suspend.c stuff got added and required it to be moved to PMUSRAM. I guess they forgot to update that part in the declaration and since the old GCC seemed to silently prefer the attribute in the definition, nobody noticed. Please apply CJKay's first patch to fix the warning.

I can only assume GCC <11 simply didn't have this warning, but it looks legitimate.

Ok thanks.

It's native aarch64, the command line I'm using is:
make HOSTCC="gcc $RPM_OPT_FLAGS" CROSS_COMPILE="" PLAT=rk3399 bl31

Is this building natively on arm64 host?

I'm using Fedora 34/35 with the distribution toolchain (gcc 11.1 GA)

Can you please provide the link to the gcc11 toolchain you use?
AFAIK latest public Arm release of the cross compiler toolchains is 10.2 (https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-a/downloads)

Hi all, I suggest we move this discussion back to the mailing list so we can track the progression a bit more easily.

Vote: Jens Wiklander (qemu and OP-TEE Dispatcher): disapprove
I too have concerns with adding more tags to the the subject of the commit message.

We have had some contact with some other TF.org projects but was not aware of the Mbed TLS project had its own solution already. Its already acknowledged we should to look for a common solution across projects. This discussion is part of that engaging. We do need to make a decision at some point.

Hi, I agree with @danh-arm. We should try to come up with a unified guidance, if possible. I think we will have a better idea of the path forward, once most of the maintainers respond. Personally, I would be open to merging both approaches to get the best of both worlds.

I think tags in the subject of the commit message are problematic in general. Since commits are expected to capture "logical unit of change", the granularity of tags and commit scope often mismatches. While not mandated in the "Contributors Guide" sections, some commits indeed use tags already (i.e. libc, spmd, docs, etc..). What if a "logical unit of change" affects multiple components (i.e. a fix affects both libc and spmd)?

Hi, I just want to make you aware that the Mbed TLS project has a different, home-rolled solution to this problem:
https://github.com/ARMmbed/mbedtls/blob/development/ChangeLog.d/00README.md

Yes, I have found that 50 chars is not enough to describe commits many times. Adding a tag would mean sacrificing space. But I am open to adding this tag to the commit message instead of the header. We already plan to add tags to the message anyways?

I think this is generally a good move. The transition period will be ugly and unavoidable but we should reap the benefits of it long term. I do agree with Varun's concern about using tags in the headers but it is a trade-off we will have to make for the benefits standard commits offers.
Varun, is the format <conventional commit tag>:<plat> or <submodule>:<short message> not sufficient? I dont think we want to pack too much into the header any way in terms of what the commit is doing.

Vote: <Varun Wadekar> (<Tegra>): <disapprove>

Hi,
A developer submitted a patch which closely relates to this issue. Please give your feedback here: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/7314

Hi,
Please use personal email and git configs/identity to push the patch to trustedfirmware.org. Few developers from the tf-a community have done this. Here is a link to contribution guidelines:

Thanks for your reply. It is difficult to submit code to community or Gerrit using company's account and system. So, I wonder whether it's permitted to upload by providing modifications or “git send-email” with patches.

Hi,
Yeah, looks like such a large system with numerous clusters will run into an issue with the current limitation of "lock_index" being "unsigned char". Please go ahead and submit a patch for review:

Yes, The power domain topology of the system can be described as 8 sockets, each socket with 10 clusters and each cluster with 8 cores. We just consider a system like this might face the constraint mentioned above, and we must modify the declaration of lock_index to solve this problem. Feature, more and more interconnected system, with numerous cpus, might face the same problem.

Hi,
The struct "non_cpu_pwr_domain_node" represents a non-leaf power domain in an SoC such as a logical cluster. As you mentioned, the lock_index declaration with "unsigned char" limits such non-cpu power domains to 64. I believe it doesn't restrict the number of CPUs in a system to be <= 64. Do you currently face an issue with PSCI implementation due to the above constraint? Can you advise what is the power domain topology of your system? I think the lock_index can be expanded using "unsigned int" if needed.