Given that there's no activity to get the submission rule issues in 9990 resolved, I followed CJKay's and jwerner's instructions and made https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/10415, hopefully more ready for inclusion.

Still happens with TF-A 2.5.

Duplicate of T873.

Note that as part of this activity the following bug in the Qemu porting of the mps2-an521 board has been found and fixed with the following pull request:

Patches available here:

FYI I removed the previous SPAM comment.

As you know, for secure and non-secure side are closely related in TF-M. In latest TF-M v1.3 release, FP support (in secure side either non-secure side) are not official announced, so the problem you reported is not a TF-M bug.

Thank you for letting us know the problem when you are using TF-M, this is helpful for all stakeholder of TF-M.

Hi Feder,
Honestly, I've only faced this issue when doing some advance scheduling manipulations on the Non-Secure side (modifying LSPACT, FPCA, etc.), so not in mainline TF-M/RTOS use-cases. Please disregard it for now.

In T921#10778, @ioannisg wrote:

However, this is only a partial solution. That is because a Secure thread re-scheduling will clear the .FPCA flag, but leave the LSPACT set, meaning that a NS IRQ with FP instructions will trigger again an FP stacking. But this FP stacking will be done on the memory where FPCAR is pointing at, and FPCAR is only updated in exception entry, if .FPCA is set. As a result, it does not seem that you avoid a stack corruption.

Let me know if you've understood this argumentation, or you need more information.

Any updates here, Feder?

However, this is only a partial solution. That is because a Secure thread re-scheduling will clear the .FPCA flag, but leave the LSPACT set, meaning that a NS IRQ with FP instructions will trigger again an FP stacking. But this FP stacking will be done on the memory where FPCAR is pointing at, and FPCAR is only updated in exception entry, if .FPCA is set. As a result, it does not seem that you avoid a stack corruption.

Hi Feder, correct; setting CPACR_S solves the problem of Secure No-Coprocessor Usage Fault.

Please try to add code below just before setting NSACR in tfm_arch_config_extensions() and let me know the result. Thanks!

Following the discussion on the meeting I would update my comment:

https://review.trustedfirmware.org/c/TF-M/trusted-firmware-m/+/10127

Anton, could you please, suggest an assignee for this one? Thanks!

Hi Feder,

I tried on my side with steps above, and I still cannot reproduce your issue.
Please confirm NSACR register is configured correctly, and check the FP related registers before the FP instruction causing crash.

Hi Alamy,

Hi Sherry,

I tried this on my side on STM32L562e_dk board with TF-M repo: enable "-mfloat-abi=softfp" in NS side and use TF-M default "-mfloat-abi=soft" option, and changed FP registers in non-secure thread first, then do PSA call. When system is in secure thread mode, no crash found even FPCCR_S.LSPACT = 1.

Do you have FP instruction in secure thread? If there is no FP instruction, the processor will not stack FP registers even LSPACT flag is set.
I tried this on my side on STM32L562e_dk board with TF-M repo: enable "-mfloat-abi=softfp" in NS side and use TF-M default "-mfloat-abi=soft" option, and changed FP registers in non-secure thread first, then do PSA call. When system is in secure thread mode, no crash found even FPCCR_S.LSPACT = 1.

TF-M currently support "-mfloat-abi=soft" as default, but doesn't support the setting in Zephyr "-mfloat-abi=softfp". Because they are totally different things.

But why? These ABIs are link-compatible, I do not see the problem. GCC allows these binaries to link successfully.

I mean TF-M itself current FP option is "-mfloat-abi=soft", not the "-mfloat-abi=softfp" which is used in Zephyr. It doesn't matter with setting for Zephyr.

TF-M currently support "-mfloat-abi=soft" as default, but doesn't support the setting in Zephyr "-mfloat-abi=softfp". Because they are totally different things.

Thanks, I submitted the following based on CJKay's patch above:
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/9990

This is the explanation for "FP_SOFTABI" in Zephyr.
"config FP_SOFTABI
bool "Floating point Soft ABI"
help

	  This option selects the Floating point ABI in which hardware floating
	  point instructions are generated but soft-float calling conventions."

Yes, it crashes even without Lazy Stacking. It is a bit more deterministic as is. It crashes in the first secure exception entry, after the transition to secure domain from non-secure.

But for Zephyr, are you using soft FP or hardware FP?

The first thing I want to confirm is "I compile zephyr and Tf-M with soft FP". As you know, TF-M is default with soft FP.

1. But for Zephyr, are you using soft FP or hardware FP?
2. Is it possible for you to share the compile options and linker options for the source file including the "Non-Secure interrupt" crashing?
3. Is it possible to show the assembly code for the "Non-Secure interrupt"?
4. Please have a try to disable lazy FP stacking, to see whether still crash.

Hi Feder,

The problem has been solved.

If you didn't change TF-M while integrating into your project, PSA call(handler mode) cannot be interrupted by non-secure interrupt like you mentioned, the reason is non-secure exceptions are de-prioritized (AIRCR.PRIS = 1) in TF-M.
Non-secure interrupt can only be active when system in thread mode.

Hi Alamy,

Merged.

Hi @ioannisg,
FYI. Feder is on holiday and will back to office on 10th May.
Thanks.

Yes, the __sramdata in the declaration is a mistake, the correct target section for that global needs to be .pmusram.data. This used to be in .sram.data once upon a time but then the suspend.c stuff got added and required it to be moved to PMUSRAM. I guess they forgot to update that part in the declaration and since the old GCC seemed to silently prefer the attribute in the definition, nobody noticed. Please apply CJKay's first patch to fix the warning.

I can only assume GCC <11 simply didn't have this warning, but it looks legitimate.

Ok thanks.

It's native aarch64, the command line I'm using is:
make HOSTCC="gcc $RPM_OPT_FLAGS" CROSS_COMPILE="" PLAT=rk3399 bl31

Is this building natively on arm64 host?