Page MenuHomePhabricator

Build can fail on systems with stack protection enabled by default
Closed, ResolvedPublic

Description

Certain GNU/Linux distributions provide a version of gcc that has stack-smashing protection enabled by default. On these systems, building the ATF can fail with undefined-reference errors from the linker even with the ENABLE_STACK_PROTECTOR build option set to "none".

For example, on Void Linux, building for the Rockchip RK3328 platform with

make CROSS_COMPILE=aarch64-linux-gnu- PLAT=rk3328 bl31

fails with a large number of errors of the form

aarch64-linux-gnu-ld.bfd: ./build/rk3328/release/bl31/ddr_parameter.o: in function `ddr_region_usage_parse':
ddr_parameter.c:(.text.ddr_region_usage_parse+0x10): undefined reference to `__stack_chk_guard'
aarch64-linux-gnu-ld.bfd: ddr_parameter.c:(.text.ddr_region_usage_parse+0x14): undefined reference to `__stack_chk_guard'
aarch64-linux-gnu-ld.bfd: ddr_parameter.c:(.text.ddr_region_usage_parse+0x84): undefined reference to `__stack_chk_fail'

This is because the build system assumes stack protection is disabled unless explicitly requested from the compiler.

Adding "ENABLE_STACK_PROTECTOR=none" to the command line above doesn't solve the problem. Note that regarding this option, the User Guide claims

The default value is set to “none”... “none” disables the stack protection.

but this is untrue; either omitting the option or setting it to "none" merely prevents an additional flag from being passed to the compiler, requesting the default behaviour instead, which may or may not include stack protection.

The solution is to modify the implementation so the build system explicitly requests the compiler not use stack protection when the feature is expected to be disabled. I'll upload a patch that makes this change.

Event Timeline

simonsouth updated the task description. (Show Details)Oct 20 2019, 7:17 PM

Hi Simon,

Thanks for reporting this bug and uploading a patch to fix it.

Out of curiosity, what is the version of the cross-toolchain you're using?

I'm using gcc 9.2.0 and GNU binutils 2.32 for AArch64, from Void Linux's cross-aarch64-linux-gnu-0.32_1 package.

sandrine-bailleux-arm closed this task as Resolved.Dec 19 2019, 12:17 PM

I know this task is closed, but I just ran into this issue with the 0348ee49135c07441d249bce556eba0c8ddd6be2 commit.

I am trying to build ATF for the Rock64 on the board itself, so my command looks like:
make ENABLE_STACK_PROTECTOR=none PLAT=rk3328 bl31

And this results in errors like the ones in the original post. Lots of undefined references.

Any ideas?

Hi,
I was able to compile TF-A for rk3328 platform using the same command and specifying the CROSS_COMPILE as aarch64-linux-gnu-
I use the publicly available toolchain https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-a/downloads .In this case, I specifically used the 8.3 x86_64 Linux hosted cross compiler in my Ubuntu 18.04 (Virtual Box image ).
If I am not wrong, you dont have to use a cross compiler. However, I dont have access to a native Arm machine to test your build command. What compiler did you use?

odeprez added a subscriber: odeprez.Jan 6 2020, 5:01 PM

Hi,

I suspect this problem arises because of building natively (although I agree this should normally work).

Which distribution and/or toolchain version is installed on the board?

Regards,
Olivier.

Thanks guys.

I use the GCC 9.2 aarch64 from the Manjaro ARM repo.
Also tried with CLANG 9.0.

Hi,

I tried on an arm64 machine (using gcc8.3) but did not reproduce the issue.

Can you add V=1 to show verbose build output and check if -fno-stack-protector option is emitted on gcc command line?

Regards,
Olivier.

Does not seem like it does.
PS: We patched it to work with clang as a test as well.

clang -c -x assembler-with-cpp -target aarch64-elf -march=armv8-a -mgeneral-regs-only -mstrict-align -DDEBUG=0 -DENABLE_BACKTRACE=0 -DCOREBOOT=0 -DPLAT_EXTRA_LD_SCRIPT -DPLAT_SKIP_OPTEE_S_EL1_INT_REGISTER -DSKIP_A57_L1_FLUSH_PWR_DWN=0 -DA53_DISABLE_NON_TEMPORAL_HINT=1 -DA57_DISABLE_NON_TEMPORAL_HINT=1 -DWORKAROUND_CVE_2017_5715=0 -DWORKAROUND_CVE_2018_3639=1 -DDYNAMIC_WORKAROUND_CVE_2018_3639=0 -DERRATA_A9_794073=0 -DERRATA_A15_816470=0 -DERRATA_A15_827671=0 -DERRATA_A17_852421=0 -DERRATA_A17_852423=0 -DERRATA_A35_855472=0 -DERRATA_A53_819472=0 -DERRATA_A53_824069=0 -DERRATA_A53_826319=0 -DERRATA_A53_827319=0 -DERRATA_A53_835769=0 -DERRATA_A53_836870=0 -DERRATA_A53_843419=0 -DERRATA_A53_855873=0 -DERRATA_A55_768277=0 -DERRATA_A55_778703=0 -DERRATA_A55_798797=0 -DERRATA_A55_846532=0 -DERRATA_A55_903758=0 -DERRATA_A55_1221012=0 -DERRATA_A57_806969=0 -DERRATA_A57_813419=0 -DERRATA_A57_813420=0 -DERRATA_A57_814670=0 -DERRATA_A57_817169=0 -DERRATA_A57_826974=0 -DERRATA_A57_826977=0 -DERRATA_A57_828024=0 -DERRATA_A57_829520=0 -DERRATA_A57_833471=0 -DERRATA_A57_859972=0 -DERRATA_A72_859971=0 -DERRATA_A73_852427=0 -DERRATA_A73_855423=0 -DERRATA_A75_764081=0 -DERRATA_A75_790748=0 -DERRATA_A76_1073348=0 -DERRATA_A76_1130799=0 -DERRATA_A76_1220197=0 -DERRATA_A76_1257314=0 -DERRATA_A76_1262606=0 -DERRATA_A76_1262888=0 -DERRATA_A76_1275112=0 -DERRATA_A76_1286807=0 -DERRATA_HERCULES_1688305=0 -DERRATA_N1_1043202=1 -DERRATA_N1_1073348=0 -DERRATA_N1_1130799=0 -DERRATA_N1_1165347=0 -DERRATA_N1_1207823=0 -DERRATA_N1_1220197=0 -DERRATA_N1_1257314=0 -DERRATA_N1_1262606=0 -DERRATA_N1_1262888=0 -DERRATA_N1_1275112=0 -DERRATA_N1_1315703=1 -DERRATA_N1_1542419=0 -DERRATA_DSU_798953=0 -DERRATA_DSU_936184=0 -DSTACK_PROTECTOR_ENABLED=0 -DCRASH_REPORTING=0 -DEL3_EXCEPTION_HANDLING=0 -DSDEI_SUPPORT=0 -DARM_ARCH_MAJOR=8 -DARM_ARCH_MINOR=0 -DCOLD_BOOT_SINGLE_CPU=0 -DCTX_INCLUDE_AARCH32_REGS=1 -DCTX_INCLUDE_FPREGS=0 -DCTX_INCLUDE_PAUTH_REGS=0 -DEL3_EXCEPTION_HANDLING=0 -DCTX_INCLUDE_MTE_REGS=0 -DENABLE_AMU=0 -DENABLE_ASSERTIONS=0 -DENABLE_BTI=0 -DENABLE_MPAM_FOR_LOWER_ELS=0 -DENABLE_PAUTH=0 -DENABLE_PIE=0 -DENABLE_PMF=0 -DENABLE_PSCI_STAT=0 -DENABLE_RUNTIME_INSTRUMENTATION=0 -DENABLE_SPE_FOR_LOWER_ELS=1 -DENABLE_SVE_FOR_NS=0 -DERROR_DEPRECATED=0 -DFAULT_INJECTION_SUPPORT=0 -DGICV2_G0_FOR_EL3=0 -DHANDLE_EA_EL3_FIRST=0 -DHW_ASSISTED_COHERENCY=0 -DLOG_LEVEL=20 -DNS_TIMER_SWITCH=0 -DPL011_GENERIC_UART=0 -DPLAT_rk3328 -DPROGRAMMABLE_RESET_ADDRESS=0 -DPSCI_EXTENDED_STATE_ID=0 -DRAS_EXTENSION=0 -DRESET_TO_BL31=0 -DSEPARATE_CODE_AND_RODATA=0 -DRECLAIM_INIT_CODE=0 -DSPD_none -DSPIN_ON_BL1_EXIT=0 -DSPM_MM=0 -DTRUSTED_BOARD_BOOT=0 -DUSE_COHERENT_MEM=1 -DUSE_DEBUGFS=0 -DUSE_ROMLIB=0 -DUSE_TBBR_DEFS=1 -DWARMBOOT_ENABLE_DCACHE_EARLY=0 -DBL2_AT_EL3=0 -DBL2_IN_XIP_MEM=0 -DBL2_INV_DCACHE=1 -DUSE_SPINLOCK_CAS=0 -DAARCH64 -Iinclude -Iinclude/arch/aarch64 -Iinclude/lib/cpus/aarch64 -Iinclude/lib/el3_runtime/aarch64 -Idrivers/arm/gic/common/ -Idrivers/arm/gic/v2/ -Iplat/rockchip/common/ -Iplat/rockchip/common/include/ -Iplat/rockchip/common/aarch64/ -Iplat/rockchip/common/drivers/pmu/ -Iplat/rockchip/common/drivers/parameter/ -Iplat/rockchip/rk3328/ -Iplat/rockchip/rk3328/drivers/pmu/ -Iplat/rockchip/rk3328/drivers/soc/ -Iplat/rockchip/rk3328/include/  -Iinclude/lib/libfdt -Iinclude/lib/libc -Iinclude/lib/libc/aarch64   -nostdinc  -Wall -Wmissing-include-dirs -Wunused -Wdisabled-optimization  -Wvla -Wshadow -Wno-unused-parameter -Wshift-overflow -Wshift-sign-overflow -Wlogical-op-parentheses -Wno-error=deprecated-declarations -march=armv8-a -ffreestanding -Wa,--fatal-warnings -D__ASSEMBLY__ -Wp,-MD,./build/rk3328/release/libc/setjmp.d -MT build/rk3328/release/libc/setjmp.o -MP -c lib/libc/aarch64/setjmp.S -o build/rk3328/release/libc/setjmp.o
ar cr build/rk3328/release/lib/libfdt.a build/rk3328/release/libfdt/fdt.o build/rk3328/release/libfdt/fdt_addresses.o build/rk3328/release/libfdt/fdt_empty_tree.o build/rk3328/release/libfdt/fdt_ro.o build/rk3328/release/libfdt/fdt_rw.o build/rk3328/release/libfdt/fdt_strerror.o build/rk3328/release/libfdt/fdt_sw.o build/rk3328/release/libfdt/fdt_wip.o
ar cr build/rk3328/release/lib/libc.a build/rk3328/release/libc/abort.o build/rk3328/release/libc/assert.o build/rk3328/release/libc/exit.o build/rk3328/release/libc/memchr.o build/rk3328/release/libc/memcmp.o build/rk3328/release/libc/memcpy.o build/rk3328/release/libc/memmove.o build/rk3328/release/libc/memrchr.o build/rk3328/release/libc/memset.o build/rk3328/release/libc/printf.o build/rk3328/release/libc/putchar.o build/rk3328/release/libc/puts.o build/rk3328/release/libc/snprintf.o build/rk3328/release/libc/strchr.o build/rk3328/release/libc/strcmp.o build/rk3328/release/libc/strlcpy.o build/rk3328/release/libc/strlen.o build/rk3328/release/libc/strncmp.o build/rk3328/release/libc/strnlen.o build/rk3328/release/libc/strrchr.o build/rk3328/release/libc/setjmp.o
ld.bfd -o build/rk3328/release/bl31/bl31.elf --fatal-warnings -O1 --gc-sections   -Map=./build/rk3328/release/bl31/bl31.map --script ./build/rk3328/release/bl31/bl31.ld ./build/rk3328/release/bl31/build_message.o ./build/rk3328/release/bl31/gic_common.o ./build/rk3328/release/bl31/gicv2_main.o ./build/rk3328/release/bl31/gicv2_helpers.o ./build/rk3328/release/bl31/plat_gicv2.o ./build/rk3328/release/bl31/rockchip_gicv2.o ./build/rk3328/release/bl31/cci.o ./build/rk3328/release/bl31/delay_timer.o ./build/rk3328/release/bl31/generic_delay_timer.o ./build/rk3328/release/bl31/params_setup.o ./build/rk3328/release/bl31/bl31_plat_setup.o ./build/rk3328/release/bl31/plat_pm.o ./build/rk3328/release/bl31/plat_topology.o ./build/rk3328/release/bl31/platform_common.o ./build/rk3328/release/bl31/pmu.o ./build/rk3328/release/bl31/soc.o ./build/rk3328/release/bl31/bl31_main.o ./build/rk3328/release/bl31/interrupt_mgmt.o ./build/rk3328/release/bl31/bl31_context_mgmt.o ./build/rk3328/release/bl31/runtime_svc.o ./build/rk3328/release/bl31/arm_arch_svc_setup.o ./build/rk3328/release/bl31/std_svc_setup.o ./build/rk3328/release/bl31/cpu_data_array.o ./build/rk3328/release/bl31/context_mgmt.o ./build/rk3328/release/bl31/errata_report.o ./build/rk3328/release/bl31/psci_off.o ./build/rk3328/release/bl31/psci_on.o ./build/rk3328/release/bl31/psci_suspend.o ./build/rk3328/release/bl31/psci_common.o ./build/rk3328/release/bl31/psci_main.o ./build/rk3328/release/bl31/psci_setup.o ./build/rk3328/release/bl31/psci_system_off.o ./build/rk3328/release/bl31/psci_mem_protect.o ./build/rk3328/release/bl31/bakery_lock_coherent.o ./build/rk3328/release/bl31/spe.o ./build/rk3328/release/bl31/bl_common.o ./build/rk3328/release/bl31/tf_log.o ./build/rk3328/release/bl31/multi_console.o ./build/rk3328/release/bl31/plat_bl_common.o ./build/rk3328/release/bl31/plat_log_common.o ./build/rk3328/release/bl31/plat_common.o ./build/rk3328/release/bl31/desc_image_load.o ./build/rk3328/release/bl31/bl_aux_params.o ./build/rk3328/release/bl31/xlat_tables.o ./build/rk3328/release/bl31/xlat_tables_common.o ./build/rk3328/release/bl31/plat_psci_common.o ./build/rk3328/release/bl31/16550_console.o ./build/rk3328/release/bl31/aem_generic.o ./build/rk3328/release/bl31/cortex_a53.o ./build/rk3328/release/bl31/plat_helpers.o ./build/rk3328/release/bl31/pmu_sram_cpus_on.o ./build/rk3328/release/bl31/bl31_entrypoint.o ./build/rk3328/release/bl31/crash_reporting.o ./build/rk3328/release/bl31/ea_delegate.o ./build/rk3328/release/bl31/runtime_exceptions.o ./build/rk3328/release/bl31/dsu_helpers.o ./build/rk3328/release/bl31/platform_mp_stack.o ./build/rk3328/release/bl31/cpu_data.o ./build/rk3328/release/bl31/cpu_helpers.o ./build/rk3328/release/bl31/spinlock.o ./build/rk3328/release/bl31/psci_helpers.o ./build/rk3328/release/bl31/context.o ./build/rk3328/release/bl31/debug.o ./build/rk3328/release/bl31/cache_helpers.o ./build/rk3328/release/bl31/misc_helpers.o ./build/rk3328/release/bl31/platform_helpers.o ./build/rk3328/release/bl31/crash_console_helpers.o -L./build/rk3328/release/lib  -lfdt -lc 
/usr/sbin/ld.bfd: ./build/rk3328/release/bl31/params_setup.o: in function `params_early_setup':
params_setup.c:(.text.params_early_setup+0x4): undefined reference to `__stack_chk_guard'
/usr/sbin/ld.bfd: params_setup.c:(.text.params_early_setup+0x8): undefined reference to `__stack_chk_guard'
/usr/sbin/ld.bfd: params_setup.c:(.text.params_early_setup+0x100): undefined reference to `__stack_chk_guard'
/usr/sbin/ld.bfd: params_setup.c:(.text.params_early_setup+0x108): undefined reference to `__stack_chk_guard'
/usr/sbin/ld.bfd: params_setup.c:(.text.params_early_setup+0x138): undefined reference to `__stack_chk_guard'
/usr/sbin/ld.bfd: ./build/rk3328/release/bl31/params_setup.o:params_setup.c:(.text.params_early_setup+0x140): more undefined references to `__stack_chk_guard' follow
/usr/sbin/ld.bfd: ./build/rk3328/release/bl31/params_setup.o: in function `params_early_setup':
params_setup.c:(.text.params_early_setup+0x15c): undefined reference to `__stack_chk_fail'
/usr/sbin/ld.bfd: ./build/rk3328/release/bl31/psci_off.o: in function `psci_do_cpu_off':
psci_off.c:(.text.psci_do_cpu_off+0x4): undefined reference to `__stack_chk_guard'
/usr/sbin/ld.bfd: psci_off.c:(.text.psci_do_cpu_off+0x8): undefined reference to `__stack_chk_guard'
/usr/sbin/ld.bfd: psci_off.c:(.text.psci_do_cpu_off+0x80): undefined reference to `__stack_chk_guard'
/usr/sbin/ld.bfd: psci_off.c:(.text.psci_do_cpu_off+0x88): undefined reference to `__stack_chk_guard'
/usr/sbin/ld.bfd: psci_off.c:(.text.psci_do_cpu_off+0xa8): undefined reference to `__stack_chk_fail'
/usr/sbin/ld.bfd: ./build/rk3328/release/bl31/psci_suspend.o: in function `psci_cpu_suspend_start':
psci_suspend.c:(.text.psci_cpu_suspend_start+0x4): undefined reference to `__stack_chk_guard'
odeprez added a comment.EditedJan 8 2020, 9:13 AM

Hi,

Unfortunately the option won't show on asm files, but only on C files.

Can you patch lib/stack_protector/stack_protector.mk to show the defined option:

diff --git a/lib/stack_protector/stack_protector.mk b/lib/stack_protector/stack_protector.mk
index b5aba1528..746ca0817 100644
--- a/lib/stack_protector/stack_protector.mk
+++ b/lib/stack_protector/stack_protector.mk
@@ -25,4 +25,6 @@ else
   endif
 endif
 
+$(info ENABLE_STACK_PROTECTOR=$(ENABLE_STACK_PROTECTOR))
+
 $(eval $(call add_define,STACK_PROTECTOR_ENABLED))

Can you try this build command and attach build.log to the ticket:

make PLAT=rk3328 clean
make PLAT=rk3328 V=1 bl31 | tee build.log

Thanks & Regards,
Olivier.

Hi,

I installed manjaro-arm on an rpi3 and cloned TF-A commit 0348ee49135c

I did not reproduce the rk3328 build issue neither with gcc9.2 nor clang 9.

Regards,
Olivier.

Okay, here's the log.

And yes it still failed after the stack_protector.mk patch.

Hi,

The log seems to show additional options versus the regular TF-A build:
-march=armv8-a -O2 -pipe -fstack-protector-strong -fno-plt

This looks to override -fno-stack-protector (and others).

Can you check if those options were added somewhere in your build env.?

Regards,
Olivier.

I am sorry guys.
I need to apologize to all who have helped me with this issue.

It was all caused by a TYPO in my build script, so the CFLAGS did not get unset before running these commands. After I fixed the typo, the arm-trusted-firmware builds fine for both rk3328 and rk3399.

I am really sorry to have wasted all your time. :(

Nerver mind :) Glad if it works now. Cheers, Olivier.