Update: Moral of the story, make sure OP-TEE OS isn't overriding your GIC configuration done in BL31! It is quite possible then to handle secure interrupts in BL31. I used the handler from ehf.c as a starting point, and had to add plat_ic_end_of_interrupt and platform specific clearing of interrupt registers (wdog).

I wasn't able to have a secure group 1 interrupt routed to the S-EL1 handler in OP-TEE with Affinity Routing Enabled and GICv3. I will not further examine this.

Hello,

I am trying to configure an interrupt for Group 1 Secure, so that it is routed to OP-TEE OS. In the file drivers/arm/gic/v3/gicv3_main.c I located the function void gicv3_driver_init(const gicv3_driver_data_t *plat_driver_data) which I have verified is being called during boot. Is my understanding correct that I can configure an interrupt as Secure Group 1 if I add an entry to this gic3v_driver_data_t? The NXP code instantiates the gicv3_driver_data_t structure like so:

static const interrupt_prop_t g01s_interrupt_props[] = {
	INTR_PROP_DESC(6, GIC_HIGHEST_SEC_PRIORITY, INTR_GROUP1S, GIC_INTR_CFG_LEVEL),
	INTR_PROP_DESC(7, GIC_HIGHEST_SEC_PRIORITY, INTR_GROUP0, GIC_INTR_CFG_LEVEL),
	INTR_PROP_DESC(78, GIC_HIGHEST_SEC_PRIORITY, INTR_GROUP1S, GIC_INTR_CFG_LEVEL),  // <--- My entry here, also tried GROUP0 as well as adding/subtracting 32 from IRQ No.
};

const gicv3_driver_data_t arm_gic_data = {
	.gicd_base = PLAT_GICD_BASE,
	.gicr_base = PLAT_GICR_BASE,
	.interrupt_props = g01s_interrupt_props,
	.interrupt_props_num = ARRAY_SIZE(g01s_interrupt_props),
	.rdistif_num = PLATFORM_CORE_COUNT,
	.rdistif_base_addrs = rdistif_base_addrs,
	.mpidr_to_core_pos = plat_imx_mpidr_to_core_pos,
};

The third entry for interrupt 78 is the entry I added, in the hope it would be routed to OP-TEE. However, when the interrupt is triggered, it is sent to GNU/Linux (which I determined via /proc/interrupts). I would have expected the interrupt to be routed to OP-TEE OS and handled by my interrupt handler there instead.

Is my general approach correct or am I on the wrong path? I have to admit I am not fully understanding the documentation regarding interrupts in TF-A, OP-TEE OS and the Arm GIC specification, so any pointers would be highly appreciated.

Thank you in advance!

Edit:
I have verified my interrupt entry is processed inside gicv3_secure_spis_config_props

According to the source code it should:

• Configure this interrupt as a secure interrupt
• Configure this interrupt as G0 or a G1S interrupt
• Set interrupt configuration
• Set the priority of this interrupt
• Target SPIs to the primary CPU
• Enable this interrupt

Therefore I would assume this watchdog interrupt should *not* be passed to GNU/Linux. It is disabled in the device tree, too. Here is the output from GNU/Linux:

# cat /proc/interrupts | grep wdog
 59:          0          0          0          0  GPC-PSCI  78 Edge      30280000.wdog
# [  194.865972] watchdog0: pretimeout event
# cat /proc/interrupts | grep wdog
 59:          1          0          0          0  GPC-PSCI  78 Edge      30280000.wdog

Edit 2: The first issue seems to be that while TF-A states "GIC only needs to be initialized once in EL3 etc.", OP-TEE actually performs its own GIC initialization and as far as I can tell so far even overrides configurations done by TF-A BL31, which I find personally quite confusing and not expected. However, I am still investigating this issue in detail so take this with a grain of salt.

OP-TEE seems to override any interrupt configuration done within BL31 (awesome!), so after calling

itr_add(...)
itr_enable(...)

with the proper interrupt number (which is offset by 32 in OP-TEE for imx8m), the interrupt is finally no longer received in GNU/Linux.

Now my problem is I have no idea where it is actually trapped and handled, my OP-TEE interrupt handler is not called, because the nintr handler void main_fiq(void) is not called (which calls gic_it_handle(&gic_data);). Interestingly, I assumed the interrupt is somehow stuck in TF-A EL3, but following runtime_exceptions.S:

/* ---------------------------------------------------------------------
	 * This macro handles FIQ or IRQ interrupts i.e. EL3, S-EL1 and NS
	 * interrupts.
	 * ---------------------------------------------------------------------
	 */
	.macro	handle_interrupt_exception label
	bl	save_gp_registers
	/* Save the EL3 system registers needed to return from this exception */
	mrs	x0, spsr_el3
	mrs	x1, elr_el3
	stp	x0, x1, [sp, #CTX_EL3STATE_OFFSET + CTX_SPSR_EL3]

	/* Switch to the runtime stack i.e. SP_EL0 */
	ldr	x2, [sp, #CTX_EL3STATE_OFFSET + CTX_RUNTIME_SP]
	mov	x20, sp
	msr	spsel, #0
	mov	sp, x2

	/*
	 * Find out whether this is a valid interrupt type.
	 * If the interrupt controller reports a spurious interrupt then return
	 * to where we came from.
	 */
	bl	plat_ic_get_pending_interrupt_type
	cmp	x0, #INTR_TYPE_INVAL
	b.eq	interrupt_exit_\label

I added a NOTICE() to plat_ic_get_pending_interrupt_type which should be called for every single IRQ/FIQ trapped to BL31 and I never get any output (i.e., it is never called?). SCR_EL3.FIQ/IRQ bit must be set, then the exception is handled here!

Any pointers would be highly appreciated.

Non-Secure: SCR_EL3.IRQ==0, SCR_EL3.FIQ==1
Secure: SEC_EL3.IRQ==0, SCR_EL3.FIQ==0

optee SPD registers an interrupt handler for S_EL1_GROUP1 interrupts. However, the targeted interrupt is delivered as Group0 EL3 interrupt, for which TF-A registered no handler. I can register my own interrupt handler for Group0 EL3 interrupts. O.K. so one could argue I simply must configure this properly myself and not rely on the OP-TEE OS/SPD/BL31 combination to work natively for OP_TEE OS itr_add and itr_enable. However, I tried this, and my interrupt handler is registered, I have verified its address is correct, but it never returns and the code does not appear to be executed:

	/* Call the interrupt type handler */
  bl hook_called_interrupt
	blr	x21
  bl hook_called_interrupt

As you can see here, I "hooked" the BL31 interrupt handler routine in runtime_exceptions.S for EL3 interrupts, and I only ever see the first hook_called_interrupt being called in my console, but my interrupt handler never returns and shows no output. Examples for my interrupt handler include:

NOTICE("My interrupt handler \n");
panic()

or

	uint32_t intr_raw;
	unsigned intr = 0;
	uint16_t val = 0;

	assert(id == INTR_ID_UNAVAILABLE);

	intr_raw = plat_ic_acknowledge_interrupt();
	intr = plat_ic_get_interrupt_id(intr_raw);
	if (intr == INTR_ID_UNAVAILABLE)
		return 0;

    /* Clear WDOG1 interrupt */
    // ....

    return (uint64_t)handle;

So the interrupt handler is presumably entered, but then its just stuck and not a single instruction appears to be executed.