What is IPC?#

There are many different ways to communicate between processor cores, including interrupts, mailboxes, IPC protocols, and shared memory regions.

IPC stands for “Inter-Processor Communication”. IPC can be any form of communication between processor cores. The AM64x is a flexible device, and the hardware can support multiple IPC implementations.

Which cores are involved in IPC?#

  • A53

  • R5F

  • M4F

  • PRU cores

What IPC features are supported in hardware?#

For more details about any of the hardware IPC features, refer to the Technical Reference Manual (TRM).

Interrupt#

A signal that can be high or low. An interrupt is typically the fastest way to transmit information from one processor core to another processor core.

Interrupts can be

  • Level sensitive

    • For example, an interrupt service routine, or “ISR”, is triggered when the interrupt line is high

  • Edge sensitive

    • For example, the ISR is triggered when the interrupt line transitions from low to high, or high to low

Mailbox#

Think of a mailbox like an interrupt signal, paired with a 32 bit register. The IPC Notify driver uses mailboxes.

Spinlock#

Spinlocks can be used to coordinate accesses to shared resources. This can help synchronize software running on different processor cores. Spinlocks can be physical hardware peripherals, or pure software implementations. For example, Linux drivers typically do NOT use hardware spinlocks. Instead, most spinlocks that are used by Linux drivers only exist as software constructs.

We do not discuss spinlocks in the rest of the Multicore academy. For more information about using a hardware spinlock from Linux, refer to the Hardware Spinlock Framework.

For more information about using a spinlock from an MCU+ core, refer to the Spinlock driver documentation and the Spinlock example.

Basic IPC concepts#

This is NOT an exhaustive training on IPC. We will briefly mention many different concepts that you can look up elsewhere if you are interested in them. Note that TI can only answer software questions about software that TI wrote; if you decide to implement your own IPC software, we are unable to answer questions about that custom software.

Polling vs interrupts#

There are 2 main methods for a processor core to receive information from another core:

  • Polling

    • The receiving core manually checks for updated information

  • Interrupts

    • The receiving core continues running its own software until it is interrupted (typically by an interrupt or a mailbox)

Some cores combine hardware interrupts with software polling.

For example, PRU cores often must be completely deterministic (i.e., we can guarantee exactly which assembly instruction executes at every single clock cycle). Thus, a PRU core cannot be preempted by an interrupt.

(unless the PRU_ICSSG’s task manager has been configured to be able to preempt certain code)

In such a scenario, a hardware interrupt can be received. However, it is up to the processor core to manually poll the core’s interrupt controller (INTC) to see whether an interrupt has been received, and process any additional information (like which interrupt was triggered).

Polling offers more control over when the receiving core responds to the transmitting core.

Polling a shared memory location typically takes longer than receiving an interrupt, but a simple memory read can be lower latency (and more deterministic) than using an IPC protocol like RPMsg.

Memory#

For more information about memory on the AM64x, refer to Memory basics.

Regions of DDR or on-chip memory can be allocated for:

  • IPC protocols

    • IPC RPMsg

  • Shared memory regions

    • Shared memory regions are directly written by the transmitting core, and read by the receiving core

    • A shared memory region can be as small as a single 32 bit word, or large enough for Kilobytes or Megabytes of data

Latency & throughput#

Many designs have specific requirements for the IPC:

  • Average latency

    • On average, the IPC may need to complete within a specific time

  • Worst-case latency

    • The IPC might need to always complete within a specific time

  • Data throughput

    • A certain amount of data may need to be transmitted between cores within a certain amount of time

Certain kinds of IPC have higher or lower latency, and higher or lower throughput. It is up to you to benchmark the IPC to ensure that it meets your design needs. As a general rule:

  • Interrupts, mailboxes (IPC Notify)

    • Low latency, low data throughput

  • IPC RPMsg

    • Easy implementation, but not optimized for latency or throughput

  • Shared memory regions

    • Higher latency, higher data throughput

Different kinds of IPC are better for different operating systems#

For more information about operating systems (or OSes), refer to Operating systems.

If your design needs the IPC to have low, deterministic latency, we suggest using a Real-Time Operating System (RTOS) - NOT standard Linux. RT Linux has been modified to be more deterministic than regular Linux, but RT Linux is NOT a true RTOS. It is up to you to determine what operating systems your design needs, and what kinds of IPC are needed between cores.

The “cycle time” is the maximum allowable time for a system to receive an input, do some processing, and provide an output. For more information about selecting the right OS and IPC to meet your design’s cycle time, refer to FAQ: How to ensure computations occur within a set cycle time?

For more information about calculating the interrupt response time of RT Linux, refer to [FAQ] Linux: How do I test the real-time performance of an AM3x/AM4x/AM6x SoC?.

For more information about testing and optimizing RT Linux, refer to the AM62x version of the Linux SDK docs: Real-Time Linux on TI SoCs