Lightweight TCP/IP stack (lwIP)#
17 min read
This module gives an overview of lwIP, features of lwIP, its integration and configuration on AM26x devices and the MCU_PLUS_SDK.
This module does not discuss all the APIs or all the features and functionalities of the lwIP integration of MCU_PLUS_SDK. Please refer to the lwIP documentation for complete API guide.
Note
Currently, lwIP version 2.1.2 is ported to the MCU_PLUS_SDK. lwIP is freely available under BSD license.
lwIP overview#
lwIP is a widely used open-source independent lightweight implementation of the TCP/IP protocol stack designed for resource-constrained embedded systems. Some advantages of lwIP are discussed below:
- Optimized Resource Utilization: With lwIP, you can save up a lot of memory on RAM and ROM by efficient buffer and data management. lwIP requires limited resources which makes it a good fit for embedded systems. lwIP uses custom tailor-made APIs to reduce memory and processing demands.
- Modularity and Configurability: lwIP's architectural elegance lies in its modular design, affording developers the ability to configure and tailor the stack to precise application requirements. This modularity not only instills adaptability but also streamlines the development process.
- Open Source Excellence and Community Engagement: As an open-source stack, lwIP thrives on a vibrant community ecosystem. This ensures continuous refinement, with an ever-evolving repository of resources, bug fixes, and collaborative insights.
- Seamless Integration Capabilities: lwIP seamlessly integrates into a diverse array of embedded platforms and operating systems, underscoring its versatility. This ease of integration facilitates a frictionless development workflow, a critical attribute for developers seeking a networking solution that seamlessly aligns with their chosen embedded environment.
- Efficient Protocol Implementation: lwIP's prowess in protocol implementation, notably TCP/IP and UDP/IP, reflects a commitment to efficient and robust networking. The optimized handling of protocols ensures enhanced performance and responsiveness, crucial metrics in scenarios where streamlined communication is imperative.
- Scalability for Diverse Applications: The scalability inherent in lwIP enables developers to deploy the stack across a spectrum of applications, from simple Internet of Things (IoT) devices to complex networked systems. This scalability factor underscores lwIP's versatility and adaptability to varying project scopes and intricacies.
- Cross-Architecture Compatibility: lwIP is engineered for seamless cross-architecture compatibility, empowering developers to effortlessly transition their networking code across diverse hardware platforms. This architectural flexibility streamlines the adaptation of applications to varied embedded environments, fostering scalability and interoperability.
- Dynamic Memory Management Precision: lwIP's dynamic memory management framework provides developers with granular control over memory allocation, enabling optimization tailored to dynamic runtime requirements. This level of precision in memory management is pivotal in resource-constrained embedded systems, ensuring judicious and efficient use of available memory.
- Real-Time Application Low Latency Assurance: Its streamlined protocol handling and efficient architecture contribute to the minimization of communication delays. lwIP uses zero-copy buffers which improve the overall performance, the buffer pointers can be passed directly (wrapped in pbufs) to the lower level from the higher level or vice-versa. Since the application is already aware of the internal buffers, it can directly read/write/re-use saving the expense of buffer copy.
- Congestion Control Strategies: lwIP integrates intelligent congestion control mechanisms, particularly for embedded systems navigating dynamic or unpredictable network conditions, thus ensuring a steadfast and dependable communication environment.
lwIP in MCU_PLUS_SDK#
lwIP is delivered in the MCU_PLUS_SDK in the form of lwip-stack library and lwip-contrib library
- lwip-config/ This folder contains lwipopts.h and lwippools.h which control the memory pools and lwIP features that are enabled for the AM26x devices
- lwip-port/ This folder contains files specific to porting lwIP to FreeRTOS, NoRTOS such as sys_arch.c and lwipopts_os.h
- lwip-stack/ This folder contains the core networking stack and essential TCP/IP and UDP/IP protocols
Note
As a part of the latest lwIP migration, the lwip-contrib folder was removed, and the contrib code exists in the lwip-stack/contrib path
Protocols and Applications offered by lwIP#
The lwIP codebase provides some example applications that can be quickly integrated in your project.
Protocols supported#
lwIP provides support for the following protocols:
- IPv4, IPv6
- ICMP - Internet Control Message Protocol
- NDP - Neighbor Discovery Protocol
- MLD - Multicast Listener Discovery
- UDP - User Datagram Protocol
- TCP - Transmission Control Protocol
- IGMP - Internet Group Management Protocol
- ARP - Address Resolution Protocol
- PPPoS - Point-to-Point Protocol over serial (not supported in MCU_PLUS_SDK)
- PPPoE - Point-to-Point Protocol over ethernet
Additional features:#
- HTTP/ HTTPS server
- SNTP Client
- SMTP, SMPTS client
- Ping
- NetBIOS nameserver
- mDNS responder
- MQTT Client
- TFTP server
- DHCP client
- DNS client (incl. mDNS hostname resolver)
- AutoIP/APIPA (Zeroconf)
- SNMP agent (v1, v2c, v3, private MIB support & MIB compiler)
- IP forwarding over multiple network interfaces
- TCP congestion control
- RTT estimation and fast recovery/ fast retransmit
- Optional Berkley like socket API
- Transparent TLS layer (ported to MbedTLS)
- 6LoWPAN (not supported in MCU_PLUS_SDK)
Note
To read about lwIP offering, please refer: lwip/src/include/lwip/opt.h.
The above file is however not used for the configuration of lwIP in MCU_PLUS_SDK. Instead, the below files determine the behavior:
To modify or see the exact feature offering for AM26x devices, please refer source/networking/lwip/lwip-config/am263x/lwipopts.h
To modify or see the memory pool configuration for AM26x devices, please refer source/networking/lwip/lwip-config/am263x/lwippools.h
lwIP Buffer and Memory management#
lwIP has to smartly allocate, re-use, and manage buffers. Since lwIP buffers can get varied sized data such as full sized TCP segments or several small data packets (e.g. ICMP replies). In order to avoid performance degradation, and efficient memory usage, lwIP provides pbufs (packet buffers) and netbufs (network buffers). The size of these packet buffers is determined by the configurations options in MCU_PLUS_SDK:
- lwipopts.h and lwippools.h
- syscfg
pbufs#
The pbufs are lwIP’s internal representation of a packet. The pbuf can allocate the memory to hold packet content dynamically or let the packet data reside in static memory. Pbufs internally are linked in a chain like a linked list. This enables a packet to span over multiple pbufs. Pbufs can be single pbuf or a chain of pbufs
Pbufs can be of majorly 4 types:
Type |
Description |
|---|---|
PBUF_RAM |
Pbuf Data is stored in RAM. lwIP recommends this to be used for Tx mostly. Pbuf Struct and its payload are allocated in a single contagious block of memory. |
PBUF_ROM |
Pbuf Data is stored in ROM. Pbuf Struct and payload are in different blocks of memory unlike in PBUF_RAM. Memory which is dynamic should generally not be attached to PBUF_ROM pbufs and PBUF_REF should be used instead. PBUF_ROM is for constant data payload that doesn’t change |
PBUF_REF |
Pbuf comes from PBUF_POOL. It is much like the PBUF_ROM except the payload can be changes so we copy the pbuf (duplicate) when queued for transmission. |
PBUF_POOL |
Payload and struct are in single block of contagious memory. Payload is in RAM. Should be used for Rx and not recommended for Tx. Payloads can be chained. |
Note
Usage of PBUF_POOL is not recommended for Tx because in case the pbuf pool becomes empty and no free pbufs are available, the communication will break.
When working with Netconn APIs, a wrapper for pbuf, called netbuf, is used. A netbuf can accommodate both, allocated and referenced data. The number of Pbufs and PCBs are controlled from lwipopts.h in lwip-config folder.
The image below shows how different types of Pbufs can be chained.
Note
Since user applications can write to the pbufs or lwIP changes the context of pbufs in TCP code path, Before the buffers are handed to DMA hardware, data cache should be flushed.
The memory and buffer management in lwIP in MCU_PLUS_SDK is done statically. A memory chunk is allocated beforehand to store the packet buffers. Memory is not allocated and freed at run-time dynamically.
Custom Pbufs#
For efficient memory management, Rx side packet handling implements custom Rx Pbufs. In addition to the general pbuf structure described above, this structure uses an additional alivePbufCount to keep track of pbufs used in the stack in the current pbuf chain, a pointer to the original buffer and its length, customPbufArgs which points to the Rx handle having all the queues.
The syscfg in MCU_PLUS_SDK gives an option to enable or disable the Custom Rx buffers and also configure the size. The default is 1536B.
For lwIP applications in the SDK, the Tx packet buffer memory is internally allocated in lwippools.h. Only the DMA Pkt Info structures are allocated via sysCfg, so this number should match the “PktInfoMem Only Count” described in the above item. To increase the Tx packet count, user needs to update the number correspondingly at “PktInfoMem Only Count” and lwippools.h and build the libs. Rx packet buffer memory is completely managed with application sysCfg, this is done by using Rx custom pbuf in lwIP.
lwIP hooks to Enet driver#
For applications using lwIP to communicate with external world, the AM26x devices carry the CPSW. Common Platform Switch (CPSW) is a hardware switch providing ethernet functionality to the AM26x devices. The AM26x devices carry a 3-port Gigabit CPSW3G subsystem. The 3-port gigabit ethernet subsystem supports two external MAC ports and one internal CPPI (communications port programming interface) or Host port. The CPSW has two Ethernet ports (Port 1/Port 2) with selectable MII, RMII, and RGMII interfaces and a single internal Communications Port Programming Interface (CPPI) port (Port 0).
Read more about CPSW here: CPSW (Common Platform Switch)
MCU_PLUS_SDK uses Enet-LLD drivers to access, configure and manage the CPSW hardware. The Enet library contains a translational layer and hooks to lwIP for efficiently integrating the lwIP input, output, callback functionalities.
The lwipif layer can be divided into three parts as discussed below:
lwip2enet layer functions#
This is the translational layer. The structures defined here wrap and internally contain lwIP structures such as netif interfaces. The functions defined here are generally called inside the lwip2lwipif functions. This includes Initialization and de-initialization of Tx and Rx objects, PbufQ to pktInfoQ related functions, Notify Rx and Tx functions, submitting and retrieval of Rx and Tx packets, and other utility functions related to Mac address, mac ports and timers. The source code for the same can be found at mcu_plus_sdk/source/networking/enet/core/lwipif/src/lwip2enet.c
lwip2lwipif layer functions#
The functions defined here are a part of the auto-generated ti_enet_lwipif.c file which is a part of your application code, generated automatically from the syscfg. This contains function that internally use the lwip2enet layer functions. Major functionality includes :
- LWIPIF_LWIP_start, called during the netif open in the application
- LWIPIF_LWIP_init, used as the init function when netif is added in the application
- Inline functions to extract source and destination IPs from the packet
- Functions to get UDP Lite checksum and validate the same
- LWIPIF_LWIP_input, consumes the Rx packets retrieved from the driver. This is then passed to the lwIP stack via the netif input functions
- LWIPIF_LWIP_stop to deinitialize and stop the Enet controller and device
- LWIPIF_LWIP_periodic_polling to periodically poll the status if netif link is up or down
- LWIPIF_LWIP_setNotifyCallbacks to set the Tx and Rx notify callback functions
- LWIPIF_LWIP_send, used as the link output function
- rxPkt and txPkt handler functions
The source code for the same can be found at mcu_plus_sdk/source/networking/enet/core/lwipif/src/lwip2lwipif.c
lwIP execution contexts#
lwIP can mainly execute in two modes, with and without an operating system.
Mainloop mode (No OS mode)#
This is generally used when no OS is running on your system. The incoming packets are directly fed to netif→input function from main loop. Raw APIs are mostly used in this implementation. The received packets are directly delivered from netif→input to lwIP. All lwIP callback functions are called in IRQ context.
API |
Description |
Mode |
|---|---|---|
lwip_init() |
Initialize all modules |
Mainloop |
ip_input() |
input function for packets based on ipv4 or ipv6 |
not called directly and added to netif->input() |
netif_input() |
Forwards a packet for input processing with ethernet_input() or ip_input() based on netif flags |
not called directly and added to netif->input() |
sys_check_timeouts() |
Handle timeouts for NO_SYS==1 |
Mainloop |
ethernet_input() |
Process received ethernet frames |
not called directly and added to netif->input() |
OS Mode#
This mode is used generally when the system is running an OS. In this mode, raw APIs and sequential APIs can be used. Raw APIs can be called only from the TCPIP thread, to call the APIs from outside it, sequential-styled APIs are used. lwIP should never be called from an interrupt as it will cause errors.
API |
Description |
Mode |
|---|---|---|
tcpip_init() |
Initialize tcpip_thread and submodules |
OS |
tcpip_input() |
Pass received packet to tcpip_thread for input processing |
not called directly and added to netif->input() |
tcpip_callback() |
Call a specific function in the thread context of tcpip_thread for easy access synchronization. |
OS |
tcpip_callbackmsg_delete() |
Free a callback message allocated by tcpip_callbackmsg_new() |
OS |
tcpip_callbackmsg_new() |
Allocate a structure for a static callback message and initialize it. |
OS |
tcpip_callbackmsg_trycallback() |
Try to post a callback-message to the tcpip_thread tcpip_mbox. |
OS |
tcpip_callbackmsg_trycallback_fromisr() |
Try to post a callback-message to the tcpip_thread mbox |
OS |
tcpip_try_callback() |
Call a specific function in the thread context of tcpip_thread |
OS |
The operating system emulation layer provides a common interface between the lwIP code and the underlying operating system kernel. lwIP itself being designed to be small and efficient doesn’t directly implement IPC mechanisms. Instead, it provides a flexible architecture that allows it to be integrated into systems using IPC mechanisms. The lwIP OS abstraction layer supports:
- Semaphores
- Mutex
- Mailboxes
Even if the underlying OS does not support them, the OS emulation layer in lwIP emulates the functionality
lwIP API Support#
lwIP uses specialized APIs that reduce the memory and processing demands for enhanced performance. lwIP supports berkeley-like socket APIs and netconn APIs.
Raw API#
Raw APIs are callback style APIs for low-level handling of raw protocol PCBs for protocols. This allows to have a more fine-grained control over network communication. They are based on the native lwIP API. They are used in callback based applications. Raw APIs are used in Mainloop (Non OS) based execution modes as well as with OS. Raw API offers a mechanism for Asynchronous communication.
The Raw API functionality mainly includes:
- API for setting up TCP or UDP connections
- API to send or receive data over TCP or UDP
- API for application polling
- API to close and abort connections
Refer the API guide for Raw APIs here: Raw API
Netconn API#
The lwIP netconn API provides a high-level abstraction for socket programming. They are sequential APIs based on blocking open-read-close paradigm. They are thread safe and to be called from non TCP/IP threads only. Netconn API offers a mechanism for Synchronous communication. Netconn APIs are not recommended to be used in a non OS based environment.
Refer the API guide for Netconn APIs here: Netconn API
BSD like socket API#
These are sequential APIs internally built over the Netconn APIs. These are standard socket API like the ones widely used for socket programming interface in UNIX-like operating systems. BSD socket APIs are intended for reference only and are not recommended being used in applications as it lacks completeness, for example there is no error handling support. Refer the API guide for BSD like sequential Socket APIs here: Socket API
The API functionality for Netconn and BSD socket APIs mainly includes:
- API to create and close sockets
- API to bind a socket to IP address and port
- API to connect to remote host IP and port or accept incoming connection
- API to listen for incoming socket connections
- API to read and write data from and to a socket
Note
Generally, calling lwIP APIs from other threads or ISR is not recommended. Only a few APIs can be called from other threads or an ISR. They are included in the following files:
- api.h
- netbuf.h
- netdb.h
- netifapi.h
- pppapi.h
- sockets.h
- sys.h
When working in a multithreaded environment, core locking and unlocking must be used must be used before and after making API calls. for example, sys_lock_tcpip_core(), sys_unlock_tcpip_core()
Knowledge Check#
References#
- Ethernet and Networking (MCU_PLUS_SDK): MCU_PLUS_SDK Overview of ethernet and networking
- Enet-LLD reference: Enet-LLD MCU_PLUS_SDK reference
- The MCU_PLUS_SDK has several examples for lwIP based on both FreeRTOS and NoRTOS. Please refer the links below to know more about them:
- Link to the official lwiP documentation: lwIP documentation
- Appnote for MbedTLS over lwIP on AM26x devices: MbedTLS over lwIP on AM26x Devices