Add USB NCM #3265
Add USB NCM #3265
Conversation
of three protocols for Ethernet over USB. This implementation in arduino-pico enables an Ethernet connection with a host PC over USB without any extra hardware. Other uses of USB like Serial, Mouse or Keyboard continue to work in parallel. It has been tested on Windows and Linux. MacOS should also work.
|
Very nice work! And thanks for the use case description. Do you know what the delta RAM usage is for this on, say, Also, on quick glance, I think this might have issues under FreeRTOS due to the async context threadsafe. I will dig a bit more to understand how you're using it in place of IRQ callbacks, and how it would interact with the existing network IRQ/packet injection which happens in a separate task. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| NCMEthernet::NCMEthernet(int8_t cs, arduino::SPIClass &spi, int8_t intrpin) { | ||
| (void) cs; | ||
| (void) spi; |
Check warning
Code scanning / CodeQL
Expression has no effect Warning
| NCMEthernet::NCMEthernet(int8_t cs, arduino::SPIClass &spi, int8_t intrpin) { | ||
| (void) cs; | ||
| (void) spi; | ||
| (void) intrpin; |
Check warning
Code scanning / CodeQL
Expression has no effect Warning
| } | ||
|
|
||
| bool NCMEthernet::begin(const uint8_t* mac_address, netif *net) { | ||
| (void) net; |
Check warning
Code scanning / CodeQL
Expression has no effect Warning
|
|
||
| if (data_len > bufsize) { | ||
| // Packet is bigger than buffer - drop the packet | ||
| discardFrame(data_len); |
Check warning
Code scanning / CodeQL
Expression has no effect Warning
| uint16_t readFrame(uint8_t* buffer, uint16_t bufsize); | ||
|
|
||
| void discardFrame(uint16_t ign) { | ||
| (void) ign; |
Check warning
Code scanning / CodeQL
Expression has no effect Warning
|
Looks like you need to define a |
|
I have tested
|
|
|
||
| extern "C" bool tud_network_recv_cb(const uint8_t *src, uint16_t size) __attribute__((weak)); | ||
| extern "C" bool tud_network_recv_cb(const uint8_t *src, uint16_t size) { | ||
| (void) src; |
Check warning
Code scanning / CodeQL
Expression has no effect Warning
| extern "C" bool tud_network_recv_cb(const uint8_t *src, uint16_t size) __attribute__((weak)); | ||
| extern "C" bool tud_network_recv_cb(const uint8_t *src, uint16_t size) { | ||
| (void) src; | ||
| (void) size; |
Check warning
Code scanning / CodeQL
Expression has no effect Warning
|
The ROM's not too big a deal, I'd say, but losing 6KB of RAM for all sketches feels kind of painful. Let me see if there's a simple SDKOverride we can do to lazy allocate the network buffers (looking at the MAP, the FWIW I was able to test the net connection from the Pico to my Ubuntu 22.04 box (for some reason IP forwarding is not quite working, and I don't want to mess with things because I have several office VPN connections and Virtmgr/Docker rules). Just moving the fortune to the GW |
|
I'm getting lots of crashes when a USB serial connection is active and the network interface is line If I don't have a serial monitor/minicom active things are fine, but OTW I get something like the above almost every time. There may be some weird interaction between USB and Ethernet processing (which were never intertwined before) when this is actuve... |
|
Oh, I have not seen that before. I will try to replicate. Maybe the packet handoff is breaking some rules still |
|
I can't seem to find the magic incantation to make a PR against your PR here, but if you look at the last 3 commits to #3266 you'll see where I was able to make the ncd_device not use any RAM (and almost no ROM) when not used by making a dummy sdkoverride, using the original ncd_device.c inside your library folder (so only compiled when used), and removing the precompiled file from libpico. |
|
Wouldn't that cause a dependency headache when upgrading |
It's a very minor one. We have hand modified files from the SDK in If you have a better way, though, of not needing the extra unused 6K of RAM for most users, I'm happy to hear it. But OTW, while I think this is pretty neat. I don't think I could make all other apps lose that space. -edit- Now that I found what seems to be a reasonable way to do this, I might also do the same for MIDI and HID which takes 1/2 KB on every sketch. With RAM prices increasing 300% in the last 30 days every byte counts. 😆 -edit- |
earlephilhower
left a comment
earlephilhower
left a comment
There was a problem hiding this comment.
I get your hesitation on doing the TUSB hackage, but 6K lost on every sketch is a real problem.
I'll take this as-is, with the 6k, and then combine the TUSB weak override for this and the MIDI, HID, and MSD devices in a follow-up.
One request, though, because getting this working is a little complicated. Can you add a small bit of documentation in the docs/ folder with the instructions you've got for Win and Linux? That way there's a way for a user to discover how to set it up without going through a (closed) PR.
2 participants
Overview
NCM (Network Control Model) is the newest of three protocols for Ethernet over USB.
This implementation creates an Ethernet connection with a host PC over USB without any extra hardware.
Other uses of USB like Serial, Mouse or Keyboard continue to work in parallel.
Use case is for smart home devices that live close to a PC or server. NCM gives the reliability of a wired connection without extra hardware like an Ethernet NIC and without the downsides of classic solutions like a custom protocol over serial. Instead, existing software like ESPHome can be used.
Testing
I have tested on Windows and Linux. No driver installation is required. MacOS should also work.
The example sketch provides 1ms ping latency.
Usage
Compile the provided example sketch.
Modify DHCP and logging as desired.
Once running, the Pico shows up as a new network interface on the host PC.
It must be configured before use. Example for Linux:
On Windows, Internet Connection Sharing can be configured: https://superuser.com/a/1899168
That even includes a DHCP server. In my tests, it works with LWIP's built-in DHCP client.
Implementation details
The PR cleanly hooks into existing USB and Ethernet infrastructure provided by
USB.handLwipIntfDev.h. The only global change is ininclude/tusb_config.hto include the NCM implemetation of tinyUSB.The class
NCMEthernet.cppinterfaces with tinyusb by defining the callbackstud_network_recv_cb(),tud_network_init_cb()andtud_network_xmit_cb().Receiving is the hardest part, as it involves passing a packet between the execution context of tinyUSB to Lwip without deadlocks or copying. The proposed solution uses a new
asnc_when_pending_workerto solve this problem. When tinyUSB callstud_network_recv_cb()we cannot call lwip directly. Instead, the packet's pointer is stored, and the worker is set to be pending.Later, the worker calls
LwipIntfDev::_irq()which fetches the packet and sends it to Lwip.The existing interrupt infrastructure in
LwipIntfDevcannot be used, as it expects a physical pin to be toggled.Polling was tested but dismissed due to high latency.
Transmitting packets does not present the same challenges. We simply call tinyUSB directly in
NCMEthernet::sendFrame.