The hype: “Rust is safer than C++”

At that point in my early ESP-RS experimentation, I want to address an important point about stability.  And to clarify, by “ESP-RS”, I mean the “std” mode built around the esp-idf-sys/svc/hal crates.

One of the arguments for Rust over C++ is a stronger memory management with clear memory ownership. Allegedly that's supposed to translate into better stability, avoiding the typical C/C++ crashes when memory references are improperly used.

My experience with Tangram rgen supported that.

The experience with ESP-RS here however does not match expectations. One little mishap in an ESP-IDF call… sneeze and the entire framework crashes.

For example, trying to initialize the EspMQTT client before the EspWifi has connected didn't just fail -- it rebooted the entire ESP32.

The reality is that ESP-RS “std” with the esp-idf-sys/svc/hal crates is essentially a ton of unsafe wrappers directly around the C ESP-IDF stack. And the ESP-IDF is generally quite touchy. For all its promises of type and memory safety, ESP-RS doesn't save us from that.

It’s all ESP-IDF and FreeRTOS

OTOH, I find the ESP-IDF to be generally predictable. Once things work, they do without surprise. I don’t have a problem with APIs blowing up predictably during development and giving me a chance to notice newbie mistakes early and taking care of them upfront.

Since I discovered the ESP32 platform 7 years ago, I preferred it over the Arduino platform. I’ve experimented with the “raw” ESP-IDF toolchain yet I rarely use it directly -- instead it’s been a backbone for something else like the Arduino platform, or the MicroPython/CircuitPython platform, and in the current case for the ESP-RS platform. Still, I’m more familiar with the ESP-IDF modules and don’t mind the mix and match, especially knowing that I can just directly call into ESP-IDF or FreeRTOS functions at any time.

Case in point in this case, we have FreeRTOS tasks for std::threads, ESP Wifi, ESP MQTT, etc., and supporting the ESP32 Camera is done directly as an ESP-IDF component right there in the ESP-RS project. It’s all working quite well. Still, we need to remind ourselves that for all that hype about how Rust is so magically superior to C/C++ (hype that I do not really buy into, in case it’s not already obvious), ESP-RS is just a bunch of fancy and sometimes way overkill wrappers around a C API with multi-core multi-thread complexity to deal with.

A Comparison Matrix

Here’s my guide to select the proper toolchain for my own projects:

• If performance does not matter, MicroPython is the easiest way to go (or CircuitPython on any modern Adafruit board).
• If performance and space matters, C++20 with the ESP-IDF 5 toolchain is the proper way to go.
• If performance and ease of programming matters, Rust with ESP-RS in “std” mode with the esp-idf-sys/svc/hal crates is a trade off, at the expense of a larger memory footprint.

The build size for a basic “blinky” project using ESP-IDF-HAL is around 700 kB (dev mode) on an ESP32-CAM. Add ESP Wifi and ESP MQTT, and that project’s build goes up to a whopping 1500 kB. That’s half the 3 MB flash partition available on an ESP32-CAM.

Some notes on how to perform basic Tasks, Threads, Synchronization functions using Rust with ESP-RS. And to clarify, by “ESP-RS”, I mean the “std” mode built around the esp-idf-sys/svc/hal crates.

This isn’t meant to be used as a canonical guide. Most of these are just a quick reminder for myself when I need something so that I don’t need to re-read the docs every time. So basically that only covers stuff I know I would need. I’m not pretending to be exhaustive here. And since I’m a newbie in Rust, I’m not pretending to be right either. Do your own due diligence by reading the official docs.

Tasks & Threads

The canonical example is in this ThreadSpawnConfiguration example:

https://github.com/esp-rs/esp-idf-hal/issues/228#issuecomment-1492483018

I haven’t seen any more official examples of that stuff.

The implementation is here:

https://github.com/esp-rs/esp-idf-hal/blob/master/src/task.rs 

This feels like a wrapper around the ESP-pthread API:

https://docs.espressif.com/projects/esp-idf/en/v4.2.2/esp32/api-reference/system/esp_pthread.html 

esp_pthread_set_cfg sets a “global” config that indicates how subsequent pthread_create() calls behave. That’s what ThreadSpawnConfiguration does.

Usage example:

ThreadSpawnConfiguration {
   name: Some("thread_name\0".as_bytes()),
   stack_size: 4096,
   priority: 15,
   pin_to_core: Some(core::Core1),
   ..Default::default()
}
.set()
.unwrap();

let thread_handle = std::thread::Builder::new()
.stack_size(stack_size)
.spawn(move || {
   //do stuff
})
.unwrap();
thread_handle.join().unwrap();

Since I’ve installed Rust for my exploratory ESP32 project twice already on different machines, I decided to actually write down my install instructions. There are a couple tricks I need to remember.

These steps are for MSYS2 on Windows. I used similar steps on my other box where I use my old fashioned Cygwin. Part of these instructions also work with PowerShell if you hate yourself that much.

My default shell is the “purple” MSYS2 MSYS one (the other shells are MinGW UCRT/Clang x86/x64; I can never remember why I would care about the difference so don’t ask me).

I already have Rust installed on this machine. I likely used “Rustup” following the default Rust install instructions.

I always customize ~/.bash_aliases, where I already added a line that sets up Rust and reminds me it’s available:

if [[ -d "$USERPROFILE/.cargo/bin" ]]; then
   RUST_PATH=$(cygpath "$USERPROFILE/.cargo/bin")
   export PATH="$PATH:$RUST_PATH"
   echo "Rust Cargo: installed."
fi

So first let’s check we have Rust and which version:

$ rustup -V
rustup 1.28.2 (e4f3ad6f8 2025-04-28)
info: The currently active `rustc` version is `rustc 1.86.0 (05f9846f8 2025-03-31)`

$ rustup show
Default host: x86_64-pc-windows-msvc
rustup home:  C:\Users\$USER\.rustup

Of note: there is no “esp” toolchain installed (yet).

OK now we need to install a bunch of stuff:

I looked at using Rust a few years ago on ESP32, but quite frankly the platform did not seem mature enough. It was really bleeding edge. Like razor sharp. I can barely stand the “better-than-thou” hype around Rust, so that was too much. Then I checked again last year when I did the SDB project and it still wasn’t looking stable enough for my needs.

This time it looks like I could at least do what I want: use Rust with ESP-RS on the ESP32-CAM and actually capture image frames. There’s now some good ESP Camera component circulating around with some tangible support:

• https://github.com/espressif/esp32-camera: An Espressif ESP32 Camera module with an OV2640 driver like the ESP32-CAM does have.
• https://github.com/jlocash/esp-camera-rs: The “original” example of how to wrap the esp-camera component into a Rust embedded project.
• https://github.com/jlocash/esp-camera-rs: A fork of that previous project.
• https://github.com/MathiasPius/esp-camera-rs: Yet another fork.
• https://github.com/Kezii/esp32cam_rs: A real example using the esp32-camera IDF component above, with an “espcam.rs” wrapper around it, and a wifi query.

To use this in an ESP-RS project, we need 3 changes:

First, we need a git checkout of the https://github.com/espressif/esp32-camera project.
In a serious project, that would be a git submodule, but for a throw-away test that can be just good ol’ boring git clone:

$ mkdir components
$ pushd components
$ git clone https://github.com/espressif/esp32-camera.git

In the main Cargo.toml, we need the esp-idf-sys crate and we need to use (import?) that esp32-camera component:

$ cargo add esp-idf-sys
$ vim Cargo.toml
[[package.metadata.esp-idf-sys.extra_components]]
component_dirs = "components/esp32-camera"
bindings_header = "components/bindings.h"
bindings_module = "camera"

Now the first I compiled this, it failed at runtime in the esp_camera::init:

I (657) cam_hal: Allocating 384000 Byte frame buffer in PSRAM
E (657) cam_hal: cam_dma_config(509): frame buffer malloc failed

That sounds like “esp_psram” is not enabled on the project. That’s done in the sdkconfig.defaults file:

CONFIG_ESP32_SPIRAM_SUPPORT=y

My own “webcam sample” for the ESP32-CAM is located here:

https://github.com/ralfoide/arduino/tree/main/ESP32-CAM/Rust/esp-rs-std-webcam

At Randall, I’ll soon have the Distant Signal panel installed. This connects to the local wifi and gets the turnout state from the local MQTT broker. I want to track the “health” of that wifi connection, as that has been an issue in the past. The simplest way is to reuse my existing Google Analytics dashboards, thus I want the CircuitPython script on the ESP32 to send pings to GA4.

The goal is to measure the “uptime” of the display. The display is working when it is able to receive messages from the MQTT server. It can receive them when it can connect to the wifi. Thus we want to track the wifi state, or some kind of proxy for it.

Since this is CircuitPython, we have AdaFruit libraries to already deal with JSON and network.

So really “all” I need is something similar to the Analytics class in Conductor 2:

• Queue stats to be sent.
• Stats are sent as JSON payloads.
• POSTs can fail, in which case they need to be retried with a backoff delay.
• ⇒ Obviously a “wifi lost” event would have no wifi to send the stat.
• Do I need to put a real date/timestamp in the events?
• ⇒ These ESP32 don’t have a “date” clock, so it’s nice if I don’t have to.
• There’s an NTP library which I’ve used on the LitterTimer experiment but it makes the logic more complicated since we need to expect that the device may not have wifi.

So what do we need to do custom pings to Google Analytics 4?

• A “GA4_CLIENT_ID”, typically configured in settings.toml
• A “GA4_MEASUREMENT_ID”, also configured in settings.toml
• A “GA4_API_SECRET”, also configured in settings.toml
• The feature is disabled if either of these settings is missing.
• Placing them in setting.toml avoids leaking these in any GIT source, and it makes it easier to configure a new panel.
• See below on where to find these.
• A base URL:
• https://www.google-analytics.com/debug/mp/collect    -- debugging version
• https://www.google-analytics.com/mp/collect          -- real pings version
• The POST URL is:
• %(base)s?api_secret=%(GA4_API_SECRET)s&measurement_id=%(GA4_MEASUREMENT_ID)s
• POST mime type: text/plain
• Payload is JSON, and is used as such in Conductor:

{
        'client_id': GA4_CLIENT_ID,
        'events': [ {
                'name': event_action ,
                'params': {
                        'items': [ ] ,
                        'value' : value_int ,
                        'currency' : USD
                }
        } ]
}

• The AdaFruit “requests” library is the simplest way to send a POST request. It accepts either a JSON (using a Python dictionary) or a string payload.
• When POSTing to the debug URL, print the response.status_code and the response.content.decode() to get valuable information on why requests fail.
• When POSTing to the real ping version, the GA server replies with 204 (Success w/ No Content) and a complete lack of response body.

Well, that escalated quickly.

I’ve been battling with this in Distant Signal:

@@ MQTT: Failed with  pystack exhausted
Traceback (most recent call last):
 File "code.py", line 498, in <module>
 File "code.py", line 354, in _mqtt_loop
 File "adafruit_minimqtt/adafruit_minimqtt.py", line 956, in loop
 File "adafruit_minimqtt/adafruit_minimqtt.py", line 1027, in _wait_for_msg
 File "adafruit_minimqtt/adafruit_minimqtt.py", line 381, in _handle_on_message
 File "code.py", line 328, in _mqtt_on_message
 File "script_loader.py", line 27, in newScript
 File "script_parser.py", line 316, in parseJson
 File "script_parser.py", line 289, in _parseGroup
 File "script_parser.py", line 214, in _parseInstructions
 File "script_parser.py", line 278, in _parseInstructions
 File "adafruit_display_text/label.py", line 88, in __init__
 File "adafruit_display_text/__init__.py", line 273, in __init__
 File "adafruit_display_text/__init__.py", line 294, in _get_ascent_descent
RuntimeError: pystack exhausted

It turns out that CircuitPython is built with a max stack depth of 15-16 calls (*), at least for this version of the MatrixPortal S3.

It’s entirely possible to rebuild it with a different stack depth, if one is inclined to do so.

The stack size is controlled by CIRCUITPY_PYSTACK_SIZE which is defined globally here:
https://github.com/adafruit/circuitpython/blob/HEAD/py/circuitpy_mpconfig.h#L455
and is also architecture dependent -- e.g. the SAM arch overrides the default.

Documentation on CIRCUITPY_PYSTACK_SIZE is here:
https://github.com/adafruit/circuitpython/blob/HEAD/docs/environment.rst#circuitpy_pystack_size 

(*) As can be seen above, the “stack size” is not really a number of calls but really a size in bytes. Thus I expect calls with more arguments will exhaust the stack faster, or whatever else needs to be pushed on the stack for each call.

At runtime, the “ustack” module can give the current max and used size:
https://docs.circuitpython.org/en/latest/shared-bindings/ustack/index.html 

Well, not on the MatrixPortal S3 port at least.

So how does one solve that? Since I do not want to rebuild CircuitPython, it means I can just amend the code to reduce stack usage.

For example the main starts with this:

if __name__ == "__main__":
   setup()
   loop()

This conveniently mimics an Arduino sketch setup. But we don’t need that “loop()” function. Just place everything under the “if main” and save one call stack entry.

The MQTT error above happened when a message was received -- I’d parse the script right there in the MQTT handler. Instead, what I do is save the script in a temporary global variable, and process it from the main loop, after the MQTT handler has returned. That way we’re “saving” about 5 calls depth on the stack.

The parser is about 5 calls deep. I could probably save one call in there, if I really have to.

Another easy strategy is to create lambdas, and queue them for processing in the main loop.

That’s more or less what I do with the delayed processing of the script, except since there’s only one thing to do, I keep the string reference around rather than wrap the processing in a lambda. Delayed processing can quickly become hard to follow, it’s harder to understand and debug.

Over there in the trains/electronics blog, I have a little write up about the HUB75 Protocol for LED Matrix Displays, as used in the Distant Signal project. Go read it there.

I’m still in the process of updating RTAC to work properly on Android 13.

RTAC uses a number of APIs which have been deprecated since Android 10. Since it was running on 2017 hardware with Android 9, that wasn’t much of a problem. Now I want to move to newer hardware that runs Android 13 so I need to deal with it.

WifiLock:

• The WifiManager.WifiLock API is still present and usable.
• I’ve added a handler class for it in TCM.
• However it’s not absolutely clear what it really provides.

WifiManager#enableNetwork is reserved to “Device Owners” and system apps.

• It seems that ConnectivityManager#requestNetwork could be an adequate replacement?
• A first try of the ConnectivityManager#requestNetwork API wasn’t conclusive.
• The app suddenly displayed a model dialog “connecting to device” with a spinner, which has the undesirable side-effect of pausing the main TCM activity.
• Configuring the request for a known SSID did… nothing. It stayed on that spinner and did not switch to that wifi network.
• Configuring the request for a known SSID and its WPA password did something weird where the wifi list showed a second entry of the SSID “for this particular app”.
• So far that didn’t work as expected. My goal is to switch the wifi to an already known SSID that has already been configured in the Android wifi setting and already set up with password et al.

To be continued.

Somehow the same wishes always circle back:

“Aww, wouldn't it be cool if <insert random unfinished project name> had a built-in editor and I could code the app on my {phone, PDA} whilst I'm using it?”

A few decades ago that's how the Hint project started, which quickly got nowhere -- mostly because I was trying to get everywhere at the same time.

And of course later I realized the goal is its own anti-goal -- it's already hard to finish one project, so adding another project inside that project pretty much guarantees that neither gets anywhere, fast.

Another month, another problem… I’ll spare you 2 days of internal discussion and I’ll just summarize it in one sentence:

I pretty much have decided to move all my open source repositories from BitBucket to GitHub.

Historically I had all my code on Google Code. When that service shut down, a lot of folks went to GitHub and I had my reservations about that service. Instead I chose BitBucket. I still have some philosophical issues with GitHub, but right now I have even more of them with BitBucket so… a lengthy migration is in order.